JPH0371838B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a single-chip color imaging device using a CCD or the like.

A single CCD is used as a solid-state image sensor, and its spatial sampling form is, for example, first and third.
In some cases, a spatial sampling format is selected in which frequency components corresponding to a primary color signal are sampled by a checkerboard grid, and frequency components corresponding to a second primary color signal are sampled by a rectangular grid. . .

FIG. 1 shows an example of the spatial sampling mode, in which the first and third primary color signals, for example, red and blue signal components R and B, are sampled in a checkerboard pattern with respect to the second primary color signal, for example, green signal component G. If the interlacing of the standard television system is not considered, the real space sampling points of each primary color signal R to B are as shown in Figure 2, with an x mark for the green signal component G, and an x mark for the green signal component G; For the red and blue signal components R and B, a checkered pattern such as ◯ and □ marks is formed, respectively.

In the figure, x is the horizontal pixel spacing, Px
is the pixel arrangement pitch, and x/Px on the horizontal axis indicates the horizontal sampling period when normalized by Px. Similarly, y is the interval between picture elements in the vertical direction, Py is the arrangement pitch of picture elements, and y/Py on the vertical axis is the vertical sampling period normalized by Py.

When FIG. 2 is expressed in frequency space, that is, in the reciprocal space of FIG. 2, it becomes as shown in FIGS. 3A and 3B. In this figure, Px·U/2π and Py·V/2π are normalized frequencies in the horizontal and vertical directions, and U and V are angular frequencies in the horizontal and vertical directions. FIG. 3A shows the sampling carrier of the green signal component G in FIG.
FIG. 3B represents the sampling carrier of the red and blue signal components R, B. In this case, the carriers of the red and blue signal components R and B are out of phase with each other, but exist at the same point on the spatial coordinates.

For the plurality of carriers shown in FIG. 3, when each primary color signal component is equal, that is, R=
When G=B, the carrier components of each primary color signal component, such as (1, n), (2, n), etc., are all canceled out. Also, if interlacing is taken into account, the carriers at the point (m, 1/2) are all canceled and disappear. If R=B, considering only the frequency domain in the figure, (1/2, 1/4) and (1/
The R and B carriers existing at point 2, 3/4) are canceled out, and the R, G, and B carriers and their sideband components do not exist at all in the frequency range required as the imaging output. No aliasing distortion occurs due to band components. Note that m and n are arbitrary real numbers.

However, in the case of R≠B, the R and B carriers and their sideband components existing at the point (1/2, 1/4) remain without being completely canceled, resulting in aliasing distortion. do. Therefore, if the input image has horizontal stripes, sideband components such as a→' shown in FIG. 4 (a→ is its baseband component) remain,
If the stripes are diagonal, sideband components such as b→' will remain, so in order to remove the aliasing to the horizontal frequency region, especially the aliasing to the high frequency region, it is necessary to remove the aliasing as shown by diagonal lines. It is sufficient to remove signal components in the dimensional frequency domain. The frequency range to be removed is isotropic around the carrier.

On the other hand, the frequency range that satisfies the Nyquist condition of the carrier when picture elements are arranged in a checkered pattern, that is, the sampling area that maximizes sampling efficiency, is a parallel hexagonal lattice shown by the broken line.
Even if the frequency range indicated by diagonal lines is removed, the sampling efficiency does not deteriorate much.

Therefore, the signal processing system for the imaging output may be configured to pass the frequency components within this parallel hexagonal lattice and remove the frequency region that affects the image quality due to aliasing.

Next, an example of a color imaging device according to the present invention having such a signal processing system will be explained in detail with reference to FIG.

The figure shows the case where digital processing technology is used, and the R~ obtained from the image sensor CCD10
The point sequential signal in B is for white balance adjustment.
The digital color signal is supplied to the A-D converter 30 through the AGC circuit 20 and converted into a digital signal, and this digital color signal is processed by the processor 40 as a white clip,
After processing such as correction, the signal is separated into a digital green signal G and a digital red and blue signal R/B by a switch SW.

The digital red and blue signals R and B are supplied to a non-separable two-dimensional digital filter 50 and given a predetermined transfer characteristic to limit a predetermined frequency range, and the output thereof is sent to a Y/C forming circuit 60. Then, luminance signal components Y _R , Y _B and color signal components C _R , _CB are formed. The limitation of the frequency domain and the means for forming the luminance signal component and color signal component will be described later.

Now, the color signal components C _R and C _B are modulated by the digital encoder 70 to form a carrier color signal, which is combined with the luminance signal components Y _R and Y _B by the combiner 80, and then D-A A converter 90 converts it into an analog signal.

On the other hand, the digital green signal G is also subjected to similar signal processing. Reference numeral 100 denotes a two-dimensional digital filter, which can match the transmission characteristics of this system to those of digital red and blue signals R and B. 110 is Y/
C forming circuit, 120 is a digital encoder, 13
0 is a synthesizer, and 140 is a DA converter. The analog output is combined with the R and B analog outputs in a synthesizer 150 to produce a final imaging output (color video signal).

Now, two steps to limit the frequency range mentioned above.
The characteristics of the dimensional digital filter 50, 100 are chosen as follows. In other words, the transfer function H(z, w) of the signal processing system for filtering the carrier and its sideband components generated by sampling the two-dimensional imaging space is as follows: H(z, w) = a ₀₀ + 〓〓 ^pq {a _2p,2q+1 z ^2p w ^2q+1 +a _2p+1,2q z ^2p+1 w ^2q } ...(1) Here, z ^-1 , w ^-1 : horizontal, vertical unit delay operator {a _n,o }: Impulse response p, q of the filtering processing system: given by an integer from −∞ to +∞. The meaning of the above transfer function is
When interpolating the pixel data of interest with surrounding pixel data, if the pixel data of interest is real pixel data, the term after Σ becomes 0, and the actual pixel data of interest itself is output, and the pixel data of interest is real. In other words, if the pixel is not to be interpolated, a value interpolated using arbitrary existing pixel data in the surrounding area will be output. The simplest example of this general transfer function as one of the embodiments of the present invention, that is, an example in which interpolation is performed using four pixel data around a pixel of interest will be described below. Therefore, the transfer function H(z, w) is H(z, w)=A{1+1/4(z ^-1 +z+w ^-1 + w)}...(2) Here, A is the normalization of the power of the primary color signal. If we choose a series such as when A = 1/2...G, 1...R, B, the frequency domain at that time will be as shown in Figure 6. That is, by transforming the above two equations, H (Z, W) = A {1 + 1/4 (Z ^-1 + Z + W ^-1 + W)} = A {1 + 1/4 (e ^-jUPx +e ^jUPx +e ^-jVPy +e ^jVPy )
} =A{1+1/2(cosUPx+cosVPy)}, and the two-dimensional points where this transfer function H(Z, W) becomes 0 and 1 can be easily found as shown in FIG. Furthermore, a line where the output of this transfer function H(Z, W) is 1/2 can also be easily found. The contour lines in the figure are all lines indicating the cutoff frequency (lines where the normalized frequency is 1/2), and the numbers inside the brackets indicate the output level after filtering. Therefore, if a contour line for the cutoff frequency that surrounds a predetermined frequency range including the carrier generated by sampling the checkered grid in this way is given, the signal in the frequency range indicated by diagonal lines in FIG. components can be removed.

Figure 7 shows the 2
This is an example of a dimensional digital filter 50, 100, and in this example, a non-divisible digital filter that cannot be expressed as a product of a horizontal direction transfer function and a vertical transfer function is used. That is, these digital filters make interpolated pixel data exist by interpolating missing pixel data through checkerboard sampling, and when the spatial sampling frequency is substantially increased, unnecessary carrier components are removed. The absence of unnecessary carrier components means that aliasing distortion will not occur. For this purpose, both of these digital filters 50 and 100 synchronize the information of adjacent picture elements on the same line and the information of picture elements on two lines adjacent to that line to form one picture element information. The digital filter 50 is configured as an average value interpolation circuit, and will be explained starting with one digital filter 50.

This has four delay elements 51 to 54 connected in series as shown in the figure, and each of the delay elements 51 and 52 has a delay time of one bit (one picture element). The remaining delay elements 53 and 54 each have a delay time of (1H-1 bit). Therefore, the synthesizer 55 synthesizes the input and output of the delay element 53, the synthesizer 56 synthesizes the input and output of the delay element 54, and these synthesized outputs are further synthesized by the synthesizer 57, and then the level shift circuit 58 synthesizes the input and output of the delay element 53. If the level is lowered to 1/8, the pixel portion (indicated by the broken line) that originally has no information will be interpolated using the four pixel data shown in FIG. The output of the level shift circuit 58 is supplied together with the output of the delay element 51 to a demultiplexer 59 to form a synchronized R and B signal.

Now, the transfer function H(zw) of the digital filter 50 configured in this way is expressed by equation (2), and its frequency characteristic H(jwPx, jwPy) is H(jwPx, jwPy) if the group delay term is omitted. = 1 + 1/2 (cos WP As a result, the frequency range indicated by diagonal lines in FIG. 4 is limited and output.

Since there is no carrier at the (1/2, 1/4) point for the digital green signal G, there is no need to limit the frequency in the same way as for the digital red and green signals R and B. However, it is preferable for the transfer characteristics of the red and blue signals R and B to be equal to the transfer characteristics of the green signal G in order to remove false color signals.

Therefore, in this example, a digital filter 100 having the same configuration as the above-described filter is provided. 10
1,102 is a 1-bit delay element, 103,10
4 is a (1H-1 bit) delay element, 105~1
07 is a synthesizer. However, since the green signal is not a point sequential signal, a demultiplexer for making the red signal R and the green signal B simultaneous is not necessary. Instead, level shift circuits 108A, 108B and a combiner 109 are provided, and the transfer function of the system up to the combiner 109 is made equal to that of the red and blue signal systems.

The band-limited digital primary color signals R, G, and B are supplied to Y/C forming circuits 60 and 110, respectively, to form luminance signal components and chrominance signal components necessary as an imaging output.

That is, the digital primary color signals R, B, and G are supplied to low-pass filters 61, 62, and 111, and the horizontal band is limited as shown in FIGS. 9 and 10 (the cutoff frequency is f _c ). Later, in the weighting circuits 63, 64, 112, for example,
NTSC luminance signal component ratio (G=0.59, R=
0.30, B=0.11). Note that the cutoff frequency f _c is approximately 800 to 900 KHz.

On the other hand, subtracters 65, 66, and 113 extract the horizontal high-frequency components R _H , B H , and G _H of the primary color signals R, B _, and G, and these are weighted so as to satisfy the carrier balance condition. The circuits 67, 68, and 114 weight G=0.50 and R=B=0.25. These high frequency components R _H , B _H , GH and the weighted low frequency components R _L , B _L , GL that have passed through the filters 61, 62, ₁₁₁ are combined into _combiners 69R, 69B,
115, the luminance signal components Y _R , Y _B ,
Y _G ′ is formed. That is, Y _R = 0.30R _L + 0.25R _H Y _B = 0.11B _L + 0.25B _H Y _G ′ = 0.59G _L + 0.50G _H …(4) Components C _R , C _B for forming color signals , C _G are the outputs R _L , B _L of the low-pass filters 61, 62, 111
_GL itself is used.

Note that when the digital filters 50 and 100 are used, a predetermined frequency region centered on (1/2, 1/4) is removed, so the resolution particularly for input images in an oblique direction deteriorates. Therefore, in this embodiment, a green signal G having no carrier at the point (1/2, 1/4) is used, and the signal component in the omitted frequency domain is compensated for by this green signal G.

Therefore, level shift circuits 108A, 108
The output of B is provided to subtractor 160. Frequency characteristics of the signal processing system up to this subtracter 160
H'(jwPx, jwPy) is H'(jwPx, jwPy) = 1-1/2(cos wPx + cos wPy)...(5) Therefore, this signal processing system has bandpass characteristics, and the diagonal lines in Figure 4 indicate that The frequency region shown becomes the passband (see FIG. 10). Therefore, this subtractor 1
If the output G _M of 60 is appropriately weighted in the weighting circuit 170 and then combined with the luminance signal component Y _G ', the high frequency components of the red and blue signals R and B can be compensated with this output α G _M. Therefore, the final brightness signal component Y _G is Y _G =0.59G _L +0.50G _H +αG _M (6), and the brightness signal Y is Y = Y _R +Y _G +Y _B (7). Note that 180 indicates a synthesizer.

As explained above, according to the present invention, it is possible to limit a predetermined frequency range including carriers generated by checkerboard sampling using a two-dimensional digital filter. It is possible to reliably remove the frequency component that causes this. Furthermore, according to the present invention, it is possible to compensate for the missing frequency components of the red and blue signals with the green signal, so it is possible to compensate for the deterioration in resolution of the input image, such as diagonal striped patterns. Furthermore, in the present invention, since the transfer functions for the three primary color signals are made the same, color false signals can be suppressed.

In addition, in the above-mentioned embodiment, a predetermined signal including carriers generated by a checkered pixel arrangement, especially carriers generated at points (2n+1/2, 2m+1/4) (n, m=0, 1, 2...) In order to remove the frequency region of
Any transfer function other than the transfer function expressed by equation (2) may be used as long as it can filter the frequency domain as described above.

The sampling form of the imaging space is not limited to that shown in FIG.

[Brief explanation of drawings]

Figure 1 is a configuration diagram showing an example of a CCD as an image sensor, Figures 2 and 3 are diagrams showing the spatial sampling points in Figure 1 and their carriers in frequency space, and Figure 4 is a diagram showing the filtering operation. Explanatory drawings, FIG. 5 is a system diagram of an example of a color imaging device according to the present invention, FIG. 7 is a system diagram showing an example of a two-dimensional digital filter, and FIGS. 6, 8, 9, and 10. are diagrams for explaining the respective operations. 10 is a CCD, 50, 100 is a two-dimensional digital filter, 60, 110 is a Y/C forming circuit, 51
-54, 101-104 are delay elements.

Claims (1)

[Claims] 1. In a color imaging device using a solid-state image sensor in which at least one primary color component is sampled in a checkerboard grid, the signal of the primary color component sampled in the checkerboard grid is supplied to a two-dimensional filter, and the signal of the primary color component sampled in the checkerboard grid is At the same time, the two-dimensional filter is configured to suppress components in the frequency domain including sampling carriers, and the two-dimensional filter also suppresses the pixel data of interest that is missing due to the checkerboard sampling, two pixel data adjacent in the horizontal direction, and two pixel data adjacent in the vertical direction. An interpolation filter is formed by interpolating the pixel data of interest using peripheral pixel data of the pixel of interest including at least two adjacent pixel data, and the frequency domain is substantially the same as the frequency domain where the baseband component exists. A color imaging device characterized by non-overlapping frequency regions.