WO1997024882A2

WO1997024882A2 - Chrominance interpolation

Info

Publication number: WO1997024882A2
Application number: PCT/IB1996/001364
Authority: WO
Inventors: Hendrik Dijkstra
Original assignee: Philips Electronics N.V.; Philips Norden Ab
Priority date: 1995-12-27
Filing date: 1996-12-05
Publication date: 1997-07-10
Also published as: WO1997024882A3

Abstract

In a method of chrominance interpolation to obtain at least one interpolated chrominance sample (c2, c3, c4) between first (c1) and second (c5) existing chrominance samples, a corresponding relative luminance step is determined for a chrominance sample (c2, c3, c4) to be interpolated, the first (c1) and second (c5) existing chrominance samples are subtracted to obtain a chrominance step, and the chrominance samples (c2, c3, c4) to be interpoled are calculated by adding to the first existing chrominance sample (c1), a product of the chrominance step and the corresponding relative luminance step. The method is advantageously used in multimedia terminals.

Description

Chrominance interpolation.

The invention relates to a metiiod and apparatus for chrominance interpolation.

EP-A-0,619,675 discloses a color image display system, in which encoded color image data stored in a first format such as YUV or YIQ, is converted to RGB format for output to a color display screen. RAM storage arrays contain lookup tables for product functions defmed in associated conversion equations. Each product function is a product of multiplication of a predetermined constant and data representing a U or V component of source image pixel. An embodiment features use of a single RAM to convert YUV data derived by chrominance subsampling. In such data, luminance of individual pixels is represented by discrete Y components, and chrominance of pairs of consecutive pixels is represented by a single U and V component for each pair.

When a chrominance interpolation is carried out by a simple repetition operation or by a simple linear interpolation using known interpolation filters, false colors may appear in the output image when consecutive luminance pixels have mutually different values.

It is, inter alia, an object of the invention to reduce the number of occurrences of false colors. To this end, a first aspect of the invention provides a method as defined in claim 1. A second aspect provides a chrominance interpolation filter as defined in claim 6. A third aspect provides a display apparatus as defined in claim 7. Advantageous embodiments are defined in the dependent claims. In a method of chrominance interpolation to obtain at least one inteφolated chrominance sample between first and second existing chrominance samples in accordance with the present invention, a corresponding luminance gradient is determined for a chrominance sample to be inteφolated, and the chrominance samples to be inteφolated are calculated in dependence upon at least one existing chrominance sample and the corresponding luminance gradient.

It appeared that the number of false colors was substantially reduced when this luminance-contour following chrominance inteφolation method was used. The method is advantageously used in multimedia terminals, graphics display systems, and in image display systems like TV receivers.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

In the drawings:

Fig. 1 illustrates a 4:4:4 sampling scheme;

Fig. 2 illustrates a 4:2:2 sampling scheme;

Fig. 3 illustrates a 4: 1: 1 sampling scheme;

Fig. 4 illustrates a 4:2:0 sampling scheme; and Fig. 5 shows a block diagram of a display apparatus comprising a chrominance inteφolation filter in accordance with the present invention.

Fig. 1 illustrates the sampling positions for a so-called 4:4:4 sampling, which is mainly used for RGB, but can also be used for YUV. At each pixel a sample is taken for Y (or R), for U (or G), and for V (or B). All three components have the same spatial resolution and bandwidth. In Figs. 1, 2 and 3, the first line shows line 1 (Ll) of field 1 (FI), and the last line shows line 3 (L3) of field 2 (F2). The squares show at which positions chrominance U, V is available, while the dots show at which positions luminance Y is available.

Fig. 2 represents a more effective sampling format, in which Y samples are measured at each pixel position, and U and V samples are measured only at every second pixel position. By that the color information has horizontally a resolution, that is half of that of luminance. The human eye does not perceive chrominance with the same clarity as luminance, therefore this type of data reduction causes very little visual loss of content. The 4:2:2 scheme reduces the data bandwidth need by a third.

Fig. 3 is an example of 4: 1: 1 sampling, often used in consumer type video products. The achievable color bandwidth in this case is only about one third of that of luminance. But in broadcasted video or in tape-recorded video, there is normally not more chroma bandwidth supported or available.

The 4:2:0 sampling scheme illustrated by Fig. 4 is used generally for MPEG and H-261 compression standards. The overall data bandwidth of 4:2:0 sampling is identical to that of 4: 1: 1 sampling. In the example in Fig. 4, a non-interlaced video source is represented.

Obviously, the subsampled chrominance information shown in Figs. 2-4 has to be upconverted to full resolution as shown in Fig. 1 before it can be displayed. The present invention can be used in this upconversion to reduce the number of false colors resulting from illegal combinations of Y, U and V, i.e. combinations which do not correspond to existing colors in the RGB domain. The sampling patterns of Figs. 2-4 are only given by way of example; the invention can also be used in chrominance inteφolations starting from non-interlaced 4:2:2, non-interlaced 4: 1 : 1, and interlaced 4:2:0 sampling schemes. In alternatives to the sampling scheme of Fig. 4, the chrominance sample el l coincides with the luminance sample yl l, or a U sample ul l (not shown) coincides with the luminance sample yl l while a V sample v22 (not shown) coincides with the luminance sample y22. The present invention can also be used in a zoom function operating in the YUV domain, to reduce the number of false colors resulting from the inteφolated values. Especially when shaφ transitions occur in the luminance signal, as often happens with pictures made by computer graphics techniques, illegal combinations of luminance Y and chrominance U, V may result from an inteφolation of the chrominance values. The invention proposes a chrominance inteφolation which follows a contour in the luminance signal as good as possible. In the following, simple chrominance samples, values and signals are used as a shorthand notation to indicate pairs of U,V chrominance samples, values and signals. Of course, this does not exclude application of the chrominance inteφolation method of the present invention on a sampling scheme as described in EP-A-0,619,675 in which the successive luminance samples are alternatingly accompanied by a U sample or a V sample.

Starting from the sampling scheme of Fig. 3, it appears that three chrominance values c2, c3, c4 have to be inteφolated between two existing chrominance values cl , c5, while all luminance values yl, y2, y3, y4, y5 are available. In one embodiment, the following inteφolation algorithm is used: dy : = y5 - yi; /* determine luminance step */ if ! dy | > threshl then dc : : = c5 - cl; /* determine chrominance step */

ydifi : = (y2 - yl) / dy; /* determine relative Y step */ if j ydif2 | < thresh2 then c2 : = cl + dc*ydif2 /* follow luminance contour */ else c2 : = (3*cl + c5) /4 /* default linear interpolation */ endif

ydifi : = (y3 - yl) / dy; I* determine relative Y step */ if i ydiO i < thresh2 then c3 : = cl + dc*ydif3 /* follow luminance contour */ else c3 : = (2*cl + 2*c5) /4 /* default linear interpolation */ endif

ydif4 : = (y4 - yl) / dy; /* determine relative Y step */ if | ydif4 | < thresh2 then c4 : = cl + dc*ydif4 /* follow luminance contour */ else c4 : = (cl -I- 3*c5) /4 I* default linear interpolation */ endif

else c2 : = (3*cl + l*c5) /4 /* default linear interpolation */ c3 : = (2*cl + 2*c5) I c4 : = (l*cl + 3*c5) /4 endif

In plain English, a luminance step dy is determined from first and fifth luminance samples yl , y5 which correspond to first and second existing chrominance samples cl, c5. If the absolute value of the luminance step dy exceeds a threshold threshl , a chrominance step dc between the existing chrominance samples cl, c5 is determined. Also, relative luminance steps ydif2, ydifi, ydif4 are calculated by dividing respective differences y2-yl , y3-yl , y4-yl between respective luminance samples y2, y3, y4 corresponding to the chrominance samples c2, c3, c4 to be inteφolated, and the first luminance sample yl , thru the luminance step dy. If a relative luminance step ydifi, ydifi, ydif4 falls below a predetermined maximum value thresh2, the corresponding chrominance sample c2, c3, c4 to be inteφolated is obtained by adding to the first existing chrominance sample cl, the product of the chrominance step dc and the relative luminance step ydif2, ydif3, ydif4. If the absolute value of the luminance step dy does not exceed the threshold threshl, all chrominance samples c2, c3, c4 to be inteφolated are obtained by means of a straightforward linear inteφolation on the basis of the first and second existing chrominance samples cl, c5. If any of the relative luminance steps ydif2, ydifi, ydif4 exceeds the predetermined maximum value thresh2, the corresponding chrominance samples c2, c3, c4 to be inteφolated is obtained by means of the straightforward linear inteφolation.

In an alternative algorithm, it is determined whether the maximum of the absolute values of ydifi, ydifi and ydif4 exceeds the second threshold thresh2, and the default linear inteφolation algorithm is used when thresh2 is exceeded by the maximum of the absolute values of ydifi, ydifi and ydif4. As a precautionary measure, the calculated chrominance values can be clipped between 0 and 255. Preferably, the first threshold threshl equals 5, while the second threshold thresh2 equals 2. In another alternative embodiment, the luminance-contour following chrominance inteφolations of c2, c3 and c4 are only used when ydifi, ydifi and ydif4, respectively, exceed a threshold -1. The divisions and multiplications may be carried out by means of look-up tables, which may operate on a reduced number of bits. The default operation may be a polynomial operation or a chrominance sample repetition, or more generally, any suitable chrominance inteφolation which is independent from the luminance signal. Instead of using threshold values threshl and thresh2 to switch between the luminance-contour following inteφolation and the default inteφolation, it is possible to evaluate the RGB values resulting from the luminance-contour following inteφolation, and to fall back to the default inteφolation if a valid conversion from YUV to RGB values is not possible.

A very simple algorithm in accordance with the invention can be summarized as follows. If the difference between y2 and yl is smaller than a threshold, then c2 is made equal to cl. In the other case, if the difference between y2 and y5 is smaller than a threshold, then c2 is made equal to c5. Otherwise, c2 is made equal to a weighted average of cl and c5. Similar rules apply for c3 and c4. In this manner, the inteφolated chrominance values are obtained in a very straightforward manner from at least one existing chrominance value and from a corresponding luminance gradient.

Instead of a hard switch between a luminance-contour following inteφolation and the default inteφolation, a soft switch is conceivable; such a soft switch may reduce the number of visible artifacts even further. The following algorithm employs such a soft switch:

threshO threshl thresh2 thresh3

a_dy = abs(y[5]-y[l]); /* absolute value of luminance step */ s_dy = sign(y[5]-y[l]); /* sign of luminance step; the sign is +1 or -1 */

for i = 2 to 4 do a_dy[i] = abs(y[i]-y[l]); /* absolute value of luminance gradient */ s_dy[i] = sign(y[i]-y[l]); /* sign of luminance gradient */ a_ydif[i] = a_dy[i] / a_dy; /* absolute value of relative luminance step */

if (a_ydif[i] > threshO) or (a_dy < threshl) then k = i/4 /* 100% default linear interpolation */ else if (a_ydif[i] > thresh2) or (a_dy < thresh3) then /* 50% linear + 50% luminance-contour following */ k = 1/2 * i/4 + 1/2 * s_dy * s_dy[i] * a_ydif[i] else /* 00% luminance-contour following interpolation */ k = s_dy * s_dy[i] * a_ydif[i]; c[i] = c[l] + k * (c[5]-c[l]); end

In plain English, if the relative luminance step exceeds a threshold 3, or if the luminance step is smaller than 5, the default inteφolation is used. If the relative luminance step exceeds a threshold 2, or if the luminance step is smaller than 3, the default inteφolation and the luminance-contour following inteφolation are mixed. Otherwise, a 100% contour-following inteφolation algorithm is used.

Starting from the 4:2:2 sampling scheme of Fig. 2, only one chrominance value c2 needs to be inteφolated between two existing chrominance values cl, c3. The following algorithm is used:

dy : = y3 - yl; /* determine luminance step */ if i dy J > threshl then dc : = c3 - cl; /* determine chrominance step */

ydif : = (y2- yl) / dy; /* determine relative Y step */ if j ydif| < thresh2 then c2 : = cl + dc*ydif /* follow luminance contour */ else c2 : = (cl + c3) 12 /* default linear interpolation */ endif

else c2 : = (cl + c3) /2 /* default linear interpolation */ endif

In plain English, a luminance step dy is determined from first and third luminance samples y 1 , y3 which correspond to first and second existing chrominance samples cl , c3. If the absolute value of the luminance step dy exceeds a threshold threshl , the chrominance step dc between the existing chrominance samples cl, c3 is determined. Also, a relative luminance step ydif is calculated by dividing the difference y2-yl between a luminance sample y2 corresponding to the chrominance sample c2 to be inteφolated, and the first luminance sample yl , thru the luminance step dy. If this relative luminance step ydif falls below a predetermined maximum value thresh2, the chrominance sample c2 to be inteφolated is obtained by adding to the first existing chrominance sample cl , the product of the chrominance step dc and the relative luminance step ydif. If the absolute value of the luminance step dy does not exceed the threshold threshl, or if the relative luminance step ydif exceeds the predetermined maximum value thresh2, the chrominance sample c2 to be inteφolated is obtained by means of a straightforward linear inteφolation on the basis of the first and second existing chrominance samples cl, c3.

Starting from the 4:2:0 sampling scheme of Fig. 4, the missing value for cl2 can easily be calculated from cl i, cl3 and yl l , yl2 and yl3, in a manner similar to that described with reference to the 4:2:2 sampling scheme. The missing value for c21 can easily be calculated from el l, c31 and yl l, y21 and y31 , also in a similar manner. For c22, it could be tried to calculate it on the basis of el l, c33, and yl l, y22 and y33. If that would not be possible without falling back to the default linear inteφolation, it is possible to calculate c22 on the basis of cl3, c31 , and yl3, y22 and y31, where the default linear inteφolation is used only if also this second attempt to calculate c22 in a luminance-contour following manner were unsuccessful.

Fig. 5 shows a block diagram of a display apparatus comprising a chrominance inteφolation filter in accordance with the present invention for use with the 4: 1: 1 sampling scheme of Fig. 3. In Fig. 5, a luminance signal Y is applied to a chain of delay circuits 1, 3, 5 and 7 for supplying horizontally adjacent luminance samples yl , y2, y3, y4 and y5. A chrominance signal C is applied to a delay circuit 11 for supplying horizontally adjacent chrominance samples cl and c5. The supplied luminance and chrominance samples are applied to a calculation circuit 9 which operates in accordance with the algorithm set out above. The calculation circuit 9 also receives the thresholds threshl and thresh2, and calculates the missing chrominance samples c2, c3 and c4. The calculated chrominance samples c2, c3 and c4 are inserted between the input chrominance samples cl and c5 by an insertion circuit 13. A conversion circuit 15 determines RGB values on the basis of the luminance values supplied by the delay circuit 7 and chrominance values supplied by the insertion circuit 13. The RGB values from the conversion circuit 13 are displayed on a display device 17.

In an alternative embodiment of a chrominance inteφolation filter in accordance with the present invention, YUV data in accordance with the 4: 1: 1, 4:2:2 or 4:2:0 sampling scheme is stored in a first memory. The data required for the inteφolation are read from the first memory and applied to the calculation circuit 9. The output data from the calculation circuit 9 is combined with the input data, and the resulting 4:4:4 sampling scheme data is stored into a second memory.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer.

Claims

Claims:

1. A method of chrominance inteφolation to obtain at least one inteφolated chrominance sample (c2, c3, c4) between first (cl) and second (c5) existing chrominance samples, the method comprising the steps of: determimng for a chrominance sample (c2, c3, c4) to be inteφolated, a corresponding luminance gradient; and calculating said chrominance sample (c2, c3, c4) to be inteφolated in dependence upon at least one existing chrominance sample (cl, c5) and said corresponding luminance gradient.

2. A method as claimed in claim 1, wherein said calculating step includes the steps of: subtracting said first (cl) and second (c5) existing chrominance samples to obtain a chrominance step; and calculating said chrominance sample (c2, c3, c4) to be inteφolated by adding to said first existing chrominance sample (cl), a product of said chrominance step and a corresponding relative luminance step.

3. A method as claimed in claim 2, wherein said relative luminance step is obtained by the steps of: calculating a first difference between a luminance sample (y2) corresponding to a chrominance sample (c2) to be inteφolated, and a first luminance sample (yl) corresponding to said first existing chrominance sample (cl); calculating a second difference between a luminance sample (y5) corresponding to said second existing chrominance sample (c5), and said first luminance sample (yl); and dividing said first difference by said second difference to obtain said relative luminance step.

4. A method as claimed in claim 1, comprising the further steps of: calculating a difference between a luminance sample (y5) corresponding to said second existing chrominance sample (c5), and a first luminance sample (yl) corresponding to said first existing chrominance sample (cl); and calculating said chrominance sample (c2) to be inteφolated from said at least one existing chrominance sample (cl, c5) in accordance with an inteφolation method independent from a luminance signal when said difference falls below a predetermined threshold value.

5. A method as claimed in claim 2, comprising the further step of: calculating said chrominance sample (c2) to be inteφolated from said at least one existing chrominance sample (cl, c5) in accordance with an inteφolation method independent from a luminance signal when said corresponding relative luminance step exceeds a predetermined maximum value.

6. A chrominance inteφolation filter to obtain at least one inteφolated chrominance sample (c2, c3, c4) between first (cl) and second (c5) existing chrominance samples, the filter comprising: means (9) for determining for a chrominance sample (c2, c3, c4) to be inteφolated, a corresponding luminance gradient; and means (9) for calculating said chrominance sample (c2, c3, c4) to be inteφolated in dependence upon at least one existing chrominance sample (cl, c5) and said corresponding luminance gradient.

7. A display apparatus, comprising: means for receiving a luminance signal (Y) and a subsampled chrominance signal (C); a chrominance inteφolation filter (9) as defined in claim 6, to obtain an inteφolated chrominance signal from said subsampled chrominance signal (C); means (15) for converting said luminance signal (Y) and said inteφolated chrominance signal into RGB signals; and a display (17) for displaying said RGB signals.