CN101729884B - Image acquiring device and image preprocessing method - Google Patents

Abstract

The invention discloses an image capture device and an applied image pre-processing method. The technique includes: receiving image data, including luminance data and chrominance data; combining chrominance data of adjacent lines to generate chrominance combined data; using a buffer group to temporarily store the above-mentioned luminance data and chrominance combined data; and The chroma combined data is obtained from the buffer group, and the corresponding chroma combined data is shared by adjacent rows to obtain processed chroma data. The processed chrominance data and luminance data output by the buffer group are used to display an image on a screen.

Description

Image capture device and image preprocessing method

technical field

The present invention relates to an image capture device and an image pre-processing method and device, in particular to a YUV image pre-processing method and device.

Background technique

In the field of digital image processing, a common encoding method is YUV encoding. The image data obtained by the YUV encoding technology includes: luminance data Y (Luminance, Luma), and chrominance data U and V (Chrominance, Chroma). U is bluishness data (another common notation is C _b ). V is reddish color data (another common label is C _r ). YUV encoding follows the following formula to include data for the three primary colors of red, green and blue (RGB):

Y＝0.299*R+0.587*G+0.114*B;

U=0.436*(B-Y)/(1-0.114); and

V=0.615*(R-Y)/(1-0.299).

Since human eyes are less sensitive to chroma (U and V) than luminance (Y), a lower sampling frequency can be used for chroma (U and V). A common method is 4:2:2 chroma sampling.

FIG. 1 illustrates 4:2:2 chroma sampling with a frame 100 of image data. Unlike luminance data, which must be sampled for each pixel, 4:2:2 chroma sampling halves the sampling frequency of chroma data (U and V), so that every two pixels share a set of chroma data (U and V) . Referring to Fig. 1, the pixel 102 of the 0th row (row) and the 0th column (column) only has sampling of luminance data Y ₀₀ and bluishness data U ₀₀ , but no sampling of reddishness data (V); The pixel 104 in the row and column 1 only has samples of luminance data Y ₀₁ and reddish chroma data V ₀₁ , but no sample of bluish chroma data (U). Pixels 102 and 104 share chrominance data U ₀₀ and V ₀₁ .

The above-mentioned image data (YUV) can be converted into a data stream (streaming) through an image compression operation. Image compression operations usually need to divide a frame into multiple sub-images, and perform operations on each sub-image. Conventional technology usually uses a buffer memory, such as a line buffer, to collect the image data of the sub-picture for use in image compression operations.

In addition, the above image data (YUV) must be temporarily stored in a buffer (linebuffer) before being played on the screen before being output.

How to reduce the number of buffers (line buffer) is an important subject in this technical field.

Contents of the invention

The invention discloses an image capture device and an applied image pre-processing method.

The image capturing device of the present invention includes a sensing unit, an image data generating unit, a first image pre-processing unit, a buffer group, a second image pre-processing unit, and an image compression unit. The sensing unit and the image data generating unit are responsible for sensing images and generating image data, wherein the image data includes luminance data and chrominance data. Luminance data is temporarily stored in the buffer group. The chrominance data is converted into chroma merged data by the first image pre-processing unit and temporarily stored in the buffer group; wherein, the action performed by the first image pre-processing unit is: merge the chrominance data of adjacent rows, so as to Reduce the data volume of chroma data. The second image pre-processing unit is located after the buffer group, and is used to obtain the above-mentioned chroma combined data, and make adjacent lines share the corresponding chroma combined data, so as to output the processed chrominance data. The processed chrominance data and brightness data provided by the second image pre-processing unit and the buffer group are used to display an image on a screen.

The present invention can be used for preprocessing of image processing operations such as image compression operations. In addition to playing on a screen, any image processing operations that need to collect sub-pictures, such as image compression operations, can also use this image pre-processing method. The method includes: receiving image data, including luminance data and chrominance data; merging chrominance data of adjacent lines to generate chrominance combined data; using a buffer group to temporarily store the luminance data and the chrominance combined data; and obtaining the chroma combined data from the buffer group, and making adjacent lines share the corresponding chroma combined data, so as to obtain processed chrominance data. The processed chrominance data and the brightness data temporarily stored in the buffer group are used for image playback or subsequent image processing operations.

Description of drawings

In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

FIG. 1 illustrates 4:2:2 chroma sampling with image data of a frame 100; and

FIG. 2 is an embodiment of the image capture device of the present invention.

Description of main component symbols:

100～one picture;

102～corresponds to the pixels in the 0th row and 0th column of the screen 100;

104～corresponds to the pixels in the 0th row and 1st column of the screen 100;

200～image capture device;

202～sensing component; 204～image data generating component;

206～the first image pre-processing unit;

208～buffer group;

210 - the second image pre-processing unit;

212～image compression unit; 214～storage medium;

216～brightness data; 218～chromaticity data;

220～chromaticity combined data; 222～processed chromaticity data;

224～compressed image; 226～screen;

Y ₀₀ -Y ₇₃ ~Brightness data;

U ₀₀ -U ₇₂ ~ bluishness data; and

V ₀₁ -V ₇₃ ~ reddish color data.

Detailed ways

FIG. 2 is an embodiment of the image capture device of the present invention. The image capturing device 200 can be a digital photographing device such as a web-cam, a digital camera, etc., and includes a sensing unit 202, an image data generating unit 204, a first image pre-processing unit 206, a buffer group 208, And a second image pre-processing unit 210 .

The sensing unit 202 can be a photosensitive device, such as a CCD or a CMOS sensor, for sensing images. The image analog signal sensed by the sensing unit 202 is amplified by the image data generating unit 204 and converted into digital image data (including data 216 and 218 ). The image data may be YUV encoded, including luma data 216 (Y) and chrominance data 218 (U and V). As shown in the figure, the luma data 216 (Y) is temporarily stored in the buffer group 208; and the chrominance data 218 is converted into the chrominance combined data 220 by the first image pre-processing unit 206 before being temporarily stored in the buffer group Group 208. The action performed by the first image pre-processing unit 206 is: merge the chrominance data of adjacent lines. Therefore, the data volume of the chroma combined data 220 is much lower than the data volume of the chroma data 218 .

When the buffer group 208 is full, it will output its content and start a new storage operation. The second image pre-processing unit 210 is located between the buffer group 208 and the image compression unit 212 to receive the chroma combined data 220 and share the chroma combined data with adjacent lines to obtain the processed chrominance data 222 . The luma data 216 and processed chrominance data 222 provided by the buffer group 208 are used to play the image on the screen 226 .

In addition, the luma data 216 and processed chrominance data 222 provided by the buffer group 208 can also be used for image compression. As shown in FIG. 2 , the image compression unit 212 is coupled to the second image pre-processing unit 210 and the buffer group 208, and performs processing on the processed chrominance data 222 and luminance data 216 provided by the above-mentioned components (210 and 208). An image compression operation. The compressed image 224 can be stored in a storage medium 214 .

In some embodiments, the image data (216 and 218) may be 4:2:2 chroma sampling; the chroma data 218 includes bluishness data (usually denoted as U or C _b ) and reddishness data ( Usually marked as V or Cr). Referring to the Y, U and V samples of the picture 100 marked in FIG. 1, the first image pre-processing unit 206 can combine the chrominance data of adjacent lines by the following operation (Expression 1):

M _y,2x = (U _2y,2x +U _2y+1,2x )/2; and

N _y,2x+1 =(V _2y,2x+1 +V _2y+1,2x+1 )/2; (Expression 1)

Wherein, y and x are variables; M _{y, 2x} and N _{y, 2x+1 form} the chromaticity combined data 220 of FIG. U _{2y+1, 2x} is the bluishness data of the pixels in row 2y+1 and column 2x of the picture; V _{2y, 2x+1} is row 2y and row 2x+1 of the picture The reddish chromaticity data of the column pixel; and V _{2y+1, 2x+1} is the reddish chromaticity data of the 2y+1 row and 2x+1 column pixel of the picture.

Taking two adjacent rows of row 0 and row 1 as an example, the chrominance data U ₀₀ in row 0 and the chrominance data U ₁₀ in row 1 are combined into chrominance combined data M ₀₀ , where M ₀₀ =(U ₀₀ +U ₁₀ )/2; the chrominance data V ₀₁ in row 0 and the chrominance data V ₁₁ in row 1 are combined into chroma combined data N ₀₁ , where N ₀₁ =(V ₀₁ +V ₁₁ )/2 ;The colorimetric data U ₀₂ in the 0th row and the chrominance data U ₁₂ in the 1st row are merged into the chrominance combined data M ₀₂ , where M ₀₂ =(U ₀₂ +U ₁₂ )/2; the chrominance data in the 0th row The chrominance data V ₀₃ and the chrominance data V ₁₃ in the first row are combined into chrominance combined data N ₀₃ , where N ₀₃ =(V ₀₃ +V ₁₃ )/2, and so on.

The capacity of the buffer group 208 can be designed according to the data amount of a sub-picture for image processing operation. Assuming that a sub-picture includes 8x16 pixels, then taking the sub-picture of rows 0-7 and columns 0-15 as an example, the data stored in the buffer group 208 includes:

Y ₀₀ , Y ₀₁ , Y ₀₂ , Y ₀₃ ,..., Y 0, ₁₅

Y ₁₀ , Y ₁₁ , Y ₁₂ , Y ₁₃ ,..., Y 1, ₁₅

Y ₇₀ , Y ₇₁ , Y ₇₂ , Y ₃ , ..., Y 7, ₁₅

M ₀₀ , N ₀₁ , M ₀₂ , N ₀₃ ,…, N 0, ₁₅

M ₁₀ , N ₁₁ , M ₁₂ , N ₁₃ ,..., N 1, ₁₅

M ₂₀ , N ₂₁ , M ₂₂ , N ₂₃ ,..., N _2,15

M ₃₀ , N ₃₁ , M ₃₂ , N ₃₃ ,..., N 3, ₁₅

Referring to the traditional technology that does not use the first and second image pre-processing units 206 and 210, the data required to be stored in the traditional buffer group includes:

Y ₀₀ , Y ₀₁ , Y ₀₂ , Y ₀₃ ,..., Y 0, ₁₅

Y ₁₀ , Y ₁₁ , Y ₁₂ , Y ₁₃ ,..., Y 1, ₁₅

Y ₇₀ , Y ₇₁ , Y ₇₂ , Y ₃ , ..., Y 7, ₁₅

U ₀₀ , V ₀₁ , U ₀₂ , V ₀₃ ,…, V 0, ₁₅

U ₁₀ , V ₁₁ , U ₁₂ , V ₁₃ ,..., V 1, ₁₅

U ₇₀ , V ₇₁ , U ₇₂ , V ₃ ,..., V _{7, 15}

The capacity of the buffer group 208 of the present invention can be reduced by 25%, greatly saving circuit area and cost.

In some implementations, the second image pre-processing unit 210 can use Expression 2 to make adjacent lines share the chroma-combined data:

U' _{2y, 2x} = U' _{2y+1, 2x} = M _{y, 2x} ; and

V' _{2y, 2x+1} = V' _{2y+1, 2x+1} = N _{y, 2x+1} ; (Expression 1)

Wherein, U' _{2y, 2x} , U' _2y+1 , 2x , V' _{2y, 2x+1} and V' _{2y+1, 2x+1} constitute the processed chromaticity data 222 in FIG. 2 .

Taking the two adjacent rows of row 0 and row 1 as an example, the chromaticity combined data M ₀₀ is copied to the pixel in column 0 of row 0 and row 1, and used as the processed chromaticity data U' ₀₀ and U' ₁₀ , where U' ₀₀ ＝U' ₁₀ ＝M ₀₀ ; the chromaticity combined data N ₀₁ is copied to the pixels in the first column of the 0th and 1st rows, and used as processed chromaticity data V' ₀₁ and V' ₁₁ , wherein V' ₀₁ =V' ₁₁ = N ₀₁ ; the chromaticity combined data M ₀₂ is copied to the pixels in the second column of the 0th and 1st rows, and used as the processed chromaticity data U' ₀₂ and U' ₁₂ , where U' ₀₂ = U' ₁₂ = M ₀₂ ; The combined chroma data N ₀₃ is copied to the pixels in the third column of the 0th and 1st rows, and used as the processed chrominance data V' ₀₃ and V' ₁₃ , wherein V' ₀₃ =V' ₁₃ =N ₀₃ ; . . .

By means of expression 2, the image data of each pixel (relating to luminance data and chrominance data) can be collected completely, so as to perform playback or image compression operation.

The method for reducing the amount of chrominance data in the present invention can be used in pre-processing of other image processing operations, and is collectively referred to as an image pre-processing method. In addition to the image compression operation mentioned in the embodiment of FIG. 2 , any image processing operation that needs to collect sub-picture data first can use this image pre-processing method. This method includes: receiving image data, including luminance data (such as Y) and chrominance data (such as U and V), and this image data can be 4:2:2 chroma sampling; execute the above expression 1, and merge adjacent The chrominance data of row is to produce chroma combined data (M and N of expression 1); Temporarily store above-mentioned luma data Y and above-mentioned chrominance combined data (M and N) with a buffer group; And from above-mentioned buffer The group obtains the above-mentioned chroma combined data (M and N), and executes the above expression 2 to make adjacent rows share the chroma combined data (M and N) to obtain the processed chroma data (U' and V'). The processed chrominance data (U' and V') and the temporarily stored luminance data (Y) in the buffer group will be used for image playback or image processing operations. The first image pre-processing unit 206 combines the chrominance data of adjacent lines to generate chrominance combined data, which can save the hardware space of the buffer group, thereby reducing the cost of the image capture system.

Claims (6)

1. image capture unit, comprising:

One sensing cell receives light source to produce an image simulation signal;

One image data generation unit receives above-mentioned image simulation signal and produces image data, and above-mentioned image data comprises brightness data and chroma data;

One first image pretreatment unit, the chroma data that merges adjacent lines is to produce the colourity merging data;

One buffer group is in order to temporary above-mentioned brightness data and above-mentioned colourity merging data; And

One second image pretreatment unit couples above-mentioned buffer group obtaining above-mentioned colourity merging data, and makes adjacent row share pairing colourity merging data must handle the back chroma data;

Wherein, chroma data is used to show that an image is in a screen after the brightness data that provided of this buffer group, the processing that provided with this second image pretreatment unit;

Wherein the following column operations of this first image pretreatment unit merges the chroma data of adjacent lines:

M _{Y, 2x}=(U _{2y, 2x}+ U _2y+1,2x)/2; And

N _y，2x+1＝(V _2y，2x+1+V _2y+1，2x+1)/2；

Wherein, y and x are variable;

M _{Y, 2x}With N _{Y, 2x+1}Form above-mentioned colourity merging data;

U _{2y, 2x}Be that a picture 2y is capable, the blue partially degrees of data of 2x row pixel;

U _2y+1,2xThe blue partially degrees of data of, 2x row pixel capable for this picture 2y+1;

V _{2y, 2x+1}The red partially degrees of data of, 2x+1 row pixel capable for this picture 2y; And

V _2y+1,2x+1The red partially degrees of data of, 2x+1 row pixel capable for this picture 2y+1;

Wherein the following column operations of this second image pretreatment unit makes adjacent row share pairing colourity merging data:

U ' _{2y, 2x}=U ' _2y+1,2x=M _{Y, 2x}And

V’ _2y，2x+1＝V’ _2y+1，2x+1＝N _y，2x+1；

Wherein, U ' _{2y, 2x}, U ' _2y+1,2x, V ' _{2y, 2x+1}With V ' _2y+1,2x+1Form chroma data after the above-mentioned processing.

2. image capture unit as claimed in claim 1, it is characterized in that, more comprise an image compression unit, couple above-mentioned buffer group to obtain above-mentioned brightness data, couple this second image pretreatment unit obtaining chroma data after the above-mentioned processing, and to above-mentioned brightness data, with handle the back chroma data and carry out an image compaction algorithms.

3. image capture unit as claimed in claim 1 is characterized in that, above-mentioned image data is the sampling of 4:2:2 colourity, and above-mentioned chroma data comprises blue partially degrees of data and red partially degrees of data.

4. image preprocessing method, comprising:

Receive image data, comprising brightness data and chroma data;

The chroma data that merges adjacent lines is to produce the colourity merging data; And

Make adjacent row share pairing colourity merging data must handle the back chroma data;

Wherein, chroma data and above-mentioned brightness data are used to show that an image is in a screen after the above-mentioned processing;

Wherein following column operations merges the chroma data of adjacent lines:

M _{Y, 2x}=(U _{2y, 2x}+ U _2y+1,2x)/2; And

N _y，2x+1＝(V _2y，2x+1+V _2y+1，2x+1)/2；

Wherein, y and x are variable;

M _{Y, 2x}With N _{Y, 2x+1}Form above-mentioned colourity merging data;

U _{2y, 2x}Be that a picture 2y is capable, the blue partially degrees of data of 2x row pixel;

U _2y+1,2xThe blue partially degrees of data of, 2x row pixel capable for this picture 2y+1;

V _{2y, 2x+1}The red partially degrees of data of, 2x+1 row pixel capable for this picture 2y; And

V _2y+1,2x+1The red partially degrees of data of, 2x+1 row pixel capable for this picture 2y+1;

Wherein following column operations makes adjacent row share pairing colourity merging data:

U ' _{2y, 2x}=U ' _2y+1,2x=M _{Y, 2x}And

V’ _2y，2x+1＝V’ _2y+1，2x+1＝N _y，2x+1；

Wherein, U ' _{2y, 2x}, U ' _2y+1,2x, V ' _{2y, 2x+1}With V ' _2y+1,2x+1Form chroma data after the above-mentioned processing.

5. image preprocessing method as claimed in claim 4 is characterized in that, above-mentioned image data is the sampling of 4:2:2 colourity, and above-mentioned chroma data comprises blue partially degrees of data and red partially degrees of data.

6. image preprocessing method as claimed in claim 4 is characterized in that, more comprises above-mentioned brightness data, carries out an image compaction algorithms with processing back chroma data, to be stored in a Storage Media.

Images