CN101834974B - Multispectral photoreceptive device and sampling method thereof - Google Patents

Abstract

The invention relates to a multispectral photoreceptive device and a sampling method thereof, wherein the method comprises the following processes that: a first merging process is used for pairwise merging and sampling among closely adjacent pixels in the same line and different columns, different lines and the same column or different lines and different columns in a pixel array and obtaining the sampling data of a first merged pixel; a second merging process is used for merging and sampling the sampling data of the first merged pixel obtained in the first merging process and obtaining the sampling data of a second merged pixel; and a third merging process is used for obtaining the sampling data of a third merged pixel in a digital space by color changing and image zooming methods. The invention is applicable for the multi-pixel sharing reading and amplifying circuit of a monolayer colored photoreceptive chip, a single-surface double-layer multispectral photoreceptive chip and a double-surface double-layer multispectral photoreceptive chip. On the basis of absorbing the kernel of a traditional pixel reading circuit, the invention has magnitude-order breakthrough on principles, basic circuits and performance and simply and directly realizes the YUV format output of sampled images on the photoreceptive device while realizing high-performance sub sampling.

Description

A kind of multi-optical spectrum light-sensitive device and the method for sampling thereof

Technical field

The present invention relates to the reading of the photosensitive pixels of sensitive chip, the sub-sampled data of the photosensitive pixels of especially large array sensitive chip reads.Specifically, the present invention relates to a kind of multi-optical spectrum light-sensitive device and the method for sampling thereof.

Background technology

The present invention be the inventor a little earlier 1 " multi-optical spectrum light-sensitive device and preparation method thereof " (PCT/CN2007/071262) and " multi-optical spectrum light-sensitive device and preparation method thereof " (Chinese application number: continuity 200810217270.2) aims to provide the realization of more concrete and preferred semiconductor circuit and chip level.

Sensor devices before, or be absorbed in color visible, or be absorbed in infrared light, seldom have the two is combined.Although other invention or application are also arranged, for example adopt the semiconductor technology (" Silicon infrared focalplane arrays " of indium cadmium, M.Kimata, in Handbook of Infrared Detection Technologies, editedby M.Henini and M.Razeghi, pp.352-392, Elsevier Science Ltd., 2002), realize the induction of visible and infrared light, but they do not obtain colour simultaneously.The method that obtains colored and infrared sensitization in the time of before is by a color sensitive device and an infrared sensor devices; the physics superposition together (as " and Backside-hybridPhotodetector for trans-chip detection of NIR light]; by T.Tokuda et al., in IEEEWorkshop on Charge-coupled Devices & Advanced Image Sensors; Elmau; Germany; May2003; and " A CMOS image sensor with eye-safe detection function using backsidecarrier injection "; T.Tokuda et al., J.Inst Image Information & Television Eng., 60 (3): 366-372, March 2006).

The inventor a little earlier 1 " multi-optical spectrum light-sensitive device and preparation method thereof " (PCT/CN2007/071262) and " multi-optical spectrum light-sensitive device and preparation method thereof " (Chinese application number: 200810217270.2), a kind of manufacture method that can simultaneously obtain the new multi-optical spectrum light-sensitive device of colored and infrared image has been proposed. this novel sensor devices, greatly expand the sensor devices dynamic range, thereby meet automobile, high performance requirements in the fields such as security protection. moreover, by it for undersized color sensitive device, the camera of using as mobile phone, also can improve significantly image quality. moreover, they can adopt existing CMOS, CCD, or other semiconductor light-sensing device manufacturing technology is made, and each technology can have very many and effective manufacture method and structural design. the present invention mainly provides a few to adopt the manufacture method of CMOS/CCD semiconductor technology.

But this new bilayer or multilayer sensor devices bring a new problem, be exactly data volume be that the twice of traditional individual layer sensor devices is even more.Although two-layer sensor devices only needs the pixel of half just can obtain the resolution equal with the individual layer sensor devices, the data of the sensor devices of the large array of high speed processing, remain an improved problem of needs.

Recent years, invented some outstanding methods large array image has been carried out to high performance sub sampling, for example reading circuit is shared, parallel (Row Binning) and (Column Binning) Sampling techniques arranged side by side are (as U.S. Pat 6,801,258B1, US6,693,670B1, US7,091,466B2, US7,319,218B2 etc.).It is worth mentioning that U.S. Pat 6,693,670B1, US7,091,466B2 and US7,319,218B2.These patents provide effect pretty good and convenient-to-running means, have realized that N is capable or N is listed as or capable be listed as with N and the use of M.

It is optimum that but these sub sampling technology do not reach.For example, adopt parallel and sub sampling technology arranged side by side is put N and be a point, the improvement of signal noise ratio (snr) of image is the highest can only be

(see U.S. Pat 7,091,466B2, US7,319,218B2).This is because walk abreast and just signal done on average side by side, thereby just the variance of random noise has been reduced

doubly, and the intensity of useful signal itself is not enhanced, and is only to replace with the mean value of several points.Usually also have the steady noise of gradual low frequency in picture signal, this partial noise does not reduce yet.

In addition, existing sub sampling technology is only to consider that individually Bayer arranges or the demand of the sub sampling of the sensitive chip that CYMG tetra-colored patterns are arranged, and also for subsequent calculations, does not do simplification.The coloured image of a Bayer pattern for example, parallel and Sampling techniques (U.S. Pat 7 arranged side by side through U.S. Micron Technologies Inc., 091,466B2, US7,319,218B2) after, remain a Bayer pattern, still need just can obtain through complicated processing the YUV image that preview and memory phase are had a preference for.And some other can improve the sub sampling circuit of signal to noise ratio, need to use complicated integrating circuit and comparator, thereby bring the increase of auxiliary circuit and the increase of frequency.

The huge restriction of the existing sub sampling technology of another one, be exactly, parallel and side by side only identical, and be not close on space between the color pixel of (middle across other pixel) and carry out.For Bayer pattern or CYMG tetra-colored patterns, identical color pixel, spatially be not close to, through parallel and side by side after image, destroyed the equally distributed characteristics on the space of original image, therefore, if this situation of non-specific adaptation of back-end processing, at linear edge, just easily produce the sawtooth effect.

Especially, for the bilayer that will pay close attention to for the present invention or multilayer sensor devices, it is narrow and mediocre that prior art just seems, because bilayer or multilayer sensor devices provide very many outstanding colour images to arrange, thereby, no matter be signal-obtaining and sub sampling, all should, for the characteristics of bilayer or multilayer sensor devices, make improvement.

Summary of the invention

Purpose of the present invention is intended to propose more superior sub sampling principle and advanced sub sampling circuit, and sub sampling and successive image are processed together with combining and optimized.The invention provides a kind of multi-optical spectrum light-sensitive device and the method for sampling thereof, in order to can overcome this faint deficiency that the data volume of bilayer or the multispectral sensitive chip of multilayer is larger.Here, the method for sampling mainly comprises sub sampling, but has also contained the full figure sampling.Need to understand, the present invention is not confined to bilayer or multilayer multi-optical spectrum light-sensitive device, and the sensor devices for individual layer, be suitable for equally.

For ease of describing the present invention and explaining it and the difference of prior art, we now provide the definition of following noun: double-deck sensor devices, double-sided light sensitive device, and two-way sensor devices.Wherein, double-deck sensor devices refers to photosensitive pixels two-layer by physically being divided into (as the inventor at (PCT/CN2007/071262) described two-layer sensor devices of patent application " multi-optical spectrum light-sensitive device and preparation method thereof " a little earlier), and every one deck all contains the photosensitive pixels of responding to special spectrum.The double-sided light sensitive device refers to that sensor devices has two photosensitive surfaces, at least sensitization a direction of each face.Two-way sensor devices refers to that sensor devices can sensitization the direction of two (usually being mutually 180 degree), that is can sensitization from the front and back of sensor devices.

A sensor devices can have bilayer simultaneously, two-sided, and in two-way these three characteristics one, and two, and all three characteristics.

In order to solve the problems of the technologies described above, the present invention has adopted following technical scheme:

A kind of multi-optical spectrum light-sensitive device, comprise the cell array of arranging with row and column, and

The first merge cells, merge sampling between the pixel for the colleague's heterotaxy of the next-door neighbour to described cell array, different row same column or different row heterotaxy in twos, obtains the sampled data of the first merging pixel;

The second merge cells, merge sampling for the first sampled data that merges pixel that the first merge cells is obtained, and obtains the sampled data of the second merging pixel.

Described multi-optical spectrum light-sensitive device, also comprise the 3rd merge cells, for the second sampled data that merges pixel that the second merge cells is obtained, merges sampling, obtains the sampled data of the 3rd merging pixel.

Described multi-optical spectrum light-sensitive device, the quadratic deviation mode of described the first merge cells or the second merge cells is the signal averaging mode between electric charge accumulate mode between identical or different color pixel or two different color pixels, wherein the quadratic deviation mode between the different color pixel (comprising electric charge accumulate mode or signal averaging mode) is in accordance with the mode of colour space transformation, to meet the requirement of colour reconstruction.

Described multi-optical spectrum light-sensitive device, described electric charge accumulate mode completes in reading electric capacity (FD).

Described multi-optical spectrum light-sensitive device, the merging sample mode based on color of described the first merge cells or the second merge cells comprises homochromy merging mode, heterochromatic merging mode, mix merging mode or selectivity abandons unnecessary color and merges mode, and the merging sample mode that the first merge cells adopts with the second merge cells is homochromy merging mode when different, also in two merge cellses, has at least a merge cells not adopt homochromy merging mode.

Described multi-optical spectrum light-sensitive device, the location-based merging sample mode of described the first merge cells or the second merge cells comprises at least one in following several mode: directly output to bus signal automatic average mode, slip a line or jump row mode and sample mode one by one.That is these several location-based merging sample modes may be used alone, can also be used in combination.

Described multi-optical spectrum light-sensitive device, the described the 3rd merging sample mode that merges sampling unit comprises: at least one in colour space transformation mode and rear end digital image scaling mode.

Described multi-optical spectrum light-sensitive device, described colour space transformation comprises the conversion of RGB to the conversion in CyYeMgX space, RGB to yuv space, or CyYeMgX is to the conversion of yuv space, wherein X is any in R (red), G (green), B (orchid).

Described multi-optical spectrum light-sensitive device, described cell array is comprised of a plurality of grand pixels that comprise at least one basic pixel, and wherein basic pixel can be passive pixel or active pixel.

Described multi-optical spectrum light-sensitive device, the basic pixel in described grand pixel is by square formation or honeycomb arrangement.

Described multi-optical spectrum light-sensitive device, the building form of described grand pixel can comprise following building form at least one: not with the 3T that reads electric capacity (FD) initiatively the pixel building form, with a 4T who reads electric capacity (FD) pixel building form initiatively.

Described multi-optical spectrum light-sensitive device, each one, grand pixel band reads the 4T active pixel of electric capacity (FD), adopts 4 sharing modes, 6 sharing modes or 8 sharing modes.

Described multi-optical spectrum light-sensitive device, described grand pixel also can comprise following building form: pixel and two the lighttight electric capacity (FD) that read that are positioned at two interlines by four square formations, arranged form, the pixel of lastrow and the pixel of next line share one and read electric capacity (FD), two are read between electric capacity (FD) and can realize that electric charge shifts, and at least one reads on electric capacity and is connected with reading circuit.

Described grand pixel can by with 2 share, 3 share or 3T that 4 are shared to read electric capacity (FD) or 4T initiatively the basic pixel of pixel form, adopt 4 bridge-type sharing modes, 6 bridge-type sharing modes or 8 bridge-type sharing modes.

Described multi-optical spectrum light-sensitive device, each grand pixel by with 2 share, 3 share or 4T that 4 are shared to read electric capacity (FD) initiatively the basic pixels of pixels form, adopt 4 bridge-type sharing modes, 6 bridge-type sharing modes or 8 bridge-type sharing modes.

Described multi-optical spectrum light-sensitive device, the full figure sample mode of described multi-optical spectrum light-sensitive device comprise line by line scan, row by row system or line by line scan, interlacing or inter-bank reading manner.

The invention also discloses a kind of method of sampling of multi-optical spectrum light-sensitive device, comprising:

The first merging process, merge sampling between the pixel for the colleague's heterotaxy of the next-door neighbour to described cell array, different row same column or different row heterotaxy in twos, obtains the sampled data of the first merging pixel;

The second merging process, merge sampling for the first sampled data that merges pixel that the first merging process is obtained, and obtains the sampled data of the second merging pixel.

The described method of sampling also comprises: the 3rd merging process, merge sampling for the second sampled data that merges pixel that the second merging process is obtained, and obtain the sampled data of the 3rd merging pixel.

The described method of sampling, the quadratic deviation sample mode of described the first merging process or the second merging process is electric charge phase add mode between identical or different color pixel or the signal averaging mode between the different color pixel, wherein the quadratic deviation mode between the different color pixel (comprising electric charge phase add mode or signal averaging mode) is in accordance with the mode of colour space transformation, to meet the requirement of colour reconstruction.

The described method of sampling, the merging sample mode based on color of described the first process or the second merging process comprises homochromy merging mode, heterochromatic merging mode, mix the merging mode or selectivity is abandoned unnecessary color mode, and in the first merging process and the second merging process, at least one merging process is not homochromy merging mode.

The described method of sampling, the location-based merging sample mode that described the first merging process or second merges sampling process comprises at least one in following several mode: directly output to bus signal automatic average mode, slip a line or jump row mode and sample mode one by one.

The described method of sampling, the merging sample mode that described the 3rd merging sampling process is carried out comprises: colour space transformation mode, rear end digital image scaling mode.

The described method of sampling, described colour space transformation comprises that the conversion of RGB to the conversion in CyYeMgX space, RGB to yuv space or CyYeMgX are to the conversion of yuv space, wherein X is any in R (red), G (green), B (orchid).

The described method of sampling, the mode of its full figure sampling comprise line by line scan, row by row system or line by line scan, interlacing or inter-bank reading manner.

The technique effect that the present invention is useful is:

Sub sampling at least is divided into to two processes, and aforesaid first merges sampling process and the second merging sampling process.First merges sampling process and second merges sampling process, usually occur between row (merging) sampling and row (merging) sampling of pixel, mainly analog signal is carried out, except electric charge addition part is only done usually in the first merging sampling process, its order and content normally can exchange.In addition, also can comprise that the 3rd merges sampling process, the 3rd merging sampling process is mainly carried out digital signal after occurring in analog-to-digital conversion.

For the first merging sampling process, be that the pixel of getting two next-door neighbours in cell array is merged.On the one hand, completed the merging of next-door neighbour's pixel, in this article, we are called the first merging pixel by the pixel after merging, it will be appreciated that, first merge pixel just for the present invention's description just, utilize this concept to refer to carry out the pixel after the first merging process, and not on represents physical, in cell array, exist one " first merges pixel "; The sampled data that data after the sampling of two next-door neighbour's quadratic deviations is called to the first merging pixel.The next-door neighbour, mean between two pixels from level, vertical, or, on angular direction, keeping close watch over and be close to, centre does not have other pixel.Next-door neighbour's situation comprises colleague's heterotaxy, different row same column, or different row heterotaxy.Generally speaking, in this merging, signal will be at least the signal averaging of two pixels, and noise can reduce

therefore, after merging, at least signal to noise ratio can be improved

times, and this merging can be carried out between the pixel of identical or different color.On the other hand, because the color of two merging can be different, i.e. color addition or average, from the three primary color theory of color, the addition of two kinds of primary colors is complementary colors of another kind of primary colors, in other words, the quadratic deviation of two different primary colors, produce the complementary color of another kind of primary colors, from the former color space, having transformed to the complementary color space, is only that colour space transformation has occurred, and we still can complete by different complementary colors colored the reconstruction.Also, by the present invention, can realize that the quadratic deviation of different color, to improve signal to noise ratio, can carry out again colour simultaneously and rebuild.Therefore whole sub sampling process also is optimized, and more adapts to the high-speed requirement of the cell array of big data quantity.A basic demand of colour space transformation is, needed RGB (or YUV, or CYMK) color can (by means such as interpolation) be rebuild in the combination of the color after conversion.

Need to understand, because common cell array comprises a plurality of pixels, the first merging sampling is just merged two pixels, and obviously, the first merging pixel that merges formation also has a plurality of.For the first different merging pixels, the color that it adopts merges mode can be identical, also can be different.When the first merging is all carried out between identical color, we are called homochromy merging mode by it; When the first merging is all carried out between different colors, we are called heterochromatic merging mode by it; When the first assembling section is carrying out between same hue, partly carrying out between different color, we are called by it merging mode that mixes; Abandoned (certainly, abandoning is optionally, for example, can not therefore have influence on colored the reconstruction) when some the unnecessary colors in cell array, such color merging mode is called selectivity and abandons unnecessary color mode.

Obvious, the second merging process is the operation to a plurality of the first merging pixels, same, the first merging pixel that can color is identical is merged; Also can be by color different first merges pixel is merged (certainly, may cause in this case trichromatic whole addition and can't reconstruct colour).

Above-mentioned homochromy merging, heterochromatic merging, mix the modes such as merging, to do the classification based on color by merging sampling, in addition, the angle of choosing from the position that merges sampling, the merging sample mode of the first merging process and the second merging process comprises: directly output to bus signal automatic average mode, slip a line or jump the row mode, sample mode, and these modes one by one two or three the time use.Except electric charge addition part can only be done usually in the first merging sampling process, the first merging process and the second merging process, except the difference of order, its mode is all identical and can exchange.

The so-called signal automatic average mode that directly outputs to bus, be exactly that the signal (color is identical or different) that needs are merged, output to the data acquisition bus simultaneously and get on, by the autobalance of (voltage) signal, obtain the mean value that needs combined signal.What is called is slipped a line or is jumped the row mode and skip exactly some row or column, thereby realizes (merging) sampling by the mode that reduces data volume.What is called is sample mode one by one, is exactly in fact not do any merging, reads according to this original pixel or first and merges pixel.These three modes have some to use simultaneously, for example, slip a line or jump the row mode can with the signal automatic average mode that directly outputs to bus or one by one sample mode use simultaneously.

The 3rd sub sampling mode that merges sampling process comprises the serial use of colour space transformation mode, rear end digital image scaling mode and these two modes.The first and second merging process are mainly to carry out on analog signal, and the 3rd sub sampling process is mainly to carry out on digital signal, after analog-to-digital conversion, carry out.By by three in different spatial or four color pixels, be used as the value on same point and be transformed into another color space, just can realize again level and (or) data on vertical direction reduce, thereby reach the effect of sub sampling.And the digital image scaling mode is sub sampling mode commonly used the most directly perceived.

The present invention has also realized the electric charge addition first when merging sampling.Current merging sampling is nearly all only accomplished voltage or current signal average, and this mode, when merging the N point, can only improve signal to noise ratio at most

doubly.This is to merge sampling because the pixel that existing merging sampling is all N same hue shares the mode of an output line, on this root output line, it is average that the voltage of each pixel or current signal must carry out (automatically), therefore, the raising of its signal to noise ratio just is to have reduced after noise merges thereby signal to noise ratio is improved at most doubly.And adopt electric charge phase add mode of the present invention, and for example, by reading capacitive charge storage, realize the cumulative of electric charge, thereby can being superposeed, signal make signal to noise ratio can improve at least N times, than the method height of signal averaging at least

doubly.That is to say, N signal merged with the method for electric charge addition, the highlyest in theory can reach N ²the effect of individual signal phase average or better (as described below), this is the means of the raising signal to noise ratio of effect highly significant.

The addition of next-door neighbour's pixel, also bring the significant effect of another one, is exactly that phase mutual interference (cross-talking) effect between pixel is weakened.This is the color due to original phase mutual interference, is legal one now, that is to say, originally belong to a part of signal of noise, become now effective signal section, therefore, N signal charge addition brings the improvement of signal to noise ratio, can approach the theoretic upper limit,

doubly, thereby, be equivalent to N ³the effect of individual signal phase average.

The electric charge addition is a merging sampling means that effect is distinguished, but it requires the pixel of merging spatially adjacent.The reason that sub sampling before can not be done is, sub sampling before only carries out between the pixel of same hue, but due between merged pixel across other pixel, thereby can't realize the electric charge addition.For the multilayer sensor devices, realize that the electric charge addition is relatively easy, because colour image is very abundant.And employing the present invention, we also can realize the electric charge addition at an easy rate on the individual layer sensor devices, as long as adopted the method for colour space transformation of the present invention.

The present invention is when full figure sampling (i.e. being sampled by highest resolution to an image), the mode that employing is lined by line scan, interlacing or inter-bank read, do not need to improve clock speed and adopt frame buffer, the full figure of large array image is read to frame per second double when clapping individual photograph.If increase AD converter and row cache, so, full figure reads frame per second and can also improve more.This method has very important value for saving mechanical shutter.

Please note of the present inventionly line by line scan, mode that interlacing or inter-bank read, from the field scan mode (interleaved scanning) in the traditional tv system, be different.Traditional field scan mode, be interlacing scan, and interlacing is read, and therefore, odd field and even field (no matter be sensitization or read) are poor in time one, i.e. field.And of the present inventionly line by line scan, mode that interlacing or inter-bank read, pixel in sensitive time, on order, be but with line by line scan, row by row system is duplicate, just the order that reads of row changes.

It is stronger that the present invention will describe new power by embodiment, the sensor devices that adaptive surface is wider and sub-sampling methods thereof.These preferred implementation methods, be only advantage and the implementation method that the present invention and present inventor's previous related invention is described as an example, rather than in order to limit the protection range of these inventions.

For the knowledgeable people of relevant industry, the above and other objects and advantages of the present invention, the details that realizes case of preferably explaining with a plurality of illustrations below reading will be fairly obvious after describing.

The accompanying drawing explanation

Fig. 1 be the passive pixel of CMOS read (sampling) circuit.

Fig. 2 be CMOS 3T initiatively pixel read (sampling) circuit.

Fig. 3 be CMOS 4T initiatively pixel read (sampling) circuit.

Fig. 4 (a) and (b) be respectively the initiatively relation of pixel and passive reading (sampling) circuit and row address selection circuit of CMOS.

Fig. 5 is the structural representation that (sampling) circuit and row address selection circuit and column address are selected circuit that reads of CMOS pixel.

Fig. 6 is one and typically with the practical CMOS pixel of row buffer memory, reads (sampling) circuit diagram.

Fig. 7 is the comparison of reading manner (a) with the reading manner (b) of CMOS pixel of CCD pixel.Notice in Fig. 7 (a) function of transmitting in turn in vertical direction between the CCD pixel.

Fig. 8 describes U.S. Pat 7,091, the schematic diagram of the basic principle of 466B2.By open the switch of the identical pixel that needs merging simultaneously, coherent signal is outputed on the sampling bus simultaneously and oneself reaches balance afterwards, thereby obtain merging the mean value of pixel.

Fig. 9 describes U.S. Pat 7,319, the schematic diagram of the basic principle of 218B2.The identical pixel that need to merge by the different time high-speed sampling, oneself reach balance after outputing on the sampling bus by coherent signal record after fully simultaneously again, thereby obtain merging the mean value of pixel.Its basic thought and U.S. Pat 7,091,466B2 is identical, has just adopted different circuit to realize.

The made a summary basic thought of the homochromy folding of existing pixel of Figure 10: the pixel (in the mode of signal averaging) that the color of adjacent grand pixel is identical merges.Figure 10 (a) is the schematic diagram that row merge, and Figure 10 (b) is the schematic diagram that ranks merge simultaneously.

What Figure 11 showed is that current reasonable 4-point is shared the initiatively reading circuit of photosensitive pixels of 4T, and on average each pixel has adopted 1.75 doors.

What Figure 12 showed is that a 6-point is shared the initiatively reading circuit of photosensitive pixels of 4T, and on average each pixel has only adopted 1.5 doors.This 6-point is shared the initiatively reading circuit of photosensitive pixels, be applicable to adopt the two-sided double-deck sensor devices of honeycomb arrangement (referring to " multi-optical spectrum light-sensitive device and preparation method thereof ", that is allow the upper and lower two-layer light sensitive diode of all three the compound pixels in a grand pixel share same electric capacity (FD) and the 3T reading circuit of reading China's application number 200810217270.2).

What Figure 13 showed is that a 8-point is shared the initiatively reading circuit of photosensitive pixels of 4T, and on average each pixel has only adopted 1.375 doors.This 8-point is shared the initiatively reading circuit of photosensitive pixels, being applicable to take 4 grand pixels is the two-sided double-deck sensor devices that basic square formation is arranged, that is allows the upper and lower two-layer light sensitive diode of all four the compound pixels in a grand pixel share same electric capacity (FD) and the 3T reading circuit of reading.

Figure 14 shows is of the present invention heterochromatic and mix the basic thought of the technology of merging: first the pixel (in the mode of signal averaging or addition) of two interior similar and different colors of same grand pixel is merged, and then do the merging merged between rear adjacent and pixel that color is identical.What Figure 14 (a) showed is the schematic diagram that a kind of two row of Bayer pattern sensor devices merge, and what Figure 14 (b) showed is the schematic diagram that a kind of two row of Baver pattern sensor devices and two row merge simultaneously.Wherein, see separately G and B, or during the merging of G and R, just formed heterochromatic merging; And by the merging of G and B, the merging of G and R, the merging of B and R, and the merging of G and G just formed and mixed merging while taking together, because the merging wherein had is between homochromy (G and G), other is between heterochromatic.The RGB primary colour image of Bayer pattern, after variegated merging, be converted into CyYeMgG complementary color image.In this drawing, by the merging of G and B, the merging of G and R, the merging of B and R, and the merging of G and G has formed first merging process.Second merging process, be to obtain the Cy on diverse location after merging, Ye, and Mg, and G value, the mode according to homochromy merging outputs on bus simultaneously, or adopts the mode of slipping a line or jumping row, skips some pixel, and reads one by one.

Figure 15 shows be variegated folding of the present invention for M more generally capable with N row combination situation (figure be 5x3, i.e. 5 row and 3 merging that are listed as).To slip a line or inter-bank and jumping row are combined with the method across being listed as, can obtain the situation of various similar Figure 15.The merging of 3 row 3 row can add to slip a line and jump by the merging mode of 2 row and 2 row and be listed as.Notice that the third line and fourth line are when carrying out cross-combining, will produce a pair of (Mg, G) signal that is positioned at same point midway.The convenience merged for the row of back, can the people for thinking Mg or G in front, to keep consistency.

Notice the symmetry of row and column, from this figure, we can extend 3x5,2x3,3x2,2x4,4x2,5x2,2x5,2x6 at an easy rate, 6x2,3x4,4x3,3x6,6x3,4x4,4x5,5x4,4x6,6x4,5x6,6x5,6x6,7x6,6x7,7x7x, 8x8, etc. various combination situations.2x2 more usefully, 2x4,4x2,4x4,3x6,6x3,6,6,4x8,8x4, and the merging factor that can keep easily the image length-width ratio of 8x8.Equally, in this drawing, the merging of G and B, the merging of G and R, the merging of B and R, and the merging of G and G, formed first merging process.Second merging process, be to obtain the Cy on diverse location after merging, Ye, and Mg, and G value, the mode according to homochromy merging outputs on bus simultaneously, and some pixels of not using (for example the 5th row in figure and the 10th row) in the middle of skipping.The color be skipped, by no longer participating in the 3rd merging sampling process of back, merge sampling process if sensor devices includes the 3rd.

What Figure 16 showed is that the extra 2x2 image that the color space matrixing brings dwindles.No matter the image of CyYeMgG is original image, or by variegated merging method of the present invention from the acquisition of Bayer RGB image, when we convert it to the YUV image, can obtain extra 2x2 and dwindle.The method of dwindling is exactly by a grand pixel of CyYeMgG, as being four pixels of concurrent and convert (Y, U, a V) pixel to, and then by (horizontal direction) U of two adjacent points and the V value average, just obtain preview and JPEG/MPEG and compress desirable YUV422 image.

What Figure 17 showed is a kind of outstanding reading circuit of the present invention.In this reading circuit, the photosensitive pixels of odd-numbered line and even number line, will share a lighttight electric capacity FD (as the FD1 in figure and FD2) that reads. and switch TG1 is used for selecting the electric capacity in the Gr light sensitive diode is transformed in FD1 and goes.Equally, switch TG2, TG3, TG4 is used to read respectively R, B, the capacitances such as Gb are to FD2, FD1, or go in FD2.Another one switch TG5, be used for the value will read in electric capacity FD1, in the mode of addition, transfers in FD2 and go (or transferring to FD1 from FD2).The photosensitive pixels of this layout, can adopt 4 bridge-types as shown in figure 18 to share reading circuit.In this circuit, read the lighttight requirement of electric capacity FD, be to realize lining by line scan as shown in figure 21, interlacing or inter-bank read necessary.

What Figure 18 showed is that 4 bridge-types for 4 grand pixel square formation Pareto diagrams of the present invention are shared reading circuit.Average each pixel of this reading circuit is used 2 triodes.Although be not the minimum shared reading circuit of a number, in other side, impayable advantage is arranged.First advantage is that, when sub sampling, by open TG1/TG3 or open TG2/TG4 simultaneously simultaneously, the pixel value Gr of odd-numbered line can be cumulative in FD1 in the mode of addition with the pixel value B of even number line, thereby reach signal plus, the double effects that noise subtracts each other.On average, adopt FD by the mode read again after two signal fused, signal to noise ratio can be increased to many

doubly, wherein 2 times of enhancings that come from signal,

doubly come from weakening of noise.Under contrast, the method that two signals are averaged before, can only, by weakening the mode of noise, be increased to signal to noise ratio many

doubly.Equally, by, controlling sequential and open TG1/TG4/TG5 or TG2/TG3/TG5 simultaneously, we also can read the pixel on diagonal in the mode of signal plus.Second advantage is, when full figure is sampled, can be by the next line pixel value being deposited to the way of FD the inside, thus realize that interlacing or inter-bank read, as shown in figure 21.

What Figure 19 showed is a kind of 6 shared reading circuits for 3 double-deck sensor devices of grand pixel honeycomb arrangement pattern of the present invention.In this reading circuit, three pixels of top layer share one and read electric capacity FD1, and three pixels of bottom share one and read electric capacity FD2, and 3 of top layers share one with 3 of bottoms and amplify and reading circuit.This top layer and bottom share the method for reading circuit, can simplified design, and when sub sampling, allow control logic become simple.With Figure 12 difference, be, the electric capacity that reads of top layer and bottom is not shared, thereby is convenient to the making of double-sided light sensitive device.

What Figure 20 showed is a kind of 8 shared reading circuits for 4 double-deck sensor devices of grand pixel square formation Pareto diagram of the present invention.In this reading circuit, four pixels of top layer share one and read electric capacity FD1, and four pixels of bottom share one and read electric capacity FD2, and 4 of top layers share one with 4 of bottoms and amplify and reading circuit.With Figure 13 difference, be, the electric capacity that reads of top layer and bottom is not shared, thereby is convenient to the making of double-sided light sensitive device.

Obviously, for the two-sided double-deck sensor devices, four grand picture elements of top layer and bottom also can adopt respectively two FD bridge-types as schematically shown in Figure 18 to share reading circuit, thereby allow the relatively independent and mode that can take interlacing or inter-bank to read separately of the reading circuit of top layer and bottom, the shutter speed when accelerating full figure and taking pictures.

What Figure 21 showed is the sampling reading circuit shown in Figure 17, for the interlacing when full figure is sampled, reads (Figure 21 (a)) or inter-bank reads the schematic diagram of (Figure 21 (b)) mode.

In the interlacing of Figure 21 (a) is read, read the first row (GrRgGrR ...) time, the second row (BGBG ...) value, after the value of the first row relevant position reads, and then move on to vacant FD interval.That is to say, when the N row pixel of the first row is being read, the individual pixel of the N-1 of the second row (or N-2, etc.) is being moved in the FD zone of vertical correspondence.After the first row runs through, and then we do not go to read to exist the value in the second row in FD, but then read the pixel value of the third line.Equally, when reading the pixel value of the third line, the pixel value of fourth line also is transferred in the FD zone simultaneously.That is to say, the pixel value of all even number lines, all moved into the FD buffer area seriatim, until the pixel value of odd-numbered line is run through entirely.Finally we read the pixel value in the even number line in the FD buffer area more in order line by line.

In the inter-bank of Figure 21 (b) reads, read the first row (GrRgGrR ...) time, the second row (BGBG ...) value, after the value of the first row relevant position reads, and then move on to vacant FD interval.That is to say, when the N row pixel of the first row is being read, the individual pixel of the N-1 of the second row (or N-2, etc.) is being moved in the FD zone of vertical correspondence.After the first row runs through, when and then we do not read the pixel value of the third line, but then read the pixel value of fourth line, and the pixel value of the third line is transferred in the FD zone simultaneously.That is to say, the row reading order according to Isosorbide-5-Nitrae, 5,8 ..., 2,3,6,7, such order.The benefit of Figure 21 (b) is, the image of front half frame, be still the Bayer arranged in patterns, thereby the little figure preview of the moment that can be used for obtaining fast taking pictures.

Interlacing reading manner shown in Figure 21 or inter-bank reading manner are different from the field scan mode in television system before.Different places mainly are, the sensitive time and the front field that are stored in the rear half frame images in buffer memory (FD) zone are about the same, thereby on effect, shutter speed is read and is doubled than line by line, but odd field and the situation in even field time in upper evening one (field) of having avoided the field scan mode of television system to bring.This situation is only suitable for, in the situation of capturing, being not suitable for continuous video record certainly.

Adopting interlacing reading manner or the inter-bank reading manner electronic shutter speed when taking pictures is very valuable means doubling in a flash.For example, suppose that it is 96MHz that pixel reads clock, and sensitive chip has the 8M pixel, so, the electronic shutter speed when clapping full figure is (96/8)=12fps, or 1/12 second.And the interlacing reading manner shown in employing Figure 19 or inter-bank reading manner, we just can be by interested that single frames, moment rises to 24fps, or 1/24 second, speed is fast one times.Reach the space shooting shutter speed of 1/24 second, mean the mechanical shutter that can save on the mobile phone camera module, and the speed of taking pictures of 1/12 second needs mechanical shutter because of hand, not tremble generation scalloping.

What Figure 22 showed is a kind of simplification disposition of a double-deck sensor devices when sub sampling: the first merging process is at first from the color pixel of bilevel redundancy, adopt the method that merges or give up, only retain the necessary color component of colour reconstruction, for example, Cy, Mg (being that B and R merge gained), G, with Ye. the 3rd merging process and then utilize the means of color space shown in Figure 16 conversion, by tetra-pixels of adjacent CyYeMgG, be converted to a YUV color, again adjacent UV component is done to 2 minutes sub samplings on horizontal direction, just obtain the YUYV422 image.This process has completed the sub sampling of a 2x2.If image is still too large, so, carrying out color space CyYeMgG to before YUV conversion, first do in can also the second merging process the CyYeMgG same hue on average rather than the full sample mode in employing figure.

This double-deck sensor devices, when full figure is sampled, can give up to fall some pixels, after also can all reading, by back-end processor, is decided and how to be processed.While all reading, data volume is many one times.So, if adopt interlacing or the capable reading manner of bar shown in Figure 18 and Figure 20, the frame per second of the moment of so just taking pictures doubles, the same with existing individual layer sensor devices speed.

Figure 22 also is enough to show, two-layer or complexity and the rich and varied property of multilayer sensor devices when sub sampling.Because two-layer or multilayer sensor devices have the possibility more than several thousand kinds on the COLOR COMPOSITION THROUGH DISTRIBUTION of grand pixel, so, sub sampling also just correspondingly has more kinds of possibilities.We can only enumerate a few method at this and set forth marrow of the present invention.

What Figure 23 showed is a kind of simplification disposition of another kind of double-deck sensor devices when sub sampling: the first merging process is by pixel addition (or average), at first it obtain the grand pixel of CyYeMgB, then, the 3rd merging process obtains the YUV color by these 4 by versicolor relation, thereby realizes the sub sampling of 2x2.Certainly, before color conversion, (in the mode of signal averaging) that also can first to the grand pixel of CyYeMgB, do same hue at the second merging process merges rather than adopts full sample mode, thereby obtains the sub sampling of higher multiple.Obviously, in figure, the grand pixel of CyYeMgB also can use the grand pixel of the similar Bayer pattern of BRGB to replace.Here, we illustrate with CyYeMgB, use CyYeMgB, CyYeMgG, and CyYeMgR, can obtain YUV or rebuild RGB.That is to say, CyYeMgG is the special case of CyYeMgX, and wherein X can be R, G, or B.

What Figure 24 showed is a kind of simplification disposition of another kind of double-deck sensor devices when sub sampling: the first merging process is first done merging by adjacent image point along continuous straight runs (in the mode of signal averaging or addition), then the pixel of the second merging process after merging again, (with signal averaging or addition, or abandoning capable mode) vertically merges.By suitable sequencing control, the merging of horizontal direction and vertical direction can complete simultaneously.This sub sampling mode, not only more rich and varied than existing sub sampling mode, much better signal to noise ratio also is provided.

Figure 25 shows is to realize that pixel of the present invention reads the principle system block schematic diagram with a kind of sensor devices of sub sampling circuit, is used for setting forth the implementation method of various functional module of the present invention in sensor devices.This principle system comprises cell array, the row address decoding controller, the column address decoding controller, sampling control circuit, amplify and analog-to-digital conversion module color transformed and sub sampling and image processing module, output control module, chip total control module (the CC module in Figure 25), and other possible module.Pixel reads the function with sub sampling, will be mainly by row address decoding controller and column address decoding controller produce corresponding control signal (row selects signal Row[i], row is controlled vector signal RS[i], column selection signal Col[j], row are controlled vector signal T[j], wherein i and j distinguish corresponding line label and row label) complete.The coordination of other module of system, will mainly be completed by the chip total control module.The 3rd merges sampling process, if any, will in color transformed and sub sampling and image processing module, complete.

Figure 26 is the relation of each control signal in Figure 25 (row choosing, row is controlled vector, column selection, row are controlled vector) and control signal on corresponding photosensitive pixels that illustrates with a concrete example (photosensitive pixels shown in Figure 17).Figure 26 describes is Gr pixel in Figure 17 and the Signal share situation (TG5 has been omitted) of B pixel.Row selects signal Row[i] and Col[j] know and mark.In this circuit diagram, reset signal RS1 and transfer gate control signal RS2 (TG1 or TG3) belong to capable control signal.Notice, RS1 is that two row are shared, RS2 every row (for example has one, TG1 belongs to RS[i], and TG3 belongs to RS[i+1]). the TG5 in Figure 17 (omitting in Figure 26) belongs to row control signal T[j]. that is to say, as much as possible pixel is only done to line operate (pixel with a line is identical) and row operations (pixel of same row is identical), and do not do the different operation of each pixel, to reduce complexity.

In the embodiment of back, we illustrate in connection with Figure 25 and Figure 26, and how sampling of the present invention and sub-sampling methods realize.

Embodiment

The multi-optical spectrum light-sensitive device of the specific embodiment of the invention, different reading with the circuit of sub sampling can realize by being similar to the circuit shown in Figure 25, comprise: include the cell array of a plurality of grand pixels, row address decoding controller, column address decoding controller, sampling control circuit, amplify and analog-to-digital conversion module color transformed and sub sampling and image processing module, output control module, chip total control module (the CC module in Figure 25), and other possible module.

First as required, arrange and take four points or 3 grand pixels that pixels are basis according to square formation or honeycomb shape.These pixels, can be passive pixel, can be also the active pixel, can be with and read electric capacity FD, also can not be with and read electric capacity FD.

In the preceding article, we have divided into the sub sampling process the first merging sampling process, the second merging sampling process and the optional the 3rd merges sampling process.Corresponding to these processes, we adopt respectively the first merge cells, the second merge cells and the 3rd merge cells to realize above-mentioned several merging sampling process.Certainly, these unit are a kind of Module Division to device from the angle of its practical function just, angle from physical device, these functional units can be to realize its function by a mould physically, can be also that a plurality of module combinations physically realize its function, or these functional units be integrated in the module of a physics.In a word, the description of the first merge cells herein, the second merge cells and the 3rd merge cells, just a kind of description its function, and its implementation physically of unspecific restriction.

Concrete, in this example, what realize required sub sampling function is row address decoding controller and column address decoding controller.The row address decoding controller will be exported two class signals, and row selects signal Row[i] (line of every row) and row control vector signal RS[i] (every row one or more line), wherein i is the label of row.Similarly, the column address decoding controller will be exported two class signals, and row are washed signal Col[i] (line of every row) and row control vector signal T[i] (every row one or more line), wherein j is the label of row.

Row selects signal Row[i] be for doing capable selection, and column selection signal Col[j] be for doing the selection of row.This is two groups of relative standards' signal.Row is controlled vector signal RS[i] be the expansion (line of every row expands to many lines of every row) to the existing capable control signal of CMOS, and row are controlled vector signal T[j], some CMOS sensor devices do not have at all, even have, are that row only have one yet.It is example that Figure 26 be take the photosensitive pixels of Figure 17, provided Row[i], RS[i], Col[j], and T[j] a concrete realization, Row[i wherein] be that two row are shared, and RS comprises two capable control signal RS1[i] (reset signal, same two row are shared) and RS2[2] (electric charge transfer control signal).

In the present invention, several row are chosen simultaneously simultaneously, and also several row are chosen simultaneously simultaneously, and even several row and several row are chosen simultaneously.Although in some technology before (as U.S. Pat 6,801,258B1, US6,693,670B1, US7,091,466B2, US7,319,218B2 etc.), have equally several row or several row are chosen simultaneously, but due to the mode difference that merges sampling, thereby it is diverse that row selects the sequential of signal and column selection signal and wave mode.For example, when the merging in carrying out Figure 14 (a) is sampled, the first row first row and the second row secondary series are chosen simultaneously, and this situation is impossible occur in sub-sampling methods before.

RS[i] and T[j] be for controlling resetting of photosensitive pixels, zero clearing, control sensitive time, and electric charge shifts, quadratic deviation, and pixel reads.Due to the symmetry of ranks, RS[i] and T[j] a variety of concrete implementations are arranged.TG1-TG5 as shown in figure 17, V _b1-V _b4deng signal, also have the RS shown in Figure 18, S, and SF signal, be RS[i] and T[j] will comprise.The present invention is not subject to the restriction of the specific implementation of these signals.

More specifically, when doing the sub sampling of arbitrary MxN factor, (M >=2, N >=2), at first do two row or two row, or the first merging sampling process of the merging sampling of two row and two row, and then complete the sub sampling of the capable x N row of M on the basis of the first merging sampling.

Sub sampling (total MxN sub sampling) after the first merging sampling, second merges sampling process, can complete alone or in combination by following variety of way: directly output to bus signal automatic average mode, slip a line or jump row mode and sample mode one by one.And the 3rd merging sampling process if any, can complete by following two kinds of modes: colour space transformation mode and rear end digital image scaling mode alone or in combination.

We know, in a cell array, contain considerable photosensitive pixels, and especially for bilayer or multilayer sensor devices, the kind of color and geometrical distribution are just very abundant.Obviously, first merges sampling process also merges pixel for a plurality of first accordingly, thereby, when carrying out the first merging sampling process, these the first merging pixels are said from the angle of the color merging of pixel, it is various as the Colour selection merged, can adopt the mode of homochromy merging, the mode of heterochromatic merging, (the partial-pixel color is identical to mix the merging mode, part is different), also also can optionally lose unnecessary color.

The conversion of color space comprises the conversion of RGB to the CyYeMgG space, and CyYeMgG is to the conversion of yuv space, and RGB is to the conversion of yuv space.

Notice the conversion of RGB to the CyYeMgG space, can complete in the analog signal space, also can complete at digital space, therefore, it can be at the first merging process, the second merging process, or do in the 3rd any one process merged in sampling process.But CyYeMgG, to the conversion of yuv space and RGB to the conversion of yuv space, can only do in the digital signal space, that is to say and can only do in the 3rd merges sampling process.

More specifically, the cell array part, be comprised of the grand pixel of one group of arrange by square formation three of plural number or four basic pixel.Wherein the basic pixel in grand pixel can be comprised of passive pixel, also can by not with the 3T of FD initiatively pixel form, also can by the 4T with a FD initiatively pixel form.

If in grand pixel, basic pixel adopts the 4T active pixel with FD, its reading circuit can adopt 4-point sharing mode (Figure 11), 6-point sharing mode (Figure 12), and 8-point sharing mode (Figure 13).

More preferably, each grand pixel can be comprised of the active of the 4T with two lighttight FD pixel, and now, its reading circuit can adopt 4-point bridge-type sharing mode (Figure 18).Correspondingly, this kind of sensor devices carrying out two row or two row for the first time, or the color during merging samplings sub sampling of two row and two row is while merging, employing be the mode of electric charge addition.What this grand pixel was back lines by line scan, and interlacing or inter-bank read the full figure sample mode possibility is provided.

For bilayer or multilayer sensor devices, except the Colour selection of the first merging sampling process is more rich and varied, when when each grand pixel, by the 4T with two FD, initiatively pixel forms, its reading circuit can also adopt 4-point bridge-type sharing mode (Figure 18), 6-point bridge-type sharing mode (Figure 19), or 8 bridge-type sharing modes (Figure 20).Correspondingly, this kind of sensor devices carrying out two row or two row for the first time, or the color during merging samplings sub sampling of two row and two row is while merging, employing be the mode of electric charge addition.

Notice, N signal, when the mode that adopts the electric charge addition merges, is limited in the improvement of signal to noise ratio

times, and N signal is limited in the improvement of signal to noise ratio when the mode that adopts signal averaging merges

doubly.Secondly, the sensor devices of this 4 shared two FD (or two row pixels share a line FD) except lining by line scan normally, outside reading line by line, also can be taked to line by line scan when carrying out the full figure sampling, the mode that interlacing is read.

For instance, when full figure is sampled, row address decoding controller and column address decoding controller, will be according to the image-region requirement of asking for, successively first by being about to Row[i] and RS[i] value is set to high or low according to device agreement, and then successively by row output Col[j] and T[j] value be set to high or lowly, make needed pixel (charge/voltage) value, can (process read/write circuit) output on output bus according to reading order.

When sub sampling, (row dwindles M doubly to the MxN decimation factor of supporting for each, row dwindle N doubly), row address decoding controller and column address decoding controller, according to MxN decimation factor and image-region requirement, corresponding each output row, to need the corresponding all Row[i of row that merge simultaneously] and RS[i] value is set to high or low, and corresponding each output is listed as simultaneously, all Col[j that then will need the row that merge] and T[j] value be set to high or low, make all pixel (charge/voltage) values that merge that need, can according to reading order (through read/write circuit), output on output bus simultaneously.Simultaneously, in the time of necessary, row address decoding controller and column address decoding controller also will be according to MxN decimation factor and image-region requirements, carry out the necessary operation of slipping a line and jumping row.

Different MxN decimation factors, may obtain different colors in the different time on output bus.Correspondingly, other system function module, as amplified and analog-to-digital conversion module, color transformed and sub sampling and image processing module, also have output control module, all needs to do corresponding co-ordination.Total control of this system, can complete by chip total control module (the CC module in Figure 25).Notice, except amplification and analog-to-digital conversion module and cell array, other module, be all mainly digital processing circuit, thereby can in device periphery, realize with comparalive ease, thereby make the wiring of sensor devices relatively simple.

Below we realize in connection with the reading circuit as Figure 26 of the photosensitive pixels shown in Figure 17, and other module of the sensor devices shown in Figure 25 provides a more specifically signal controlling flow process.

At first, carry out zero clearing and sensitization and control: it is by Vb1 and the whole zero setting of Vb2 that a kind of simple zero clearing is controlled, and this requires Vb1 and Vb2 to become the signal that row is controlled vector.Another kind method is, first FD1 and FD2 are resetted (the RS1 zero setting in Figure 26), and TG1 and TG2 open (RS2 in Figure 26 sets high) simultaneously, and the electric charge in Gr in photosensitive pixels and R is fallen clearly.After this, RS1 is set high to RS2 zero setting.After this, under the irradiation of light, the light sensitive diode of Gr and R starts stored charge.

When needs read the electric charge in Gr, three kinds of ways are arranged.The first is directly by TG1/RS2 and Row[i] open, the electric charge in Gr is transferred in FD1, and therefrom (conversion by electric charge to voltage) reads the charge value in Gr.The second way is, then the final step of the first way read the charge value in Gr after, then FD1 is resetted, and reads the electric charge (voltage) of FD1 under reset mode, in order to be used for, the charge value of the Gr that just read is carried out to correlated sampling.The third is before the charge value in reading Gr, first FD1 is carried out to reset samples, and this method can be disturbed the value in Gr, thereby less than the second.Now, the column selection signal Col[j that the column address decoding controller must be corresponding by Gr] open to the measurement (may be twice, be once wherein the measurement under reset mode) of Gr output to amplify and analog-to-digital conversion module in go.

According to Row[i], Col[j], and RS2[i] value, chip total control module CC can calculate the color of the pixel read, and it is processed accordingly.Different colors, may enter different amplifying circuits, and carry out different analog-to-digital conversion process and obtain digital signal.

The digital signal of photosensitive pixels will be put buffer memory into, by color transformed and sub sampling and image processing module, do further processing.In the situation that sub sampling is not done in the full figure sampling, for large array image sensor devices, usually do not do any color transformed yet.Thereby, under this pattern, chip total control module CC can make corresponding control, allow the digital signal of photosensitive pixels skip color transformed and sub sampling module, directly enter into image processing module.After the image included through sensor devices is processed, then outputed to the external interface of sensor devices by output module.

When full figure is sampled, it should be noted that and line by line scan, interlacing or inter-bank reading manner.In the case, reset zero clearing and the sensitive time of odd-numbered line and even number line control and carry out simultaneously.When interlacing is read, after the pixel of even number line (the first row) is all read, the row address decoding controller is not to remove to select to read next line immediately, but, first the odd-numbered line of back (the second row) is transferred in the shared FD of FD and even number line and gone, and then start reading of the third line.When inter-bank reads, if the label of the first row since 0, so, the reading order of the row of front field is 0,3,4,7,8,11,12,15 ..., then the reading order of field is, 1,2,5,6,9,10,13,14 ...Certainly more complicated order can also be arranged.The row striden across in the middle of when front field is read in centre, first be placed in the FD used once and keep in, and by the time reads lower field and read.

Line by line scan, the difference of the field scan mode in interlacing or inter-bank mode and traditional tv is, of the present invention, lines by line scan, and in interlacing or inter-bank mode, the sensitization sequential of pixel is carried out fully line by line, and the field scan mode is not.

When sub sampling, situation can be complicated a lot.But, for a concrete sensor devices, sub sampling factor M xN can only support a few.Correspondingly, chip total control module CC, the row address decoding controller, and the column address decoding controller can only be considered the MxN sub sampling factor of supporting.For example, the sensor devices of 500 everything elements, can only consider to support 2x2,2x1,4x4, four kinds of situations of 8x8.

Second merges sampling process does not relate to the electric charge addition usually, thereby what usually adopt is following three modes: directly output to bus signal automatic average mode, slip a line or jump row mode, sample mode one by one.These three modes are all very traditional and simple, are well known to those skilled in the art, and at this, do not repeat.The 3rd to merge sampling process be to complete in the digital picture space, employing be also relative standard's digital image scaling technology.Below we only do the careful explanation of signal controlling flow process to the first merging sampling process so that using method of the present invention is more prone to understand.

For grand pixel as shown in figure 17, in the first merging process, two kinds of merging modes may be arranged: first kind of way is that Gr and B merge, and R and Gb merge; The second way is that Gr and Gb merge, and R and B merge.

Merge mode for the first, in chronological order:

1.t0 constantly: the row address decoding controller can be by the zero setting of the RS1 corresponding to FD1 (resetting) as shown in figure 26.

2.t1 constantly: open TG1 and TG3 (RS2[i] and RS2[i+1]), by the electric charge of Gr and B light sensitive diode (PD), transfer in FD1 simultaneously.Now, RS1 can set high.

3.t2 constantly: and then open row and select Row[i] and column selection Col[j] (supposition Gr is in the capable j row of i), the electric charge in FD1 (magnitude of voltage) is outputed on output bus.

4.t3 constantly: the null value next also can read in FD1 is used to do correlated sampling.

The operation of first two steps can be carried out i and the capable all pixels of i+1 simultaneously, and the pixel after the third and fourth step is combined is read successively.Therefore, if do not do correlated sampling, an average clock pulse can read a pixel, if do correlated sampling, average two clock pulse can read a pixel.This carries out according to the pixel location order of priority.This merging mode, also can carry out according to following color order of priority.

Merge mode for the second, the sequential situation is complicated a lot.Two kinds of processing modes are at this time arranged.The first processing mode is to be undertaken by the color order of priority: first the Gr of full line and Gb merging sampled, then carried out the merging sampling of B and R, or by reversed sequence.This mode is fairly simple, and the time sequencing of control signal is as follows:

5.t0 constantly: the row address decoding controller can be by the zero setting of the RS1 corresponding to FD1 and FD2 (resetting) as shown in Figure 17 and Figure 26.

6.t1 constantly: open TG1 and TG4 (RS2[i] and RS2[i+1]), by the electric charge of Gr and Gb light sensitive diode (PD), transfer to respectively in FD1 and FD2 simultaneously.Now, RS1 can set high.

7.t2 constantly: open TG5, the electric charge in FD2 is transferred in FD1 and gone

8.t3 constantly: and then open row and select Row[i] and column selection Col[j] (supposition Gr is in the capable j row of i), the electric charge in FD1 (magnitude of voltage) is outputed on output bus.

9.t4 constantly: the null value next also can read in FD1 is used to do correlated sampling.

The operation of first three step can be carried out i and capable all Gr and the Gb pixel of i+1 simultaneously, and the pixel after the third and fourth step is combined is read successively.Therefore, if do not do correlated sampling, an average clock pulse can read a pixel, if do correlated sampling, average two clock pulse can read a pixel.This reading manner has destroyed the natural ordering of pixel opsition dependent, needs back-end processing to proofread and correct.In order to keep consistency, the first merges mode, also can be undertaken by the color mode of priority.

The second processing mode is that the opsition dependent order of priority carries out: first first Gr and Gb merging have been sampled, then carried out the merging sampling of first B and R, so repeatedly.The time sequencing of the control signal of this mode and top the first processing mode are similar, but can only serial process between pixel, can not parallel processing.In other words, processing first t0-t5 that merges pixel constantly, can not process second and merge pixel.This requires higher system clock.Fortunately, carrying out after the sub sampling, the quantity of pixel greatly reduces, and therefore, system clock frequency also is unlikely to Gao get Tai and goes against accepted conventions.

For the preferred circuit in the present invention is realized, when sub sampling, the correlated sampling effect is little, thereby can save.Thereby top sequential is just simple a lot.

For washing fixed pixel sampling order, chip total control module CC will correspondingly control and amplify and analog-to-digital conversion module, and color transformed and sub sampling and image processing module also have output control module, in order to different colors is done to different processing.More detailed introduction has exceeded category of the present invention.

Existing sub sampling mode is mainly to carry out between the pixel of same hue, and the main pixel adopted is average and slip a line or jump the mode of row.These methods, for colourful bilayer or multilayer sensor devices, just seem and too limit to and be pale and weak.The sub-sampling methods that the present invention proposes by the mode of colour space transformation, can be carried out between same hue, also can between different color, carry out, and also can mix and carry out (part is carried out between same hue, and part is carried out between different color).In addition, the signal of the electric charge addition that the present invention proposes merges mode, only n signal need be merged, and just can reach close to n ³the effect of individual signal plus.Thereby sub-sampling methods of the present invention is compared existing sub-sampling methods, can there is better picture quality, special, when the present invention is used for to bilayer or multilayer sensor devices, just can produce countless simple and outstanding sub sampling mode.

Above we just take individual layer and double-deck sensor devices and minority 3T/4T initiatively pixel be example, marrow of the present invention and content are described.The condition that these are concrete is not restriction of the present invention.On the contrary, the present invention is used for to more complicated design, as 5T/6T active pixel, or the multilayer sensor devices, its effect may be more obvious.

Claims (21)

1. a multi-optical spectrum light-sensitive device, is characterized in that, comprises the cell array of arranging with row and column, and

The first merge cells, merge sampling between the pixel for the colleague's heterotaxy of the next-door neighbour to described cell array, different row same column or different row heterotaxy in twos, obtains the sampled data of the first merging pixel, wherein, at least comprises the merging between the different color pixel;

The second merge cells, merge sampling for the first sampled data that merges pixel that the first merge cells is obtained, and obtains the sampled data of the second merging pixel;

The quadratic deviation mode of described the first merge cells or the second merge cells is to shift the signal averaging mode between the electric charge accumulate mode realized or two different color pixels by electric charge between identical or different color pixel, and the quadratic deviation mode between the different color pixel is in accordance with the mode of colour space transformation, to meet the requirement of colour reconstruction.

2. multi-optical spectrum light-sensitive device as claimed in claim 1, is characterized in that, also comprises the 3rd merge cells, for the second sampled data that merges pixel that the second merge cells is obtained, merges sampling, obtains the sampled data of the 3rd merging pixel.

3. multi-optical spectrum light-sensitive device as claimed in claim 1, is characterized in that, described electric charge accumulate mode completes in reading electric capacity (FD).

4. multi-optical spectrum light-sensitive device as claimed in claim 1, it is characterized in that, the merging sample mode based on color of described the first merge cells or the second merge cells comprises homochromy merging mode, heterochromatic merging mode, mix merging mode or selectivity abandons unnecessary color and merges mode, and the merging sample mode that adopts with the second merge cells of the first merge cells is homochromy merging mode when different.

5. multi-optical spectrum light-sensitive device as claimed in claim 1, it is characterized in that, the location-based merging sample mode of described the first merge cells or the second merge cells comprises at least one in following several mode: directly output to bus signal automatic average mode, slip a line or jump row mode and sample mode one by one.

6. multi-optical spectrum light-sensitive device as claimed in claim 2, is characterized in that, the described the 3rd merging sample mode that merges sampling unit comprises: at least one in colour space transformation mode and rear end digital image scaling mode.

7. as claim 1 or the described multi-optical spectrum light-sensitive device of 6 any one, it is characterized in that, described colour space transformation comprises the conversion of RGB to the conversion in CyYeMgX space, RGB to yuv space, or CyYeMgX is to the conversion of yuv space, wherein X is any in R (red), G (green), B (orchid).

8. multi-optical spectrum light-sensitive device as claimed in claim 1, is characterized in that, described cell array is comprised of a plurality of grand pixels that comprise at least one basic pixel, and wherein basic pixel can be passive pixel or active pixel.

9. multi-optical spectrum light-sensitive device as claimed in claim 8, is characterized in that, the basic pixel in described grand pixel is by square formation or honeycomb arrangement.

10. multi-optical spectrum light-sensitive device as claimed in claim 8 or 9, it is characterized in that, the building form of described grand pixel comprise following building form at least one: not with the 3T that reads electric capacity (FD) initiatively the pixel building form, with a 4T who reads electric capacity (FD) pixel building form initiatively.

11. multi-optical spectrum light-sensitive device as claimed in claim 10, is characterized in that, described with a 4T active pixel that reads electric capacity (FD), adopts 4 sharing modes, 6 sharing modes or 8 sharing modes.

12. multi-optical spectrum light-sensitive device as claimed in claim 8, it is characterized in that, described grand pixel comprises following building form: four basic pixel and two lighttight electric capacity (FD) that read that are positioned at two interlines of arranging by square formation, consist of, the pixel of lastrow and the pixel of next line share one and read electric capacity (FD), two are read between electric capacity (FD) and can realize that electric charge shifts, and read on electric capacity and be connected with reading circuit at least one.

13. multi-optical spectrum light-sensitive device according to claim 8, it is characterized in that, described grand pixel is with 2 shared, 3 shared or 4 the shared 3T of electric capacity (FD) or basic pixel compositions of 4T active pixel of reading, and adopts 4 bridge-type sharing modes, 6 bridge-type sharing modes or 8 bridge-type sharing modes.

14. according to the described multi-optical spectrum light-sensitive device of claim 1, it is characterized in that, the full figure sample mode of described multi-optical spectrum light-sensitive device comprises two kinds of modes, a kind of be line by line scan, row by row system, a kind of be line by line scan, interlacing or inter-bank reading manner.

15. the method for sampling of a multi-optical spectrum light-sensitive device, is characterized in that, comprising:

The first merging process, merge sampling between the pixel for the colleague's heterotaxy of the next-door neighbour to cell array, different row same column or different row heterotaxy in twos, obtains the sampled data of the first merging pixel, wherein, at least comprises the merging between the different color pixel;

The second merging process, merge sampling for the first sampled data that merges pixel that the first merging process is obtained, and obtains the sampled data of the second merging pixel;

The quadratic deviation sample mode of described the first merging process or the second merging process is to shift the electric charge phase add mode that realizes or the signal averaging mode between the different color pixel by electric charge between identical or different color pixel, and the quadratic deviation mode between the different color pixel is in accordance with the mode of colour space transformation, to meet the requirement of colour reconstruction.

16. the method for sampling as claimed in claim 15, is characterized in that, also comprises: the 3rd merging process, merge sampling for the second sampled data that merges pixel that the second merging process is obtained, obtain the sampled data of the 3rd merging pixel.

17. the method for sampling as claimed in claim 15, it is characterized in that, the merging sample mode based on color of described the first merging process or the second merging process comprises homochromy merging mode, heterochromatic merging mode, mix the merging mode or selectivity is abandoned unnecessary color mode, and in the first merging process and the second merging process, at least one merging process is not homochromy merging mode.

18. the method for sampling as claimed in claim 15, it is characterized in that, the location-based merging sample mode that described the first merging process or second merges sampling process comprises at least one in following several mode: directly output to bus signal automatic average mode, slip a line or jump row mode and sample mode one by one.

19. the method for sampling as claimed in claim 16, is characterized in that, the merging sample mode that described the 3rd merging process carries out comprises: colour space transformation mode, rear end digital image scaling mode.

20. the method for sampling as described as claim 15 or 19, it is characterized in that, described colour space transformation comprises that the conversion of RGB to the conversion in CyYeMgX space, RGB to yuv space or CyYeMgX are to the conversion of yuv space, wherein X is any in R (red), G (green), B (orchid).

21. the method for sampling as claimed in claim 15, is characterized in that, the mode of its full figure sampling comprises two kinds of modes, a kind of be line by line scan, row by row system, a kind of be line by line scan, interlacing or inter-bank reading manner.

Images