CN101848343B - Image sensor with integral image output - Google Patents

Abstract

An image sensor with integral image output includes a pixel circuit, a line accumulator, and a volume accumulator. The pixel circuit includes a plurality of pixel elements for extracting pixel values of a plurality of pixels in an image. The line accumulator is coupled to the pixel circuit and used for accumulating pixel values from a first pixel to a target pixel in a target pixel line of an image to obtain a line pixel accumulated value. The line accumulator is coupled to the line accumulator, and is used for accumulating the accumulated value of the line pixels output by the line accumulator to a pixel integral value corresponding to the target pixel in a previous pixel line of the target pixel line to be used as a pixel integral value of the target pixel, and outputting the pixel integral value of the target pixel to form an integral image.

Description

Image sensor with integral image output

technical field

The present invention relates to an image sensor, and more particularly to an image sensor with integrated image output.

Background technique

More and more image recognition (Image recognition) technologies are applied to small embedded systems, such as the face detection (Face detection) function in digital cameras. In addition to the image sensor, such an embedded system is equipped with a high-speed embedded processor for processing the image extracted by the image sensor to recognize image features. In recent years, Haar-like image features have been widely used in the field of image recognition, especially face detection. Using Haar-like image features combined with adaptive gain (Adaptive Boost, AdaBoost) algorithm, the image can be roughly but quickly divided into two groups, and through repeated execution of this classification process, the image can be quickly divided into two groups. It is divided into multiple image clusters to achieve the purpose of image recognition.

The AdaBoost algorithm is currently the most commonly used classifier fusion (Classifier Fusion) method, which can combine multiple weak classifiers (weak classifiers) into a strong classifier (strong classifier). In detail, the AdaBoost algorithm is to give a weight to each piece of data in the original image set, and update the weight of the data according to the classification results of the sub-classifier established by it, and then generate the next sub-classifier according to the weight. The data. Because the new sub-classifier is mainly aimed at reinforcing the insufficient (misclassified) part of the previous classifier, the weight of the data that was wrongly classified in the previous time is increased; otherwise, the weight of the data that is correctly classified is reduced. That is, a new sub-classifier is re-established accordingly. The detailed steps are as follows:

Suppose Dt(i) is the data importance distribution after the tth iteration (iteration), and i refers to the i-th data. At the beginning, it is assumed that the importance of each data is equal, so

D ₁ (i) = 1/m (1)

Among them, m is the number of data samples.

Suppose h is a sub-classifier function and x is a data point, then h _t (x) is the result of this sub-classifier classifying data point x. In addition, assuming that y is the correct classification result (+/-1, dichotomy), if the classification is correct h(x)=y, y*h(x)=+1, then the importance of the data point can be Decrease; if the classification error h(x)≠y, y*h(x)=-1, then the importance of the data point needs to be increased. Furthermore, assuming that α is the weight required to correct the distribution of data importance, and Z is the normalization factor, the weight α after iteration can be calculated by using the error rate ε _t , and then use α and whether the classification is correct to correct the data importance. Sex distribution D(i):

${\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; 1</span>}\mathrm{<span\; class="notranslate">\; no</span>}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{{\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; e</span>}}^{{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span>}}{{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Then use D _t+1 (i) to train a new sub-classification function h _t+1 , after T iterations, the final classifier function is:

$\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; sign</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Finally, according to the number of votes (that is, the classification result H(x)), it is determined which category the test data belongs to.

Figures 1(a)-(d) are schematic diagrams of known Haar-like image features. Please refer to Figure 1(a)~(d), Haar-like image features can be regarded as a group composed of multiple blocks, such as 2 blocks (Figure 1(a)), 3 blocks (Figure 1(b) ) and 4 squares (Figure 1(c)), and their eigenvalues are the sum of the pixel values of all pixels in the area covered by these squares in the image, white is +1, and black is -1. For example, the sum of the pixel values of all pixels in the area covered by the middle white square in the image 100 in the three square groups is subtracted from the sum of the pixel values of all the pixels in the area covered by the left and right black squares in the image 100 (as shown in Figure 1(d) )), this eigenvalue can generally be calculated from the integral image.

For example, FIG. 2( a ) and FIG. 2( b ) are schematic diagrams of known eigenvalue calculation methods. Please refer to Figure 2(a) first, the integral value of pixel P(x ₂ , y ₂ ) in image 200 is defined by the upper left pixel O(x ₁ , y ₁ ) to pixel P(x ₂ , y ₂ ) The sum of the pixel values of all pixels within the square of . Next, please refer to Figure 2(b), the eigenvalue of the square ABCD is AB-C+D, where A is the eigenvalue of the square surrounded by pixel O to pixel A, and B is the characteristic value of the square surrounded by pixel O to pixel B C is the characteristic value of the square formed by pixel O to pixel C, and D is the characteristic value of the square surrounded by pixel O to pixel D.

From the above, it can be known that when recognizing image features, it is necessary to rely on a high-speed processor to perform integral image calculations, and then use this integral image to calculate feature values and perform functions such as image recognition and face detection. The cumbersome calculation process will consume a lot of computing performance of the processor. For example, for an embedded processor in a digital camera that needs to support multiple functions at the same time, it will inevitably cause a huge burden.

Contents of the invention

In view of this, the present invention provides an image sensor with integral image output, which can provide an output format of integral image.

The present invention proposes an image sensor with integrated image output, which includes a pixel circuit, a line accumulator, and a volume accumulator. Wherein, the pixel circuit includes a plurality of pixel elements for extracting pixel values of a plurality of pixels in the image. The line accumulator is coupled to the pixel circuit and is used for accumulating pixel values from the first pixel to the target pixel in the target pixel line of the image to obtain the line pixel accumulation value. The volume accumulator is coupled to the line accumulator, and is used for accumulating the line pixel accumulation value output by the line accumulator to the pixel integration value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line as the pixel of the target pixel Integral value, and output the pixel integral value of the target pixel.

The present invention proposes an image sensor with integrated image output, which includes pixel circuits, line accumulators and N individual accumulators. Wherein, the pixel circuit includes a plurality of pixel elements for extracting the pixel values of the plurality of pixels in the image, and the pixel circuit can be divided into MxN square areas, wherein M and N are positive integers. The line accumulator is coupled to the pixel circuit, and is used for accumulating pixel values from the first pixel to the target pixel in the target pixel line of each square area of the image to obtain the line pixel accumulation value. N individual accumulators are coupled to the line accumulators, and are used to respectively accumulate the line pixel accumulation values output by the line accumulators to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in each square area Take as the pixel integral value of the target pixel, and output the pixel integral value of the target pixel.

The present invention proposes an image sensor with integrated image output, which includes a pixel circuit, a line accumulator, and a volume accumulator. Wherein, the pixel circuit includes multiple pixel elements for extracting pixel values of multiple pixels in the image, wherein the pixel circuit can be divided into MxN square areas, where M and N are positive integers. The line accumulator is coupled to the pixel circuit and is used for accumulating pixel values from the first pixel to the target pixel in the target pixel line in each square region of the image to obtain line pixel accumulation values. The volume accumulator is coupled to the line accumulator, and is used to accumulate the line pixel accumulation value output by the line accumulator to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in each square area as The pixel integration value of the target pixel, and output the pixel integration value of the target pixel.

The present invention proposes an image sensor with integrated image output, which includes a pixel circuit, a line accumulator, and a volume accumulator. Wherein, the pixel circuit includes multiple pixel elements for extracting pixel values of multiple pixels in the image, wherein the pixel circuit can be divided into multiple square areas, and the divided areas are not limited to equal sizes. The line accumulator is coupled to the pixel circuit and is used for accumulating pixel values from the first pixel to the target pixel in the target pixel line in each square region of the image to obtain line pixel accumulation values. The volume accumulator is coupled to the line accumulator, and is used to accumulate the line pixel accumulation value output by the line accumulator to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in each square area as The pixel integration value of the target pixel, and output the pixel integration value of the target pixel. Wherein, the line accumulator includes resetting when crossing the longitudinal boundary or the lateral boundary of the above-mentioned square area, and the volume accumulator includes resetting when crossing the lateral boundary of the above-mentioned square area.

Based on the above, the image sensor with integrated image output of the present invention adds a group of integration circuits in the image sensor, and can perform an accumulation operation pixel by pixel line for the pixel values received by the image sensor, and Output the accumulated integral image to the back-end processor for use. Accordingly, the load on the processor to calculate the integral image can be reduced.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail with reference to the accompanying drawings.

Description of drawings

Figures 1(a)-(d) are schematic diagrams of known Haar-like image features.

FIG. 2( a ) and FIG. 2( b ) are schematic diagrams of known eigenvalue calculation methods.

FIG. 3 is a block diagram of an image sensor with integrated image output according to an embodiment of the invention.

FIG. 4( a ) and FIG. 4( b ) are examples of calculated integral images according to an embodiment of the present invention.

FIG. 5( a ) and FIG. 5( b ) are examples of calculating feature values according to an embodiment of the present invention.

FIG. 6 is a block diagram of an image sensor with integrated image output according to an embodiment of the invention.

FIG. 7 is a block diagram of an image sensor with integrated image output according to an embodiment of the invention.

FIG. 8 is an example of calculating integral images according to an embodiment of the present invention.

FIG. 9A is a block diagram of an image sensor with integrated image output according to an embodiment of the invention.

FIG. 9B is a block diagram of an image sensor with integrated image output according to an embodiment of the invention.

FIG. 10 is an example of calculating integral images according to an embodiment of the present invention.

FIG. 11( a ) and FIG. 11( b ) are examples of calculated integral images according to an embodiment of the present invention.

FIG. 12( a ) and FIG. 12( b ) are examples of calculated integral images according to an embodiment of the present invention.

FIG. 13( a ) and FIG. 13( b ) are examples of calculated integral images according to an embodiment of the present invention.

FIG. 14 is an example of calculating integral images according to an embodiment of the present invention.

FIG. 15 is an example of calculating integral images according to an embodiment of the present invention.

FIG. 16 is a schematic diagram showing the change of the value of the line accumulator with time according to an embodiment of the present invention.

FIG. 17 is a schematic diagram illustrating changes in volume accumulator values over time according to an embodiment of the present invention.

[Description of main component symbols]

100, 200, 800, 1000, 1210, 1310, 1400, 1500: Image

310, 610, 710, 910: pixel circuit

320, 630, 730, 810, 930, 1010: line accumulator

322, 632, 732, 932: first adder

324, 634, 734, 934: first buffer

330, 640, 940, 1020: volume accumulator

332, 642, 742, 942: second adder

334, 644, 744, 944: Second buffer

510, 1221, 1225: bottom right pixel

520, 1222, 1226: bottom left pixel

530, 1223: upper right pixel

540, 1224: upper left pixel

620, 720, 920: pre-processing unit

650: multiplexer

740, 820: first body accumulator

750, 830: second volume accumulator

752: Third Adder

754: Third buffer

760: multiplexer

936: The first multiplexer

946: second multiplexer

950: third multiplexer

1110, 1220, 1320: sub-integral images

1120: Integral image after restoration

Detailed ways

The image sensor contains millions of pixels, and its design adopts a field-integrated reading system that transmits pixel-by-pixel line. Wherein, the color filter of the image sensor includes color pixel circuits arranged in a line-staggered manner in RGB color space. The output port of the image sensor provides a variety of output formats, such as RGB, YCrCb, YUV, etc., to meet different application requirements of back-end devices. The present invention adds an integral circuit in the image sensor to calculate the integral image of the image extracted by the image sensor, and can provide a new integral image output format to the back-end processor as a face detection For reference purposes.

In detail, the ADABOOST algorithm is usually implemented by software, but it is difficult to implement by hardware, because the algorithm is quite complicated and requires a huge circuit to process data. Accordingly, the present invention directly configures a simple accumulator circuit in the image sensor to calculate integral data, and outputs the calculated data as an integral image. Based on this integral data, the back-end processor only needs to do simple addition and subtraction calculations to obtain the accumulated pixel values of all pixels in the target area, and use it to perform face detection, which can speed up feature comparison. In order to make the content of the present invention more clear, the following describes the detailed implementation of the image sensor with integral image output according to the present invention with regard to the cases of calculating integral images of a single image or divided images respectively.

FIG. 3 is a block diagram of an image sensor with integrated image output according to an embodiment of the invention. Referring to FIG. 3, the image sensor of this embodiment includes a pixel circuit 310, a line accumulator 320 and a volume accumulator 330, and their functions are described as follows:

The pixel circuit 310 includes a plurality of pixel elements for extracting pixel values of a plurality of pixels in an image. The pixels in the pixel circuit 310 may be composed of Charge Coupled Devices (CCD) or Complementary Metal-Oxide Semiconductor Devices (CMOS), without limitation. In addition, the image signals extracted by these pixels can be converted into digital pixel values through an analog-to-digital converter (ADC), and then output to the back-end device for processing and use by the back-end device. It is worth mentioning that, in addition to a plurality of pixel elements, the pixel circuit 310 also includes elements such as scan lines, data lines, gate drivers, and source drivers, which are commonly used elements in known pixel circuits. , so its function will not be repeated here.

The line accumulator 320 is coupled behind the pixel circuit 310, and is used to receive the pixel value of each pixel in the image from the pixel circuit 310, and perform an accumulation action on the pixel values from the first pixel to the target pixel in the target pixel line in the image to obtain Line pixel accumulation value. In detail, the line accumulator 320 includes a first adder 322 and a first buffer 324 . Among them, the first adder 322 accumulates the pixel value of the target pixel received by it to the line pixel accumulation value recorded in the first buffer 324, and the first buffer 324 is used to record the target pixel in the target pixel line until the target pixel Accumulate the accumulated line pixel value, and output the line pixel accumulated value. Wherein, the first buffer 324 is reset after the first adder 322 accumulates the pixel values of all the pixels in a pixel line, and the first adder 322 continues to accumulate the pixel values of the next pixel line, so that each pixel The accumulation of lines can achieve the effect of line integration.

The volume accumulator 330 is coupled after the line accumulator 320, and is used to accumulate the line pixel accumulation value output by the line accumulator 320 to the pixel integration value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line as the target The pixel integral value of the pixel. Wherein, the pixel integral value of the pixel of the image output by the volume accumulator 330 forms the integral image of the image. In detail, the volume accumulator 330 includes a second adder 332 and a second buffer 334 . Wherein, the pixel integral value of each pixel in the previous pixel line of the target pixel line is recorded in the second buffer 334, and its capacity only needs to accommodate the data of the pixel integral value of one pixel line. For the first pixel line, the recorded pixel integral value starts to accumulate from zero. The second adder 332 is to accumulate the line pixel accumulation value up to the target pixel in the target pixel line output by the line accumulator 320 to the pixel integral of the pixel corresponding to the target pixel in the previous pixel line recorded in the second buffer 334 value to get the pixel integral value of this target pixel.

It is worth mentioning that the so-called pixel corresponding to the target pixel in the previous pixel line is the pixel in the same row as the target pixel, that is, the pixel above the target pixel. After the second adder 332 calculates the target pixel After the pixel integral value of the pixel is output, the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line recorded in the second buffer 334 is replaced by the pixel integral value of the target pixel as the next time It is used to calculate the pixel integral value of the next scan line.

In addition, the pixel integral value of the target pixel in this embodiment is directly output after the calculation by the second adder 332 in the volume accumulator 330 is completed. In another embodiment, the pixel integral value of the target pixel may also be stored in the second buffer 334 first, and then sequentially output by the second buffer 334, or wait for the second buffer 334 to record a whole After the pixel integration value of each pixel in the target scan line, it is output again without limiting its range.

Regarding the process of calculating the integrated image by the above image sensor, an example is given below to illustrate. FIG. 4( a ) and FIG. 4( b ) are examples of calculated integral images according to an embodiment of the present invention. Please refer to Figure 4(a) and Figure 4(b) at the same time, assuming that Figure 4(a) is the original image (Rawimage) extracted by the image sensor, and Figure 4(b) is the original image of Figure 4(a) Integral image of the image. Wherein, when the image sensor calculates the pixel integral value of the third pixel of the fourth scan line, the line accumulator first accumulates the pixel values of the first pixel to the third pixel of the fourth scan line in the original image. Pixel values (that is, 1, 4, 0), and the line pixel accumulated value 1+4+0=5 is obtained. Then, the volume accumulator adds the accumulated pixel value of this line to the pixel integral value of the corresponding pixel (ie, the third pixel) in the previous scan line (ie, the third scan line), and obtains the pixel integral value of the target pixel. +14=19. This pixel integral value updates the pixel integral value of the corresponding pixel in the previous scan line recorded in the volume accumulator, and is used as the pixel for calculating the corresponding pixel (ie the third pixel) in the next (ie, the fifth scan line) Points are used.

The integral image calculated by the above example is then output to the back-end processor. At this time, the back-end processor only needs to perform simple addition and subtraction operations to obtain the desired feature value of the image feature. FIG. 5( a ) and FIG. 5( b ) are examples of calculating feature values according to an embodiment of the present invention. Please refer to Figure 5(a) and Figure 5(b) at the same time, assuming that the processor wants to obtain the eigenvalues of the square A in the original image, the edge pixels corresponding to the four corners of the square A can be found from the integral image respectively, namely The pixel integral values of the lower right pixel 510 , the lower left pixel 520 , the upper right pixel 530 and the upper left pixel 540 are added and subtracted to obtain the characteristic value 0+18−6−6=6 of the block A. This integral value is the sum of the pixel values of all the pixels of square A in the original image 0+2+1+1+1+1=6.

Through the structure of the above-mentioned image sensor, the integrated value of the pixel can be calculated at the first time when the image sensor extracts the pixel value, and then the format of the integrated image can be output for use by the subsequent processor. It is worth mentioning that, in order to improve the flexibility of the image sensor output image, the present invention also includes configuring a pre-processing unit and a multiplexer in the image sensor, so as to provide a variety of different output image formats, as follows An example will be given in detail.

FIG. 6 is a block diagram of an image sensor with integrated image output according to an embodiment of the invention. Referring to FIG. 6, the image sensor of this embodiment includes a pixel circuit 610, a pre-processing unit 620, a line accumulator 630, a volume accumulator 640 and a multiplexer 650, and its functions are described as follows:

The pixel circuit 610 is used to extract pixel values of multiple pixels in the image, and the pre-processing unit 620 performs pre-processing such as space correction or distortion correction on the pixel values output by the pixel circuit 610 .

The line accumulator 630 includes a first adder 632 and a first buffer 634, which are coupled after the pre-processing unit 620, for receiving the pixel values of the pre-processed pixels from the pre-processing unit 620, and for The pixel values from the first pixel to the target pixel in the target pixel line are accumulated to obtain the line pixel cumulative value.

The volume accumulator 640 includes a second adder 642 and a second buffer 644, which are coupled after the line accumulator 630, and are used to accumulate the line pixel accumulation value output by the line accumulator 630 to the previous pixel line of the target pixel line The pixel integral value of the pixel corresponding to the target pixel is used as the pixel integral value of the target pixel.

The multiplexer 650 is coupled to the pre-processing unit 620 and the volume accumulator 640, which can choose to directly output the pixel value of each pixel in the image, or output The integrated pixel value after accumulation.

Through the structure of the above-mentioned image sensor, a variety of output image formats can be provided for the user or the back-end processor to choose, that is, when the general mode is selected, the extracted original image is output, and when the face detection mode is selected , the integral image of the extracted image is output, which can provide flexibility in the selection of the output image.

The image sensors in the above embodiments all carry out integration operations on the images of a single color extracted by them. and other color spaces) to perform integral operations on the data, and then obtain integral images of multiple color spaces. In detail, a set of line accumulators and volume accumulators can be configured in the image sensor for the original image of each color space extracted by the pixel circuit to calculate the integral image of the image in the color space and output to Backend processor usage. Wherein, the manners of calculating the integrated image by the line accumulator and the volume accumulator are the same as or similar to those of the foregoing embodiments, and thus will not be repeated here.

In addition, in order to reduce the calculation amount of the line accumulator and the volume accumulator in the image sensor, so as to speed up the calculation speed of the integral image, the present invention also includes dividing the image into multiple square areas, and then calculating the integral image for each square area , and combined into a new integral image format for output to the back-end processor. After the processor obtains the integrated image in this format, it only needs to perform simple addition and subtraction operations to obtain the feature value of a specific block in the image. Another embodiment will be described in detail below.

FIG. 7 is a block diagram of an image sensor with integrated image output according to an embodiment of the invention. Please refer to FIG. 7, the image sensor of this embodiment includes a pixel circuit 710, a pre-processing unit 720, a line accumulator 730, a first volume accumulator 740, a second volume accumulator 750, and a multiplexer 760, and its functions are divided into as follows:

The pixel circuit 710 is used to extract pixel values of multiple pixels in the image, and the pre-processing unit 720 performs pre-processing such as space correction or distortion correction on the pixel values output by the pixel circuit 710 . Wherein, the pixel circuit 710 can be equally divided into an upper left area, an upper right area, a lower left area and a lower right area, and is used for extracting pixel values of pixels in each area. Similarly, in another embodiment, the pixel circuit 710 can also be equally divided into MxN square areas, where M and N are positive integers, and the scope thereof is not limited here.

The line accumulator 730 includes a first adder 732 and a first buffer 734, which are coupled after the pre-processing unit 720, for receiving the pixel values of the pre-processed pixels from the pre-processing unit 720, and for The pixel values from the first pixel to the target pixel in the target pixel line in each area are accumulated to obtain the line pixel accumulated value.

The first volume accumulator 740 includes a second adder 742 and a second buffer 744, which are coupled after the line accumulator 730, and are used to accumulate the line pixel accumulation values of each area output by the line accumulator 730 into the same area The pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line is used as the pixel integral value of the target pixel in this area.

In detail, the line accumulator 730 can first accumulate the pixel values from the first pixel to the target pixel in the target pixel line in the upper left area of the image to obtain the line pixel accumulated value, and the first volume accumulator 740 accumulates the line pixel The value is added to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the upper left area to be the pixel integral value of the target pixel in the upper left area.

In addition, the line accumulator 730 can accumulate the pixel values from the first pixel to the target pixel in the target pixel line in the upper right area of the image to obtain the line pixel accumulation value, and the line pixel accumulation value is accumulated by the first volume accumulator 740 to The pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the upper right area is used as the pixel integral value of the target pixel in the upper right area. Accordingly, the first volume accumulator 740 completes the integral image calculation of the upper left area and the upper right area of the image.

The second volume accumulator 750 includes a third adder 752 and a third buffer 754, which are coupled after the line accumulator 730, and are used to accumulate the line pixel accumulation values of each area output by the line accumulator 730 into the same area The pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line is used as the pixel integral value of the target pixel in this area.

In detail, the line accumulator 730 can first accumulate the pixel values from the first pixel to the target pixel in the target pixel line in the lower left area of the image to obtain the line pixel accumulated value, and the second volume accumulator 750 accumulates the line pixel The value is added to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the lower left area to be the pixel integral value of the target pixel in the lower left area.

In addition, the line accumulator 730 can accumulate the pixel values from the first pixel to the target pixel in the target pixel line in the lower right area of the image to obtain the line pixel accumulation value, and the line pixel accumulation value is accumulated by the second volume accumulator 750 The pixel integral value of the pixel corresponding to the target pixel in the previous pixel line to the target pixel line in the lower right area is used as the pixel integral value of the target pixel in the lower right area. Accordingly, the second volume accumulator 750 completes the integral image calculation of the lower left area and the lower right area of the image.

The multiplexer 760 is respectively coupled to the first volume accumulator 740 and the second volume accumulator 750 of the pre-processing unit 720, which can choose to directly output the image in the output image format according to the output image format set by the user or the back-end processor. The pixel value of each pixel, or the pixel integral value of the lower left area and the lower right area of the output image after accumulation.

Regarding the process of calculating the integrated image by the above image sensor, an example is given below to illustrate. FIG. 8 is an example of calculating integral images according to an embodiment of the present invention. Please also refer to FIG. 8 , the image 800 on the left is the original image extracted by the image sensor, and the image can be equally divided into 2x2 square areas. Wherein, when the image sensor calculates the pixel integral value of the second pixel of the second scan line in the upper right area, the line accumulator 810 first accumulates the 5th to 6th pixel of the second scan line in the original image. The pixel value of pixels (ie 5, 0), and the line pixel cumulative value 5+0=5 is obtained.

Then, the first volume accumulator 820 accumulates the pixel accumulation value of this line to the pixel integral value of the corresponding pixel (i.e. the sixth pixel) in the previous scanning line (i.e. the first scanning line) to obtain the pixel of the target pixel The integral value 5+1=6. This pixel integral value updates the pixel integral value of the corresponding pixel in the previous scan line recorded in the first volume accumulator 820, and is used as the corresponding pixel (ie the 6th pixel) in the next (ie the 3rd scan line) to calculate ) for the pixel integral value.

Similarly, when the image sensor calculates the pixel integral value of the fourth pixel of the fourth scan line in the lower left area, the line accumulator 810 first accumulates the first pixel of the eighth scan line in the original image to The pixel value of the fourth pixel (ie 1, 4, 0, 2), and the line pixel accumulated value 1+4+0+2=7 is obtained.

Then, the second volume accumulator 830 accumulates the pixel accumulation value of this line to the pixel integral value of the corresponding pixel (ie, the 4th pixel) in the previous scan line (ie, the 7th scan line), and obtains the pixel of the target pixel The integral value is 7+18=25. The pixel integral value updates the pixel integral value of the corresponding pixel in the previous scan line recorded in the second volume accumulator 830 .

It is worth mentioning that the structure of the above image sensor utilizes two volume accumulators to respectively accumulate the pixels in the upper half and the lower half of the image. However, in one embodiment, if a reset circuit is added to the volume accumulator, and reset/reset/re-accumulate after the volume accumulator finishes accumulating the pixel integral values of the upper half of the pixels, then it can be re-accumulated. The hardware cost can be saved by continuing to use the same bulk accumulator to accumulate the integrated pixel value of the lower half of the pixels. Hereinafter, another embodiment will be given in detail.

FIG. 9A is a block diagram of an image sensor with integrated image output according to an embodiment of the invention. Referring to FIG. 9A, the image sensor of this embodiment includes a pixel circuit 910, a pre-processing unit 920, a line accumulator 930, a first volume accumulator 940 and a third multiplexer 950, and their functions are described as follows:

The pixel circuit 910 is used to extract pixel values of multiple pixels in the image, and the pre-processing unit 920 performs pre-processing such as space correction or distortion correction on the pixel values output by the pixel circuit 910 . Wherein, the pixel circuit 910 can be equally divided into an upper left area, an upper right area, a lower left area and a lower right area, and is used for extracting pixel values of pixels in each area. Similarly, in another embodiment, the pixel circuit 910 can also be equally divided into MxN square areas, where M and N are positive integers, and the scope thereof is not limited here.

The line accumulator 930 includes a first adder 932, a first buffer 934, and a first multiplexer 936, which are coupled after the pre-processing unit 920, and are used to receive from the pre-processing unit 920 the information of the pre-processed pixels. The pixel values are accumulated, and the pixel values from the first pixel to the target pixel in the target pixel line of each area in the image are accumulated, so as to obtain the line pixel accumulated value. It is worth noting that whenever the first adder 932 has accumulated the pixel values of all pixels in a pixel line in each area of the image, it can send a signal 0 to the first multiplexer 936, and the first multiplexer 936 resets the value of the first buffer 934 and continues to use the first buffer 934 to accumulate the pixel values of the pixels in the next pixel line.

The volume accumulator 940 includes a second adder 942, a second buffer 944 and a second multiplexer 946, and is used for accumulating the line pixel accumulation value of each area output by the line accumulator 930 to the target pixel line in the same area The pixel integral value of the pixel corresponding to the target pixel in the previous pixel line is used as the pixel integral value of the target pixel in this area.

In detail, the line accumulator 930 can first accumulate the pixel values from the first pixel to the target pixel in the target pixel line in the upper left area of the image to obtain the line pixel accumulation value, and the volume accumulator 940 accumulates the line pixel accumulation value The pixel integral value of the pixel corresponding to the target pixel in the previous pixel line to the target pixel line in the upper left area is used as the pixel integral value of the target pixel in the upper left area.

In addition, the line accumulator 930 can re-accumulate the pixel values from the first pixel to the target pixel in the same target pixel line in the upper right area of the image to obtain the line pixel accumulated value, and the volume accumulator 940 accumulates the line pixel The value is added to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the upper right area to be the pixel integral value of the target pixel in the upper right area. Accordingly, the volume accumulator 940 completes the integral image calculation of the upper left area and the upper right area of the image.

It is worth noting that after the volume accumulator 940 has accumulated the pixel integral values of the pixels in the upper left region and the upper right region, it can send a signal 0 to the second multiplexer 946, and the second multiplexer 946 converts the second The value of the buffer 944 is reset, and the second buffer 944 is continuously used to accumulate the pixel integral values of the pixels in the lower left area and the lower right area.

After the volume accumulator 940 is reset, the line accumulator 930 can accumulate the pixel values from the first pixel to the target pixel in the target pixel line in the lower left area of the image to obtain the line pixel accumulation value, and the volume accumulator 940 can use this The line pixel accumulation value is added to the pixel integration value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the lower left area to be the pixel integration value of the target pixel in the lower left area.

In addition, the line accumulator 930 can also re-accumulate the pixel values from the first pixel to the target pixel in the same target pixel line in the lower right area of the image to obtain the line pixel accumulated value, and the volume accumulator 940 accumulates the line pixel The value is added to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the lower right area to be the pixel integral value of the target pixel in the lower right area. Accordingly, the volume accumulator 940 completes the integral image calculation of the lower left area and the lower right area of the image.

The third multiplexer 950 is respectively coupled to the pre-processing unit 920 and the volume accumulator 940, which can select and directly output the pixel value of each pixel in the image according to the output image format set by the user or the back-end processor, Or output the pixel integral values of the lower left area and the lower right area of the image after accumulation.

FIG. 9B is a block diagram of an image sensor with integrated image output according to another embodiment of the present invention. Referring to FIG. 9B , the image sensor of this embodiment includes a pixel circuit 910 , a pre-processing unit 920 , a line accumulator 930 , a first volume accumulator 940 and a third multiplexer 950 . Different from the image sensor in FIG. 9A, the image sensor in this embodiment configures the first multiplexer 936 between the first adder 932 and the first buffer 934, and the second multiplexer The device 946 is disposed between the second adder 942 and the second buffer 944 to achieve the effect of resetting the first buffer 934 and the second buffer 944 . In addition, the functions of these components are the same or similar to those in FIG. 9A , so details will not be repeated here.

Regarding the process of calculating the integrated image by the above image sensor, an example is given below to illustrate. FIG. 10 is an example of calculating integral images according to an embodiment of the present invention. Please also refer to FIG. 10 , the image 1000 on the left is the original image extracted by the image sensor, and the image can be equally divided into 2x2 square areas. Wherein, when the image sensor calculates the pixel integral value of the second pixel of the second scan line in the upper right area, the line accumulator 1010 first accumulates the 5th to 6th pixel of the first scan line in the original image. The pixel value of pixels (ie 5, 0), and the line pixel cumulative value 5+0=5 is obtained.

Then, the volume accumulator 1020 accumulates the pixel accumulation value of this line to the pixel integral value of the corresponding pixel (ie, the sixth pixel) in the previous scan line (ie, the first scan line) to obtain the pixel integral value of the target pixel 5+1=6. This pixel integral value updates the pixel integral value of the corresponding pixel in the previous scan line recorded in the volume accumulator 1020, and is used as the calculation method for the corresponding pixel (ie the 6th pixel) in the next (ie the 3rd scan line) Pixel integration value is used.

It is worth noting that after the image sensor has calculated the pixel integral value of the fourth pixel of the fourth scan line in the upper right area, the accumulated pixel integral value in the volume accumulator 1020 is reset, and can be used to accumulate the lower left The pixel integration value of the region. When the image sensor is calculating the pixel integral value of the second pixel of the first scan line in the lower left area, the line accumulator 1010 first accumulates the first pixel to the first pixel of the fifth scan line in the original image. The pixel values of 2 pixels (ie 0, 1), and the line pixel accumulated value 0+1=1 is obtained. Then, the volume accumulator 1020 accumulates the pixel accumulation value of this line to the pixel integration value (reset) of the corresponding pixel in the previous scan line or directly loads a new value for re-accumulation, and obtains the pixel integration value of the target pixel 1.

Similarly, when the image sensor calculates the pixel integral value of the fourth pixel of the fourth scan line in the lower left area, the line accumulator 1010 first accumulates the first pixel of the eighth scan line in the original image to The pixel value of the fourth pixel (ie 1, 4, 0, 2), and the line pixel accumulated value 1+4+0+2=7 is obtained.

Then, the volume accumulator 1020 accumulates the pixel accumulation value of this line to the pixel integration value of the corresponding pixel (ie, the 4th pixel) in the previous scan line (ie, the 7th scan line), and obtains the pixel integration value of the target pixel 7+18=25. The pixel integral value updates the pixel integral value of the corresponding pixel in the previous scan line recorded in the volume accumulator 1020 .

The sub-integral image calculated in the above example is output to the back-end processor. At this time, the back-end processor only needs to perform simple addition and subtraction operations to restore the integral image and use it to obtain the feature value of the image feature. FIG. 11( a ) and FIG. 11( b ) are examples of calculated integral images according to an embodiment of the present invention. Please refer to Figure 11(a) and Figure 11(b) at the same time, where Figure 11(a) is the sub-integral image 1110 obtained after the calculation of the above image sensor, and Figure 11(b) is the restored Integral image 1120 . Wherein, the upper left area of the integral image 1120 is the same as the upper left area of the sub-integral image 1110 , so the processor can directly apply the pixel integral value in the upper left area of the sub-integral image 1110 . However, if the processor wants to obtain the pixel integral value of the second pixel of the first scan line in the lower left area of the integral image 1120, in addition to referring to the pixel at the corresponding position in the lower left area (ie, the second pixel of the first scan line In addition to the pixel integral value of the pixel), it is also necessary to refer to the pixel integral value of the same row of pixels (that is, the second pixel of the fourth scan line) in the lower edge of the upper left area, and add the two to restore the integral image 1120 The pixel integral value of the target pixel, that is, 1+13=14.

Similarly, if the processor wants to obtain the pixel integral value of the third pixel of the third scan line in the lower right area of the integrated image 1120, in addition to referring to the pixel at the corresponding position in the lower right area (ie, the third scan line In addition to the pixel integral value of the third pixel of the upper left area), it is also necessary to refer to the pixel integral value of the pixel in the lower right corner of the upper left area (that is, the fourth pixel of the fourth scan line), and the pixels in the same line of the lower edge of the upper right area (ie The pixel integral value of the 3rd pixel of the 4th scan line) and the pixel integral value of the pixel of the same column at the right edge of the lower left area (that is, the 4th pixel of the 3rd scan line), and the three are added to restore The pixel integral value of the target pixel of the integrated image 1120 is 14+25+19+18=76.

In addition, the above-mentioned new integral image can also be used to obtain the characteristic value of a specific region in the original integral image. For the calculation of feature values of a specific region that only includes a single segmented region, FIG. 12( a ) and FIG. 12( b ) are examples of calculating integral images according to an embodiment of the present invention. Wherein, FIG. 12( a ) is the original image 1210 , and FIG. 12( b ) is the sub-integrated image 1220 obtained after the above image sensor calculation.

It can be known from FIG. 12( a ) that the pixel integral value of all the pixels contained in the square A of the original image 1210 is 0+2+1+1=4. Next, please refer to FIG. 12(b), if you want to use the sub-integral image 1220 to obtain the pixel integral values of all pixels contained in the square A in the original image 1210, you need to find the corresponding square A' in the sub-integral image 1220, and The pixel integral values of all pixels contained in the block A are calculated as 14+0-4-6=4 by using the pixel integral values of the lower right pixel 1221 , the lower left pixel 1222 , the upper right pixel 1223 , and the upper left pixel 1224 of the block A'.

Similarly, it can be known from FIG. 12( a ) that the pixel integral value of all pixels contained in the square B in the original image 1210 is 1+0+1+4=6. Next, please refer to FIG. 12(b), if you want to use the sub-integral image 1220 to obtain the pixel integral values of all pixels contained in the square B in the original image 1210, you need to find the corresponding square B' in the sub-integral image 1220, and Calculate the pixel integral value of all pixels included in the block B with the pixel integral values of the lower right pixel 1225 and the lower left pixel 1226 of the block B', 13+0-0-7=6, wherein the upper right pixel and the upper left pixel Located outside the lower right area, its pixel integral value is set to 0.

For the calculation of feature values of a specific region including multiple segmented regions, FIG. 13( a ) and FIG. 13( b ) are examples of calculating integral images according to an embodiment of the present invention. Wherein, FIG. 13( a ) is the original image 1310 , and FIG. 13( b ) is the sub-integrated image 1320 obtained after the above image sensor calculation.

It can be seen from FIG. 13( a ) that the pixel integral value of all the pixels contained in the block A in the original image 1310 is 12+7+12+6=37. Next, referring to Fig. 12(b), if the sub-integral image 1320 is to be used to obtain the pixel integral values of all the pixels contained in the block A in the original image 1310, since the block A includes 4 sub-divided regions, it is necessary to Block A cuts block A1, block A2, block A3 and block A4 respectively, and calculates the pixel integral values of all pixels contained in these blocks by the above method, and finally adds the pixel integral values of these blocks to obtain the block The pixel integral value of all pixels contained in A (25+0-6-7)+(7+0-0-0)+(18+0-6-0)+(6+0-0-0)=12 +7+12+6=37.

It is worth mentioning that the architecture of the above image sensor divides the image into MxN square areas. However, in one embodiment, the image can be divided into several square sub-regions, each sub-region is divided/surrounded by 2 vertical boundaries (Vertical boundary) and 2 horizontal boundaries (Horizontal boundary). The division method can be set by the user, and it is not limited to a chessboard division, nor is it limited that each sub-area must be of equal size, but each sub-area must be a rectangle, so as to have more flexibility in use.

Regarding the process of calculating the integral image/segmentation by the above image sensor, an example is given below to illustrate. FIG. 14 is an example of calculating integral images according to an embodiment of the present invention. Please refer to FIG. 14 , the image 1400 on the left is the original image extracted by the image sensor, and the image can be divided into five areas: left area A, upper right area B, middle area C, right middle area D, and lower right area E. Sub-areas of unequal size, each sub-area is rectangular and is divided/surrounded by 2 vertical boundaries and 2 horizontal boundaries. Similarly, in another embodiment, the pixel circuit can also be divided into several square sub-regions, and the division method can be set by the user. Regions must all be rectangular.

It is worth noting that when the image sensor calculates the integral value of each pixel, it scans the image sequentially and is not affected by the segmentation. During sequential scanning, whenever the vertical boundary is crossed, the line accumulator is reset to re-accumulate the line pixel accumulation value of the sub-region on the segment of the pixel line. When scanning sequentially, if it is found that the target pixel has crossed the horizontal boundary compared with the corresponding pixel in the previous scan line, the volume accumulator will be re-accumulated or overwritten by directly loading new values to obtain the sub-pixel of the target pixel. Pixel integration value.

Regarding the process of calculating the integrated image by the above image sensor, an example is given below to illustrate. Fig. 15 is an example of a calculation integral image according to an embodiment of the present invention; Fig. 16 is a schematic diagram showing the value of a line accumulator changing with time according to an embodiment of the present invention; Fig. 17 is a schematic diagram according to an embodiment of the present invention The schematic diagram of the volume accumulator value changing with time shown in the embodiment. Please refer to Figures 15, 16, and 17 at the same time. When the image sensor calculates the pixel integral value of the third pixel of the first scan line in the image 1500, because it crosses the vertical boundary, the line accumulator accumulates again to obtain the line pixel The accumulated value is 3, and the sub-integrated image value for that pixel is 3.

When the image sensor calculates the pixel integral value of the second pixel of the fourth scan line in the image 1500, since neither of them crosses the vertical and horizontal boundaries, both the line accumulator and the volume accumulator continue to accumulate. Therefore, the accumulated value of the line accumulator is 1+4=5, and the accumulated value of this line pixel is added to the old value of this pixel (i.e. the 2nd pixel) in the body accumulator, i.e. the previous scanning line (i.e. the 3rd pixel) scanning line), and obtain the sub-pixel integral value 8+5=13 of the target pixel, and update it to the sub-pixel integral value of the corresponding pixel in the volume accumulator.

When the image sensor calculates the pixel integral value of the third pixel of the fourth scan line in the image 1500, both the line accumulator and the volume accumulator re-accumulate because they both straddle the vertical and horizontal boundaries. Therefore, the accumulated value of the line accumulator is the pixel value, which is 0, and the accumulated value of the volume accumulator is the accumulated value of the line pixel of the pixel, so the sub-pixel integral value of the target pixel is obtained as 0, and updated is the subpixel integral value of the corresponding pixel in the volume accumulator.

Similarly, when calculating the 4th pixel line in the image 1500, because the 3rd pixel to the 8th pixel all cross the horizontal boundary, therefore, the 3rd pixel to the 8th pixel of the volume accumulator need to be accumulated again, that is, load Enter new data to overwrite the old accumulated value. By analogy, after the pixels of each pixel line in the image 1500 are scanned, the integrated image of the image 1500 can be obtained.

According to the above-mentioned embodiments, no matter the image is divided into different types or shapes of squares, the image sensor can calculate the integral image using only one line accumulator and one volume accumulator.

To sum up, in the image sensor with integral image output of the present invention, that is, by combining a set of line accumulators and volume accumulators with the image sensor, the image sensor can extract the In the first time, the integral image of the extracted image is calculated, thereby reducing the consumption of computing resources of the processor and speeding up the speed of image processing. In addition, by dividing the image into multiple regions, and using the volume accumulator to calculate the integral image of each region, the integral image can also be obtained quickly. The above-mentioned line accumulator only needs to use the capacity to store a group of accumulated pixel values, and the volume accumulator only needs to use the capacity to store the pixel values in one pixel line, so it can save the cost of the image sensor , so that the image sensor has an additional integrated image output format that can be provided to the back-end processor for use.

Although the present invention has been disclosed as above with the embodiments, it is not intended to limit the present invention. Those skilled in the art can make some changes and modifications without departing from the spirit and scope of the present invention, so the protection of the present invention The scope is to be determined as defined by the appended claims.

Claims (53)

1. An image sensor with integral image output, comprising: A pixel circuit, including a plurality of pixel elements, for extracting pixel values of a plurality of pixels in an image; a line accumulator, coupled to the pixel circuit, for accumulating pixel values from a first pixel to a target pixel in a target pixel line of the image to obtain a line of pixel accumulation values; and an integrated accumulator, coupled to the line accumulator, for accumulating the line pixel cumulative value output by the line accumulator to a pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line taking the pixel integral value of the target pixel as the pixel integral value of the target pixel, and outputting the pixel integral value of the target pixel. 2. The image sensor with integral image output as claimed in claim 1, wherein the line accumulator comprises: a first buffer for recording the line pixel accumulation value accumulated by these pixels in the target pixel line; and A first adder is used for receiving the pixel value of the target pixel and adding it to the accumulated value of the line pixels recorded in the first buffer. 3. The image sensor with integral image output as claimed in claim 1, wherein the volume accumulator comprises: a second buffer for recording the pixel integral values of the pixels in the previous pixel line of the target pixel line; and a second adder, used for accumulating the line pixel accumulation value output by the line accumulator to the pixel integration value of the pixel corresponding to the target pixel in the previous pixel line recorded in the second buffer to as the pixel integral value of the target pixel, and output the pixel integral value of the target pixel. 4. The image sensor with integral image output as claimed in claim 3, wherein the second adder further comprises writing the accumulated pixel integral value of the target pixel into the second buffer to replace the The pixel integral value of the pixel corresponding to the target pixel in the previous pixel line. 5. The image sensor with integral image output as claimed in claim 1 , further comprising: A multiplexer, coupled to the pixel circuit and the volume accumulator, is used to output the pixel value and the pixel integral value of each of the pixels in the image. 6. The image sensor with integral image output as claimed in claim 1 , further comprising: A pre-processing unit is arranged between the pixel circuit and the line accumulator, and is used for performing a pre-processing on the pixel values of the pixels output by the pixel circuit. 7. The image sensor with integral image output as claimed in claim 6, wherein the pre-processing includes spatial correction or distortion correction. 8. The image sensor with integrated image output as claimed in claim 1, wherein the line accumulator is further reset after each accumulation of pixel values of all pixels in a pixel line of the image. 9. The image sensor with integrated image output as claimed in claim 1, wherein the volume accumulator includes sequentially outputting pixel integral values of the pixels in the target pixel line. 10. The image sensor with integral image output as claimed in claim 1 , wherein the volume accumulator includes an output value of all pixels in the target pixel line at a time after obtaining the pixel integral values of all pixels in the target pixel line Pixel integration value. 11. The image sensor with integral image output as claimed in claim 1, wherein the pixel integral values of the pixels of the image output by the volume accumulator form an integral image of the image. 12. The image sensor with integral image output as claimed in claim 1, wherein the pixel circuit further comprises extracting pixel values of a plurality of color spaces of a plurality of pixels in the image respectively. 13. The image sensor with integral image output as claimed in claim 12, wherein the line accumulator further comprises accumulating the color spaces of the first pixel to the target pixel in the target pixel line of the image Pixel values to obtain the line pixel summation values for each of these color spaces. 14. The image sensor with integral image output as claimed in claim 13, wherein the volume accumulator further includes adding the line pixel accumulation values of each of the color spaces output by the line accumulator to the target pixel line respectively A pixel integral value of each of these color spaces corresponding to the target pixel in the previous pixel line as the pixel integral value of each of these color spaces of the target pixel, and output each of these color spaces of the target pixel The pixel integral value of . 15. The image sensor with integral image output as claimed in claim 12, wherein the color space comprises one of RGB, YCrCb and YUV. 16. The image sensor with integral image output as claimed in claim 1, wherein the pixel corresponding to the target pixel in the previous pixel line of the target pixel line is the same as the target pixel in the previous pixel line pixels in the same row. 17. An image sensor with integrated image output comprising: A pixel circuit, including a plurality of pixel elements, used to extract pixel values of a plurality of pixels in an image, wherein the pixel circuit can be divided into MxN square areas, wherein M and N are positive integers; A line accumulator, coupled to the pixel circuit, for accumulating pixel values from a first pixel to a target pixel in a target pixel line of each of the square regions of the image to obtain a line of pixel accumulation values; and N individual accumulators, coupled to the line accumulator, are used to accumulate the line pixel accumulation value output by the line accumulator to the pixel line corresponding to the target pixel in the previous pixel line of the target pixel line in each of these square areas. A pixel integral value of the pixel is used as the pixel integral value of the target pixel, and the pixel integral value of the target pixel is output. 18. The image sensor with integral image output as claimed in claim 17, wherein the pixel circuit comprises an upper left area, an upper right area, a lower left area and a lower right area. 19. The image sensor with integral image output as claimed in claim 18 , wherein the line accumulator includes accumulating the pixel values of the first pixel to the target pixel in the target pixel line of the upper left region of the image, The line pixel cumulative value is obtained, and a first volume accumulator among the volume accumulators accumulates the line pixel cumulative value to the pixel line corresponding to the target pixel in the previous pixel line of the target pixel line in the upper left area. The pixel integral value of the pixel is used as a sub-pixel integral value of the target pixel in the upper left area. 20. The image sensor with integral image output as claimed in claim 19, wherein the line accumulator includes accumulating the pixel values of the first pixel to the target pixel in the target pixel line of the upper right region of the image, The line pixel cumulative value is obtained, and the line pixel cumulative value is accumulated by the first volume accumulator among the volume accumulators to the pixel line corresponding to the target pixel in the previous pixel line of the target pixel line in the upper right area. The pixel integral value of the pixel is used as the sub-pixel integral value of the target pixel in the upper right area. 21. The image sensor with integral image output as claimed in claim 20, wherein the line accumulator includes accumulating the pixel values of the first pixel to the target pixel in the target pixel line of the lower left region of the image, The line pixel cumulative value is obtained, and the line pixel cumulative value is accumulated by a second volume accumulator among the volume accumulators to the pixel line corresponding to the target pixel in the previous pixel line of the target pixel line in the lower left area. The pixel integral value of the pixel is used as the sub-pixel integral value of the target pixel in the lower left area. 22. The image sensor with integral image output as claimed in claim 21 , wherein the line accumulator includes accumulating the pixel values of the first pixel to the target pixel in the target pixel line of the lower right region of the image , to obtain the line pixel cumulative value, and the second volume accumulator in these volume accumulators accumulates the line pixel cumulative value to the previous pixel line of the target pixel line in the lower right region corresponding to the target pixel The pixel integral value of the pixel is used as the sub-pixel integral value of the target pixel in the lower right area. 23. The image sensor with integral image output as claimed in claim 17, wherein the pixel circuit comprises MxN square regions which can be equally divided. 24. The image sensor with integral image output as claimed in claim 17, wherein the line accumulator comprises: A first buffer for recording the line pixel accumulation value accumulated by the pixels in each of the square regions in the target pixel line; and A first adder is used for receiving the pixel value of the target pixel in each of the square areas and adding it to the accumulated value of the line pixels recorded in the first buffer. 25. The image sensor with integral image output as claimed in claim 17, wherein each of the volume accumulators comprises: A second buffer for recording the pixel integral value of each of these pixels in each of these square areas in the previous pixel line of the target pixel line; and a second adder, used for accumulating the line pixel accumulation value output by the line accumulator to the pixel corresponding to the target pixel in the previous pixel line in each of the square regions recorded in the second buffer The pixel integral value is used as the pixel integral value of the target pixel, and the pixel integral value of the target pixel is output. 26. The image sensor with integral image output as claimed in claim 25, wherein the second adder further comprises writing the accumulated pixel integral value of the target pixel into the second buffer to replace the The pixel integral value of the pixel corresponding to the target pixel in the previous pixel line. 27. The image sensor with integrated image output as claimed in claim 17, further comprising: A multiplexer, coupled to the volume accumulators, is used to output pixel integral values of pixels in each of the square regions in the image. 28. The image sensor with integrated image output of claim 17, further comprising: A pre-processing unit is arranged between the pixel circuit and the line accumulator, and is used for performing a pre-processing on the pixel values of the pixels output by the pixel circuit. 29. The image sensor with integrated image output as claimed in claim 28, wherein the pre-processing includes spatial correction or distortion correction. 30. The image sensor with integral image output as claimed in claim 17, wherein the line accumulator further comprises reset after each accumulation of pixel values of all pixels in a pixel line of each of the square regions of the image . 31. The image sensor with integrated image output as claimed in claim 17, wherein each of the volume accumulators sequentially outputs pixel integral values of each of the pixels in the target pixel line of each of the square regions. 32. The image sensor with integral image output as claimed in claim 17, wherein the volume accumulator comprises outputting the target pixel at a time after obtaining the pixel integral values of all the pixels in the target pixel line of each of the square regions The pixel-integrated value of all pixels in the line. 33. The image sensor with integral image output as claimed in claim 17, wherein the pixel integral values of the pixels in each of the square regions of the image output by the volume accumulators form an integral image of the image. 34. An image sensor with integrated image output comprising: A pixel circuit comprising a plurality of pixel elements for extracting pixel values of a plurality of pixels in an image, wherein the pixel circuit can be divided into a plurality of square regions; A line accumulator, coupled to the pixel circuit, for accumulating pixel values from a first pixel to a target pixel in a target pixel line of each of the square regions of the image to obtain a line of pixel accumulation values; and an integrated accumulator, coupled to the line accumulator, for accumulating the line pixel accumulation value output by the line accumulator to the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in each of the square areas A pixel integral value of is used as the pixel integral value of the target pixel, and the pixel integral value of the target pixel is output. 35. The image sensor with integrated image output as claimed in claim 34, wherein the line accumulator is reset upon crossing a longitudinal boundary of the square regions or changing scanned pixel lines. 36. The image sensor with integrated image output as claimed in claim 35, wherein the volume accumulator includes a reset across the lateral boundaries of the square regions. 37. The image sensor with integral image output as claimed in claim 34, wherein the pixel circuit comprises an upper left area, an upper right area, a lower left area and a lower right area. 38. The image sensor with integral image output as claimed in claim 37, wherein the line accumulator includes accumulating the pixel values of the first pixel to the target pixel in the target pixel line of the upper left region of the image, The line pixel cumulative value is obtained, and the volume accumulator accumulates the line pixel cumulative value to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the upper left area as A sub-pixel integral value of the target pixel in the upper left area. 39. The image sensor with integral image output as claimed in claim 38, wherein the line accumulator includes accumulating the pixel values of the first pixel to the target pixel in the target pixel line of the upper right region of the image, The line pixel cumulative value is obtained, and the volume accumulator accumulates the line pixel cumulative value to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the upper right area as The sub-pixel integral value of the target pixel in the upper right area. 40. The image sensor with integral image output as claimed in claim 39, wherein the line accumulator includes accumulating the pixel values of the first pixel to the target pixel in the target pixel line of the lower left region of the image, To obtain the line pixel cumulative value, and the volume accumulator accumulates the line pixel cumulative value to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the lower left area as The sub-pixel integral value of the target pixel in the lower left area. 41. The image sensor with integral image output as claimed in claim 40, wherein the line accumulator includes accumulating the pixel values of the first pixel to the target pixel in the target pixel line of the lower right region of the image , to obtain the line pixel cumulative value, and the volume accumulator accumulates the line pixel cumulative value to the pixel integral value of the pixel corresponding to the target pixel in the previous pixel line of the target pixel line in the lower right area Taking as the sub-pixel integral value of the target pixel in the lower right area. 42. The image sensor with integral image output as claimed in claim 34, wherein the pixel circuit comprises MxN square regions, wherein M and N are positive integers. 43. The image sensor with integral image output as claimed in claim 34, wherein the line accumulator comprises: A first buffer for recording the line pixel accumulation value accumulated by the pixels in each of the square regions in the target pixel line; and A first adder is used for receiving the pixel value of the target pixel in each of the square areas and adding it to the accumulated value of the line pixels recorded in the first buffer. 44. The image sensor with integral image output as claimed in claim 43, wherein the line accumulator further comprises: A first multiplexer, used for resetting the first buffer after the first buffer accumulates the pixel values of all the pixels in a pixel line of each of the square regions of the image. 45. The image sensor with integral image output as claimed in claim 34, wherein the volume accumulator comprises: A second buffer for recording the pixel integral value of each of these pixels in each of these square areas in the previous pixel line of the target pixel line; and a second adder, used for accumulating the line pixel accumulation value output by the line accumulator to the pixel corresponding to the target pixel in the previous pixel line in each of the square regions recorded in the second buffer The pixel integral value is used as the pixel integral value of the target pixel, and the pixel integral value of the target pixel is output. 46. The image sensor with integral image output as claimed in claim 45, wherein the second adder further comprises writing the accumulated pixel integral value of the target pixel into the second buffer to replace the The pixel integral value of the pixel corresponding to the target pixel in the previous pixel line. 47. The image sensor with integral image output as claimed in claim 45, wherein the second adder further comprises: A second multiplexer is used for resetting the second buffer after the second adder accumulates the integrated pixel values of all the pixels of the target pixel line of the image. 48. The image sensor with integrated image output of claim 34, further comprising: A third multiplexer, coupled to the volume accumulator, is used to output pixel integral values of pixels in each of the square regions in the image. 49. The image sensor with integrated image output of claim 34, further comprising: A pre-processing unit is arranged between the pixel circuit and the line accumulator, and is used for performing a pre-processing on the pixel values of the pixels output by the pixel circuit. 50. The image sensor with integrated image output as claimed in claim 49, wherein the pre-processing includes spatial correction or distortion correction. 51. The image sensor with integrated image output as claimed in claim 34, wherein the volume accumulator includes sequentially outputting pixel integral values of each of the pixels in the target pixel line of each of the square regions. 52. The image sensor with integral image output as claimed in claim 34, wherein the volume accumulator comprises outputting the target pixel at a time after obtaining pixel integral values of all pixels in the target pixel line of each of the square regions The pixel-integrated value of all pixels in the line. 53. The image sensor with integrated image output as claimed in claim 34, wherein the pixel integral values of the pixels of each of the square regions of the image output by the volume accumulator form an integral image of the image.

Images