JP2003234967A - High-speed imaging device - Google Patents

Abstract

(57) [Problem] To provide a high-speed imaging device which requires a remarkably small mounting area, can perform high-speed processing, and can obtain a high compression ratio. An A / D conversion circuit for A / D converting a pixel signal output from a pixel circuit having an array of a plurality of pixels arranged in an array, and the A / D conversion circuit are provided.
And a compression circuit 12 for compressing the pixel data output by the. The compression circuit 12 includes an A / D conversion circuit 1
1 outputs two-dimensional discrete cosine transform (D
CT) and a coding circuit 12b for performing variable-length coding on a plurality of output lines on image data converted by the two-dimensional discrete cosine conversion circuit.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a high-speed image pickup device capable of taking an image at high speed using a solid-state imaging sensor, and more particularly to an image signal taken at high speed using a CMOS image sensor as a solid-state imaging sensor. The present invention relates to a high-speed image pickup apparatus which has a function of compressing an image and is suitable for parallel output of image signals.

[0002]

2. Description of the Related Art Conventionally, an image pickup camera provided with a CMOS image sensor as a solid-state image sensor is disclosed in, for example, "A. Krymsk i.D.
kon, A .; Anderson, et. al. , "A
high-speed 500 frame / s 102
4 × 1024 CMOS image sensor ”
Dig. Tech. Papers Symp. on
VLSI Circuits, No. 14-3, Ju
ne 1999 ”. This allows the CMO
High-speed imaging cameras that take advantage of the characteristics of the S image sensor are expected.

[0003]

However, the high-speed imaging camera described in this document uses the successive approximation type converter as the A / D converter, and the 8-bit A / D of one data is used.
Since about 10 clocks are required for D conversion, high-speed imaging has a limit. In addition, since the amount of digital output data increases as the imaging speed increases, it is necessary to record as many high-speed images as possible in a limited image memory. The high-speed image pickup camera of (1) did not have an image compression function.

Therefore, a high-speed image pickup apparatus capable of eliminating such inconvenience can have the following configuration.

That is, this high-speed image pickup device has m rows × n.
A pixel circuit composed of a plurality of pixels arranged in an array of columns and a pixel signal from this pixel circuit
A / D conversion means for A / D conversion, an image compression means for image-compressing the output of the conversion means in parallel, and an output means for outputting the digital output of the image compression means in parallel are provided.

In a high-speed image pickup device having this image compression means, an A / D is provided on the output side of an image sensor as an image circuit.
The circuit forming the converting means is connected, and the circuit forming the image compressing means is connected to the output side of this circuit. However, when the conventional image compression circuit is used as it is in this high-speed imaging device, it is necessary to use a two-dimensional Huffman coding table in order to increase the compression rate, and the circuit scale becomes large, so there is room for improvement. Further, since the number of output lines of each digital signal component after image compression is fixed, the compression rate is still low, and there is room for improvement also in this point.

The present invention has been made in order to overcome such a situation, and provides a high-speed image pickup device which requires a significantly small mounting area, can perform high-speed processing, and can obtain a high compression rate. That is the purpose.

[0008]

The high-speed image pickup device according to the present invention reduces the amount of output data by incorporating an image compression circuit in an image sensor by utilizing the fact that a system circuit, which is a characteristic of a CMOS LSI, can be made on-chip. Reduce. Image compression is performed by 4 × 4 point two-dimensional discrete cosine transform (DC
T) and an entropy coding method for multiple lines, which are suitable for high-speed image compression. As a result, the method can be implemented in an extremely small area, is a method suitable for parallel processing, and can be formed with high compression.

A specific structure of the high-speed image pickup device according to the present invention is such that a pixel circuit having an array of a plurality of pixels arranged in a two-dimensional array and pixel signals output from the pixel circuit are arranged in columns or rows. An A / D conversion circuit for A / D conversion with
A compression circuit that compresses pixel data output from the A / D conversion circuit and outputs the pixel data in parallel, the compression circuit converting the image data output from the A / D conversion circuit into a two-dimensional discrete cosine transform (DCT). It is basically provided with a two-dimensional discrete cosine transform circuit and a coding circuit for performing variable-length coding of a plurality of output lines on the image data converted by the two-dimensional discrete cosine transform circuit.

Preferably, the pixel circuit is a CMOS image sensor formed by using an LSI (large-scale integrated circuit) made of CMOS (complementary MOS). At this time,
For example, the pixel circuit, the A / D conversion circuit, and the compression circuit are arranged in parallel so as to process pixel signals output from pixels in the column direction of the pixel circuit in column parallel. Preferably, the A / S is formed by the CMOS LSI.
The D conversion circuit and the compression circuit are incorporated in the pixel circuit, and the pixel circuit is made on-chip. This on-chip structure is, for example, one chip in which the pixel circuit handling an analog signal is made into a chip, and another chip in which the A / D conversion circuit and the compression circuit handling a digital signal are made into a chip. It is a two-chip structure consisting of and.

In the high-speed image pickup apparatus having various structures described above, the two-dimensional discrete cosine transform circuit is DA (D
It is desirable to use a circuit that performs discrete cosine transform for each block with 4 × 4 pixels as one block by using the distributed arithmatic) method. This 2
The one-dimensional discrete cosine transform circuit includes, for example, a first one-dimensional DCT calculator that applies a one-dimensional discrete cosine transform to a pixel signal converted into a digital value by the A / D conversion circuit, and the first one-dimensional DCT. A transposed matrix circuit that applies the operation result of the operator to the transposed matrix operation, and a second one-dimensional D that applies the operation result of this transposed matrix circuit to the one-dimensional discrete cosine transform again.
And a CT calculator.

Further, it is preferable that the encoding circuit is a circuit for encoding the data processed by the two-dimensional discrete cosine transform circuit for each type of coefficient. As an example, the encoding circuit divides the data of each block processed by the two-dimensional discrete cosine transform circuit into a DC coefficient and an AC coefficient, a DC coefficient of each block, and a DC coefficient of the adjacent block. And a circuit that calculates a difference value between the two and a one-dimensional Huffman coding table, and performs a zigzag scan scan on the AC coefficients of a plurality of blocks adjacent to each other to form a one-dimensional It is converted into data, the number of effective coefficients that are continuous with the effective coefficient of the one-dimensional data (zero run length) is calculated, and both the effective coefficient and the zero run length coefficient are encoded separately based on the one-dimensional Huffman encoding table. And a circuit.

Further, the encoding circuit is an encoded D
The C coefficient, the AC coefficient, and the zero run length coefficient are separately FIF
It is desirable to have a circuit that outputs while controlling the output rate with an O (First-In First-Out) buffer.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 shows an outline of a CMOS image sensor as a high-speed image pickup device according to this embodiment. As shown in the figure, this CMOS image sensor is based on the pixel circuit 1
0, an A / D conversion circuit 11, and an image compression circuit 12.

The pixel circuit 10, the A / D conversion circuit 11,
The image compression circuit 12 and the image compression circuit 12 are arranged in parallel, and the pixel signals read from the pixel circuit 10 are processed in parallel in columns. In addition, the sensor has a high sensitivity and an electronic shutter function that is essential for high-speed imaging and C
DS (correlated double sampling) is provided.

In the present embodiment, this CMOS image sensor is composed of an LSI using CMOS. Therefore, the other A / D conversion circuit 11 and the image compression circuit 12
Also has a system circuit, which is a feature of CMOS LSI, and is formed on-chip. However, it is difficult to integrate all circuits in one chip due to the chip area.
Therefore, an analog chip C1 in which the pixel circuit 10 that handles an analog signal is on-chip and an A that handles a digital circuit are used.
A two-chip architecture is adopted, which is a digital chip C2 in which the / D conversion circuit 11 and the image compression circuit 12 are on-chip. With this two-chip configuration, it is possible to suppress the influence of noise from the digital circuit in the analog circuit.

The pixel circuit 10 has a size of 512 × 512.
Two-dimensional array-shaped pixel area 1 formed by a plurality of pixels
0a, CMOS APS (Active Pix
(eSensor), it is configured as a circuit capable of performing simultaneous readout on a row-by-row basis. Pixel signals read from the pixel circuit 10 row by row are converted into differential signals by the differential amplifier 10b, amplified, and then output differentially via the buffer 10c. The differential amplifier 10b has four columns,
The buffers are arranged every eight columns, and the output pins are 128 pins.

The pixel signals for each column output from the pixel circuit 10 forming the analog chip C1 are sent to the A / D conversion circuit 11 of the digital chip C2.

The A / D conversion circuit 11 includes a sample / hold amplifier 11a and a pipeline A / D converter 11.
b, by which pipeline A / D conversion is performed. The differential type 1.5 bit / stage pipeline A / D converter is characterized by high speed, low power consumption, and a small mounting area.
The A / D conversion circuit 11 is configured by arranging 128 D converters in 8 bits, 1 channel / 4 columns.

The image compression circuit 12 includes a two-dimensional DCT circuit 12a for performing a two-dimensional discrete cosine transform (DCT), an encoding circuit 12b for performing a variable length encoding process on the conversion result, and an encoded signal. And an output circuit 12c for outputting

In the case of column-parallel processing, DA (Distr)
4x using the ibuted Arithmetic method
If a two-dimensional DCT circuit of 4 pixels (this is one block) is configured, it can be realized with an extremely small circuit. Therefore, the two-dimensional DCT circuit 12a of the present embodiment has 128 DCT circuits in parallel for each block. It is arranged. This allows column-parallel processing for 512 pixel columns.

DCT by the two-dimensional DCT circuit 12a
As for the calculation, as shown in FIG. 2, the first one-dimensional DCT calculator 21 performs DCT calculation once, and the result is transposed RAM.
This is achieved by transposing in the unit 22 and performing the DCT operation on the transposed result again by the second one-dimensional DCT calculator 23. As a result of considering the scale of hardware and the compression ratio using this DCT operation configuration, a DCT of 4 × 4 pixels
The circuit has been found to be optimal. From this point, as described above, it is convenient to perform the DCT operation in units of 4 × 4 pixels as one block.

The one-dimensional DCT calculators 21 and 22 perform column parallel processing to speed up the processing. Also, D
There is regularity of the CT algorithm. That is, since the cos coefficient, which is the orthogonal basis of the DCT calculation, has periodicity, the one-dimensional DCT calculators 21 and 22 can be configured with a simple circuit as shown in FIG. 3 by paying attention to this periodicity. it can.

In the DCT operation, DA (Distri) is an efficient operation method with respect to the product sum operation of fixed coefficients.
Butted Arithmetic) method is applied. That is, the product-sum operation with the input data cos coefficient is processed in bit string units, not in normal word units.

To perform the sum of products operation in units of bit strings,
The coefficient ROM stores the multiplication result of the input data and the cos coefficient in advance. When each bit string state of the input data is used as an address, the result corresponding to this address is read and the result is accumulated from MSB to LSB to obtain the DCT coefficient. Further, the number of product-sum operations is halved by performing the operation by 2 bits at a time.

By this two-dimensional DCT calculation, the DC coefficient reflecting the DC component of the pixel signal and the AC coefficient reflecting the AC component are calculated. In this operation, one block can be performed in 32 clocks, that is, a two-dimensional DCT operation can be performed at a rate of 2 clocks / pixel, and an extremely high operation speed can be obtained. The data of the DC coefficient and the AC coefficient, which are the calculation results, are sent in parallel to the encoding circuit 12b at the next stage.

In this two-dimensional DCT circuit 12a, since the DCT calculator is composed of only the adder without using the multiplier, the circuit can be simplified and downsized accordingly.

The encoding circuit 12b employs a multiple output line variable length encoding method as will be described in detail below, and in order to increase the compression rate, as shown in FIG. 4 × 16 pixels) are collectively encoded.

In the encoding circuit 12b, the DCT-processed data is divided into DC coefficient and AC coefficient, and encoded by different encoding methods.

Regarding the DC coefficient, the difference value (ΔDC) from the adjacent DC coefficient (DC) is encoded using the one-dimensional Huffman encoding table.

On the other hand, the AC coefficient is converted into one-dimensional data by efficiently performing a zigzag scan operation on four blocks as schematically shown in FIG. ) Is divided into. Each coefficient of this effective integer (AC) and zero run length (ZRL) is 1
It is encoded separately using the dimensional Huffman encoding table. The image encoding table that is normally used is
Since bit serial output is assumed, the code amount for identifying each coefficient is large, but in the present embodiment, by specifying the output pin of each encoded data in advance,
The code amount for identifying data is reduced. That is,
Both the effective coefficient and the zero run length coefficient are encoded using a one-dimensional encoding table.

The DC coefficient, AC, of the pixels for these four blocks
The coefficient and the zero run length coefficient are coded separately and sent to the output circuit 12c at the next stage. As described above, the encoding circuit 12b performs high-compression encoding using Huffman encoding, which is entropy encoding, and outputs encoded coefficients from different output pins for each type, Higher compression is possible.

The output circuit 12c stores a predetermined number of FIFOs (Firsts) for temporarily storing the codes of the DC coefficient, the AC coefficient, and the zero-run length coefficient obtained by the coding.
-In First-Out) buffer. The code of each coefficient is output while controlling the output rate for each coefficient by this buffer.

Specifically, the output of the CMOS image sensor is output by a plurality of lines, and signal lines are assigned to the DC coefficient, the AC coefficient, and the zero run length coefficient, respectively. Thereby, as described above, the one-dimensional Huffman coding table can be used. Further, the sum of the codes of the DC coefficient, the AC coefficient, and the zero run length coefficient is calculated in the FIFO buffer for each coefficient, and the output pin is optimally assigned to each code. As a result, the code of each coefficient (component) obtained by efficient coding is output to the external memory.

An example of this output pin assignment is shown in FIG.
It shows in (c). For example, the code amounts DC, AC, and ZRL of four blocks that are one-dimensionally encoded are shown in FIG. The sum of the code amounts DC, AC and ZRL is calculated as shown in FIG. Based on the result of this calculation, as shown in FIG. 7C, the data amount of each output pin is optimally distributed so as to be substantially uniform.

Here, each coefficient is identified by assigning a separate output pin to each coefficient. The memory allocation information is recognized, for example, by storing it in the header for each frame.

As a result, the output pin assigned to the output of the code of each coefficient is dynamically changed according to the code data amount of each coefficient, and the output code amount output from each output line is substantially equalized. Therefore, a high image compression ratio can be obtained.

FIG. 6 is a diagram for explaining the operation of the process centered on the image compression circuit 12 described above. Further, FIG. 7 illustrates a more detailed circuit block diagram of the A / D conversion circuit 11 and the image compression circuit 12 that processes the pixel signals read by the pixel reading of four blocks of the pixel circuit 10 in column parallel. To do.

The present inventor has evaluated an image picked up by the high-speed image pickup apparatus according to the present invention and has confirmed that a very high compression ratio and PSNR (SNR after compression) can be obtained in a natural image.

It should be noted that the present invention is not limited to the configurations of the above-described embodiment and its variations, and a person skilled in the art will be within the scope of the present invention as set forth in the claims. It can be implemented by being modified into various forms.

[0042]

As described above, according to the high-speed image pickup device of the present invention, the pixel signal output from the pixel circuit having the array of a plurality of pixels arranged in an array is A / D conversion circuit for A / D conversion and this A / D conversion circuit
And a compression circuit that compresses pixel data output from the D conversion circuit and outputs the compressed pixel data in parallel. The compression circuit converts the image data output from the A / D conversion circuit into a two-dimensional discrete cosine transform (DC).
Since the two-dimensional discrete cosine transform circuit for T) and the coding circuit for performing the multiple output line variable length coding on the image data converted by the two-dimensional discrete cosine transform circuit are provided,
The mounting area is extremely small, high-speed processing is possible, and a high compression rate can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a CMOS imaging sensor as a high-speed imaging device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a two-dimensional DCT circuit used in the embodiment.

FIG. 3 is a diagram showing one of the two-dimensional DCT circuits used in the embodiment.
3 is a block diagram of a three-dimensional DCT circuit. FIG.

FIG. 4 is a diagram illustrating a scan operation for each coefficient in the encoding circuit used in the embodiment.

FIG. 5 is a diagram for explaining optimization of output pin (output line) allocation operation for each coefficient in the encoding circuit used in the embodiment.

FIG. 6 is an explanatory diagram of a scheme of the image compression circuit according to the embodiment.

FIG. 7 is a detailed block diagram of an A / D conversion circuit and an image compression circuit for processing pixel signals read from pixels of four blocks of the pixel circuit in column parallel according to the embodiment.

[Explanation of symbols]

10 pixel circuits 11 A / D conversion circuit 12 Image compression circuit 12a two-dimensional DCT circuit 12b encoding circuit 12c output circuit

Continued front page    (72) Inventor Masataka Tsuji             1-9-8 Shibuya, Shibuya-ku, Tokyo Stock market             Company Photolon (72) Inventor Shinichi Kuwamura             1-9-8 Shibuya, Shibuya-ku, Tokyo Stock market             Company Photolon (72) Inventor Sake Yoshichi             1-9-8 Shibuya, Shibuya-ku, Tokyo Stock market             Company Photolon (72) Inventor Hiroshi Nagai             1-9-8 Shibuya, Shibuya-ku, Tokyo Stock market             Company Photolon (72) Inventor Toru Inoue             1-9-8 Shibuya, Shibuya-ku, Tokyo Stock market             Company Photolon F-term (reference) 4M118 AA10 AB01 BA14 DD09 FA06                 5C024 CX54 CY45 GY31 HX01 HX23                       HX29                 5C059 KK00 KK06 KK13 LA01 LA04                       MA23 MC01 MC33 MC34 MC38                       ME02 SS00 UA02 UA25 UA32                 5C078 AA04 BA57 CA34 DA01

Claims (10)

[Claims] 1. A pixel circuit having an array of a plurality of pixels arranged in a two-dimensional array, and an A / D conversion circuit for A / D converting pixel signals output from the pixel circuit in units of columns or rows. A compression circuit that compresses pixel data output from the A / D conversion circuit and outputs the compressed pixel data in parallel. The compression circuit converts the image data output from the A / D conversion circuit into a two-dimensional discrete cosine transform (DCT). 2) Discrete cosine transform circuit for performing the above), and a high-speed image pickup device provided with a coding circuit for performing multiple output line variable length coding on the image data converted by the two-dimensional discrete cosine transform circuit. 2. The high-speed image pickup device according to claim 1, wherein the pixel circuit is a CMOS (complementary MOS) LS.
A high-speed image pickup device which is a CMOS image sensor formed by using I (large-scale integrated circuit). 3. The high-speed image pickup device according to claim 2, wherein the pixel circuit, the A / D conversion circuit, and the compression circuit perform column parallel processing on pixel signals output from pixels in a column direction of the pixel circuit. High-speed imaging device arranged in parallel so as to process. 4. The high-speed image pickup device according to claim 2, wherein the CMOS LSI incorporates the A / D conversion circuit and the compression circuit into the pixel circuit, and the pixel circuit is on-chip. High-speed imaging device. 5. The high-speed image pickup device according to claim 4, wherein the on-chip structure is one chip in which the pixel circuit that handles an analog signal is made into a chip, and the A / D conversion circuit that handles a digital signal. And a high-speed image pickup device having a two-chip structure comprising the compression circuit and the other chip. 6. The high-speed image pickup device according to claim 1, wherein the two-dimensional discrete cosine transform circuit is DA (Distr).
4 x using the ibuted Arithmetic method
A high-speed imaging device that is a circuit that performs a discrete cosine transform for each block with 4 pixels as one block. 7. The high-speed image pickup device according to claim 6, wherein the two-dimensional discrete cosine transform circuit subjects the pixel signal converted into a digital amount by the A / D converter circuit to one-dimensional discrete cosine transform. 1, a one-dimensional DCT calculator, a transposed matrix circuit that applies the operation result of the first one-dimensional DCT operator to a transposed matrix operation, and the operation result of this transposed matrix circuit again to a one-dimensional discrete cosine transform Second one-dimensional DC
A high-speed imaging device including a T calculator. 8. The high-speed image pickup device according to claim 6, wherein the encoding circuit is a circuit for encoding the data processed by the two-dimensional discrete cosine transform circuit for each type of coefficient. 9. The high-speed imaging device according to claim 8, wherein the encoding circuit divides the data of each block processed by the two-dimensional discrete cosine transform circuit into a DC coefficient and an AC coefficient, The difference value is calculated between the DC coefficient of each block and the DC coefficient of the adjacent block,
A circuit that encodes this difference value based on a one-dimensional Huffman encoding table, and converts the AC coefficients of a plurality of blocks that are adjacent to each other into one-dimensional data by performing zigzag scan scanning and converts the one-dimensional data. A high-speed imaging apparatus having a circuit for calculating the number of effective coefficients and the number of consecutive invalid coefficients (zero-run length) and separately encoding both the effective coefficient and the zero-run length coefficient based on a one-dimensional Huffman encoding table. 10. The high-speed image pickup device according to claim 9, wherein the compression circuit separately stores an encoded DC coefficient, an AC coefficient, and a zero run length coefficient in a FIFO (First-In).
A high-speed image pickup apparatus having a circuit that outputs while controlling an output rate with a First-Out buffer.