JPH10262189A - Imaging device - Google Patents

Abstract

(57) [Problem] In a conventional spatial convolution operation, an independent circuit for performing a spatial convolution operation on an image pickup output signal is required. SOLUTION: An optical means 2 capable of moving an optical image 5 in a spatial one-dimensional or two-dimensional direction (7), and capable of changing the transmittance while forming the optical image 5 (8). Transmission image display means 3 and storage type two-dimensional area image pickup device 4 optically coupled to image display means 3
The step (8) of changing the transmittance of the image display means 3 in synchronization with the movement of the optical image 5 is repeatedly performed, and the obtained optical image 3 is photoelectrically converted and integrated in the image sensor 4. Is performed to perform a spatial convolution operation on the optical image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus, and more particularly to an image pickup apparatus in which an image pickup output signal of an image pickup apparatus is subjected to spatial convolution operation and outputted.

[0002]

2. Description of the Related Art Spatial convolution calculation such as a spatial filter of an image signal is performed by multiplying successive pixel signals by respective coefficients and adding them.

[0003]

In the conventional spatial convolution operation as described above, since the spatial convolution operation is performed on the image pickup output signal (one-dimensional signal), an independent circuit for the operation is required. There is a problem that high-speed processing is required. It is an object of the present invention to provide an imaging device in which the output signal of the imaging device is the calculation result of the spatial convolution operation as it is, and the above-mentioned problem is solved.

[0004]

According to the present invention, in order to achieve the above object, according to the present invention, the transmittance of a transmission type image display means, which is a component of the imaging apparatus of the present invention, is made to correspond to a coefficient of a spatial convolution operation. In addition, the optical image that has passed through the image display means is photoelectrically converted and integrated by a storage type two-dimensional area image pickup device (including a two-dimensional area CCD image pickup device) for successive pixels, so that the output of the image pickup device is obtained. The signal itself is made the operation result of the spatial convolution operation.

That is, an image pickup apparatus of the present invention comprises: an optical unit capable of moving an optical image in a spatial one-dimensional or two-dimensional direction; and a transmission-type optical unit capable of forming the optical image and changing the transmittance. An image display means, and a storage type two-dimensional area image pickup device optically coupled to the image display means, wherein a step of changing the transmittance of the image display means in synchronization with movement of the optical image is repeatedly performed. Then, the obtained optical image is subjected to photoelectric conversion and integration in the image pickup device, so that a spatial convolution operation is performed on the optical image.

Further, the image pickup apparatus of the present invention comprises an optical unit, a transmission type image display unit capable of forming an optical image by the optical unit and changing the transmittance, and optically coupled to the image display unit. A two-dimensional area CCD image pickup device capable of moving accumulated charges on an image pickup surface in a spatial one-dimensional or two-dimensional direction, and performing photoelectric conversion of an optical image by the image pickup device. A spatial convolution operation is performed on an optical image by repeatedly moving the accumulated charges in synchronization with a change in transmittance.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below based on embodiments of the present invention with reference to the accompanying drawings. FIG.
Shows a first embodiment of the imaging apparatus of the present invention. Note that both the present embodiment and a second embodiment to be described later are examples of an imaging apparatus that performs a convolution operation in a spatial one-dimensional direction. However, pixel movement is performed in both horizontal and vertical directions. Thus, a configuration in which spatial two-dimensional convolution operation is performed can be adopted.

In FIG. 1, incident light from a subject 1 is imaged on a transmission type image display device 3 by an optical system 2. As will be described later, the optical system 2 displays an image temporally moving at least over a pixel range to be subjected to the spatial convolution operation in the direction opposite to the one-dimensional direction in which the spatial convolution operation is performed, on the transmission type image display device 3. Is controlled by the optical image movement control device 7 so as to form an image (subject image 5).
In this case, at a certain point on the transmission type image display device 3, the image 5 corresponding to the pixel to be subjected to the spatial convolution operation is continuously displayed in the operation order.

Further, the transmission type image display device 3 includes an optical system 2.
Is controlled by the transmittance control device 8 so that the transmittance corresponding to each coefficient corresponding to each target pixel of the spatial convolution operation which moves and forms an image is provided in pixel units and in synchronization with the movement of the image. That is, focusing on one point on the transmission type image display device 3, a transmittance corresponding to the coefficient of the spatial convolution operation is given in synchronization with the movement of the image, and the transmitted light multiplied by the coefficient of the spatial convolution operation is Injection is performed sequentially according to the calculation order. When the optical image passes through, the image 6 multiplied according to the coefficient of each pixel of the display device enters the storage type imaging device 4. At this time, if the display device 3 and the imaging device 4 are arranged sufficiently close, the image will not be blurred. Further, when the distance is large to some extent, another optical system may be inserted between the display device 3 and the imaging device 4 to form an image again (this is not shown).

[0010] Figure 2 illustrates the process of calculating convolution of the spatial one-dimensional direction of pixels y _n of the image pickup device on a time axis in the first embodiment described above. As shown in the figure, the imaging device of the present invention multiplies the level x of the optical image at successive times from time t1 to t2 by a coefficient (transmittance) a of the operation at that time, and integrates over the above time ( Signal added in the time direction)

(Equation 1) Is obtained (accumulated in the imaging device y _n is) it can be seen. This is the same as the result of the spatial convolution operation formed by another conventional circuit.

In the configuration of FIG. 1, as a mechanism for temporally moving the optical system 2 over a pixel range to be subjected to spatial convolution calculation, for example, Ishizuka et al.
Correction of Camera Shake with TR, Prism and Angle Sensor ", Nikkei Electronics, 1992, 7.6, (No. 558), and other configurations can be applied.

Further, regarding the transmission type image display device 3,
The transmittance is controlled by the transmittance control device 8 so as to give a transmittance corresponding to each coefficient corresponding to each target pixel of the spatial convolution operation in synchronization with the movement of the optical system 2 described above. As the transmission type image display device whose transmittance can be controlled, for example, a liquid crystal display device can be used. As described above, the optical image moving temporally within the range of the spatial convolution operation and multiplied by the operation coefficient is stored in the storage type (two-dimensional area) imaging device 4.
By integrating with, the result of the convolution operation is
It is obtained as an output signal of this imaging device.

Note that this calculation must be performed within a period that satisfies the dynamic resolution of the image system to be applied, for example, 1/60 second in a television system. Also,
The above operation completes a series of spatial convolution operations.
When performing the spatial convolution operation continuously, the same operation is repeated by returning the imaging position of the optical system 2 to the original position, or
The configuration is such that the spatial convolution calculation order is reversed and the calculation is performed without returning the imaging position of the optical system 2 to the original position.

Next, a description will be given of a second embodiment of the imaging apparatus according to the present invention. FIG. 3 shows a second embodiment of the imaging apparatus of the present invention. This embodiment is the same as the first embodiment (see FIG. 1) in that the result of the spatial convolution operation is obtained as the output signal of the imaging device. In particular, the optical image is a storage type imaging device. Before reaching, the same optical image is given the coefficient of the spatial convolution operation by the transmission type image display device constituting the imaging device of the present invention, and the integration is performed by the accumulation type imaging device.

However, the difference is that in FIG. 3, the imaging position (optical image) is not moved by the optical system 2 (that is, the optical system is fixed), and the CCD imaging device 9 is used as the imaging device. The point is that the charge image in space is moved by this (CCD imaging device) instead of the movement of the optical image. C
The CD imaging device generates a charge according to an optical image by applying a drive pulse generated by the CCD drive control device 10 to each electrode, and spatially moves the charge stored on the imaging surface. Can be. By performing this operation in accordance with the procedure shown in FIG. 4 described below, at the end of this operation, the result of the spatial convolution operation can be obtained as electric charges on the CCD elements constituting the CCD imaging device.

FIG. 4 is an explanatory diagram for explaining the operation of the spatial convolution operation in the second embodiment of the imaging apparatus of the present invention. FIG. 4 shows an example of a spatial convolution operation of one-dimensional space and having three terms.
Here, a description will be given of how temporally the generated charges are accumulated in the pixel yn _{+ 1} of the CCD image pickup device and the charges obtained by the spatial convolution operation are obtained.

First, at time t = t1, the transmittance of the transmissive image display device is set to a1 at the pixel yn _-1 of the CCD image sensor.
As a result, a signal charge of y _n−1 = x _n−1 · a1 is stored. Next, the charges are spatially moved on the imaging surface. Since the CCD image sensor is generally separated for each pixel, this operation is a discrete convolution operation,
The minimum unit is one pixel interval.

Next, at time t = t2, the CCD image pickup device
Pixel y_nThe transmittance of the transmission type image display device is a2
Thus, y_n= X_n-1・ A1 + x_n・ A2 signal power
The load is stored. Further, the electric charge is moved. further,
At time t = t3, the pixel y of the CCD image pickup device_{n + 1}, Transparent
By setting the transmittance of the image display device to a3, y
_{n + 1}= X_n-1・ A1 + x_n・ A2 + x_{n + 1}・ A3 signal
Electric charge is stored. This value is the pixel y_n3 taps
This is the result of performing the space convolution operation of the loop.

In the second embodiment, as in the first embodiment, an example of the spatial one-dimensional convolution operation has been described. However, by further moving horizontally and performing the same operation, the spatial A two-dimensional spatial convolution operation can be performed. In the second embodiment, the CCD is also used.
The above operation needs to be performed within a period that satisfies the dynamic resolution of the applied imaging system.

In any of the above-described embodiments, it may be necessary to determine which original image has undergone the spatial convolution operation. In any of the above-described embodiments, the spatial convolution operation must be performed before starting the spatial convolution operation. This problem can be solved if the original image is taken from an imaging device (in the second embodiment, a CCD imaging device) into another memory.

[0021]

In the prior art, the spatial convolution operation for spatial filtering or the like has been performed by providing a device for this purpose outside the image pickup device. The device usually includes an AD converter, a memory, a multiplier, an adder, a DA
It consists of a converter and the like. According to the present invention, these devices are not required, and the output of the imaging device is directly the result of the spatial convolution operation.

[Brief description of the drawings]

FIG. 1 shows a first embodiment of the imaging apparatus of the present invention.

2 shows a process of calculating convolution of the spatial one-dimensional direction of pixels y _n of the image pickup device on a time axis in the first embodiment.

FIG. 3 shows a second embodiment of the imaging apparatus of the present invention.

FIG. 4 shows an operation of a spatial convolution operation in the second embodiment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 subject 2 optical system 3 transmission type image display device 4 storage type imaging device 5 subject image formed on display device 6 multiplied image 7 optical image movement control device 8 transmittance control device 9 CCD imaging device 10 CCD drive control device

Claims (2)

[Claims] 1. An optical unit capable of moving an optical image in a spatial one-dimensional or two-dimensional direction, a transmission-type image display unit capable of forming the optical image and changing a transmittance, and A storage type two-dimensional area image pickup device optically coupled to the image display means; and a step of repeatedly changing a transmittance of the image display means in synchronization with movement of the optical image, thereby obtaining an optical image. An image pickup apparatus, wherein a spatial convolution operation is performed on an optical image by performing photoelectric conversion and integration of the image in the image pickup device. 2. An optical unit, a transmission type image display unit capable of forming an optical image by the optical unit and changing transmittance, and optically coupled to the image display unit and spatially one-dimensional. Alternatively, a two-dimensional area CCD capable of moving accumulated charges on the imaging surface in a two-dimensional direction
And performing spatial convolution operation on the optical image by repeating the movement of the accumulated charge in synchronization with a change in transmittance of the image display unit during photoelectric conversion of an optical image by the image sensor. An imaging device characterized in that: