JPH09181981A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image different in number of pixels to various output devices. SOLUTION: An optical axis 142 of a CCD 103 with respect to an image pickup plane is displayed by tilting parallel flat plates 140 and one image is obtained by compositing plural images obtained picking up the image at plural displaced positions respectively. The number of pixels of the image is changed in a various way by controlling the displacement direction and the displacement amount or the like of the optical axis 142. Thus, the image having an optimum number of pixels is fed to a display section 110, a print section 114 and a storage section 116 or the like.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup device capable of picking up images in accordance with the number of pixels required by various output devices.

[0002]

2. Description of the Related Art In recent years, as the performance of personal computers has increased, and high-performance output devices such as color printers have become widespread, and multimedia has become available, there are increasing opportunities to input, display, store, and print images. There is a scanner as an image input means, but this can capture images printed on paper and photographs in high resolution, but it can not capture three-dimensional objects, it takes time to input images, poor operability, etc. There was a problem. Therefore, V as an image input means
Image processing systems using TR cameras have begun to spread widely.

FIG. 5 is a block diagram showing the structure of a conventional image processing apparatus. The control unit 517 adaptively outputs the address signal 531, the data signal 532, and the control signal 533, thereby transmitting / receiving and controlling data to / from each controlled block. Each controlled block is selected by the address signal 531. The data signal 532 includes the control unit 51.
7 and data to be controlled between the controlled block. The control signal 531 is the address signal 53 when the control information is exchanged between the controlled blocks.
1. Arbitration is performed so that an error such as a collision of the data signal 532 does not occur. The main memory 518 has a controller 51
Stores programs for operating 7, control information for each controlled block, data such as image data,
Their contents are changed as necessary. Although not shown, an input device such as a keyboard or a mouse is connected via an address signal 531, a data signal 532, and a control signal 533 so that the user can input data sequentially.

The lens 501 is fixed to a frame (not shown), and its optical axis 520 forms an image on the CCD 503. Photoelectric conversion elements are two-dimensionally arranged in the CCD 503, and charges corresponding to the intensity of light incident on each photoelectric conversion element are accumulated. Further, the photoelectric conversion elements of the CCD 503 are provided with complementary color filters 502 corresponding to Cy (cyan), Ye (yellow), Mg (magenta), and G (green) in a checkered pattern. Further, the lens unit 501 is controlled by zooming, AF, etc. by a control signal 523 from the lens driving unit 507. The lens driving unit 507 is controlled by the control signal 529 from the camera control unit 511.

The CCD 503 sequentially outputs, as an analog image signal 521, the charge information accumulated in the photoelectric conversion elements which are two-dimensionally arranged according to the timing control signal 524 of the CCD driving section 508. This analog image signal 5
Reference numeral 21 is an analog signal including charge information of each photoelectric conversion element as voltage information. The operation state of the CCD driving unit 508 is controlled by the control signal 530 supplied from the camera control unit 511, and at the same time, the synchronization signal 525 supplied from the synchronization signal generating unit 512 causes the two-dimensional arrangement of the photoelectric conversion elements. The read position is controlled.

The color processing section 504 first A / D converts the supplied analog image signal 521 into a digital signal. Next, complementary colors Cy, Ye, and M that became digital signals
Y (luminance) / C (color difference) data is generated from the g and G image data, and the C signal is color difference separated into U and V data. This YUV signal is converted into R by color space conversion processing.
Convert to (red), G (green), and B (blue) data. White balance adjustment is performed based on the RGB signals generated here. Next, the gamma conversion is performed on the RGB data after the white balance adjustment, the color space conversion is performed again, and the Y / C data is restored. The conversion at this time is an inverse conversion of the previous color space conversion. Further, as a correction, correction such as linear matrix conversion or clip matrix conversion is performed to obtain a digital YUV signal.

Although the human eye is sensitive to changes in luminance, it is not as sensitive to luminance as it is to luminance, so it is considered that there is no problem in terms of image quality even if the U and V components are dot-sequential. Further, this has an advantage that it leads to a reduction in the memory amount, and is supplied to the image input control unit 505 as image data 522 of the YU / V signal (U and V components are dot sequential). Also,
The pixel number of the image data output here is the CCD 503.
In general, the number of pixels is approximately 640 × vertical 480, which is approximately the same as that of a TV, depending on the arrangement of the photoelectric conversion elements. The luminance and color difference (U or V component) of each pixel are represented as 8-bit data. Although the flow of color processing has been generally described here, the processing method is not limited to this, and color processing may be performed as it is as an analog signal and A / D conversion may be performed at the final stage. Good.

The image input control unit 505 clips from the position designated by the control unit 517 at a designated size, and stores image data in the frame memory 506 at a designated timing. The image input control unit 505 performs storage and reading with respect to the frame memory 506 by an address signal 526, image data 527, and a frame memory control signal 528. The image data once stored in the frame memory 506 is set by the control unit 517 to be transferred to the main memory 518 by the image input control unit 505, and then the image data is transferred to the main memory 518.

The image data stored in the main memory 518 is displayed on the display unit 510, the print unit 514, and the storage unit via the display unit control unit 509, the print control unit 513, and the storage control unit 515 in response to the user's request. It is output to 516. The display control unit 509 includes a color space conversion unit, a display memory, a memory control unit thereof, and a D / A conversion unit. When displaying the image data on the display unit 510, the image data is first transferred to the display control unit 509. The display control unit 509 has a memory having a capacity corresponding to the number of pixels displayed on the display unit 510, and stores image data in a designated area in the memory. At this time, the format of the image data processed in the display control unit 509 is expressed in RGB, and therefore the image data is converted into data expressed in RGB by color space conversion. By storing this data in a predetermined position of the display memory, it is possible to combine the data with the image displayed on the display unit 510. The data stored in the display memory is read in accordance with the control signal 533 for controlling the display unit. D / A
It is displayed by being converted and supplied to the display unit 510.

Similarly, when outputting to the print unit 514, the image data is transferred to the print control unit 513, converted into the data format used by the print unit 514, and then the print data signal 536 and the print control signal 5 are transmitted.
37 and are supplied to the printing unit 514 and printed. Similarly, when storing in the storage unit 516,
The image data is transferred to the storage control unit 515, and the storage unit 51 uses the storage data signal 538 and the storage control signal 539.
6 is supplied to and accumulated. It is also possible to read this accumulated image data and supply it to another output unit.

Further, although not shown in FIG. 5,
The communication i / f may be provided as another output unit to enable communication with other devices. here,
Although a camera capable of inputting an image as an image input means has been described as a component, other image input devices such as a still camera and a scanner may be used.

[0012]

As described above, according to the conventional example, it is possible to output the image data input from various image input devices to various output devices. But,
The number of pixels of a commonly used VTR camera is 64 vertical
While 0 × horizontal 480 pixels is used, in a display device such as a display, a vertical 320
The number of pixels smaller than that of a VTR camera, such as 240 pixels horizontally, is used. However, when outputting to a printer, a picture of such a size results in a very small picture.

As described above, various image output devices have a pixel size suitable for each, but the pixel size of the image input device is constant, so that an image output device requiring a particularly large number of pixels. Then, the process of expanding was needed. However, there is a problem that some kind of interpolation is necessary to enlarge the image, and the deterioration of the image quality cannot be avoided.

Further, in such an image processing apparatus, the image data is output not only to one image output apparatus but also to a plurality of image output apparatuses, and various trial and error such as examination of layout is carried out to obtain a final output result. Work to obtain. Therefore, when there is only one setting of the number of pixels of the image input device, there is a problem that an output result suitable for each output device cannot be obtained.

The present invention has been made to solve the above problems, and an object thereof is to obtain an image pickup apparatus capable of obtaining an image having a pixel number suitable for various output apparatuses.

[0016]

According to the present invention, a large number of photoelectric conversion elements are two-dimensionally arranged on an image pickup surface to output an image signal, and an optical axis for displacing the optical axis of the image pickup means with respect to the image pickup surface. Displacement means and synthesizing means for synthesizing a plurality of image signals obtained by the imaging means for imaging a plurality of positions of the optical axis displaced by the optical axis displacing means are provided.

[0017]

According to the present invention, by changing the amount and direction of displacement of the optical axis by the optical axis displacement means, when a plurality of images taken at each displacement position are combined, the number of pixels is different.
A single image can be obtained and the image can be sent to various output devices depending on the number of pixels.

[0018]

FIG. 1 is a block diagram showing an embodiment of the present invention. In FIG. 1, 101 to 139 correspond to 501 to 539 in FIG. 5, respectively, and therefore description thereof will be omitted. Here, only parts different from FIG. 5 will be described. In FIG. 1, a transparent parallel plate 140 is arranged between the lens 101 and the CCD 103, and a parallel plate driving unit 141 is provided. This parallel plate 14
The optical axis 120 is slightly shifted by 0, and the optical axis 142
Then, an image is formed on the CCD 103. This makes it possible to change the resolution of the CCD 103, that is, the number of pixels.

This principle diagram is shown in FIG. In FIG. 2A, the parallel plate 201 is perpendicular to the plane of the drawing and is parallel plate 201.
It is possible to rotate around an axis passing through the center of.
The parallel plate 201 is initially parallel to the photoelectric conversion element surface of the CCD 202. At this time, the optical axis 203 incident on the parallel plate 201 and the optical axis 204 after transmission are aligned with each other. Next, when the parallel plate 201 is rotated by a slight angle, the optical axis 203 is bent as shown in FIG. 2B, and the optical axis 204 is displaced in parallel to the optical axis 203.
The light of 05 will enter. The amount of displacement depends on the rotation angle and the refractive index of the parallel plate. Therefore, even if the number of photoelectric conversion elements arranged in the CCD 202 remains unchanged,
The light incident on the optical axis 204 is photographed before and after the parallel plate 201 is slightly rotated, and the photographed images are combined to obtain an image in which the number of pixels in the upward direction of the paper surface is substantially increased. Will be possible. In this description, the increase in the number of pixels in one direction is described.
By increasing the number of pixels, the number of pixels in two dimensions can be increased.

Next, the relationship between the color filter 102 provided in the CCD 103 and the number of pixels will be described. FIG.
Shows an example of the color filter. The figure shows a part of a color filter, in which Cy (cyan) 301, Ye (yellow) 302, Mg (magenta) 303, and G (green) 304 filters are arranged in a checkered pattern.
The Mg 303 and G 304 are arranged so as to be inverted for each line. In the case of the conventional example in which the parallel plate 140 is not provided, the first field is color-processed by the pair indicated by 305, and the second field is color-processed by the pair indicated by 306 to obtain color information.
At this time, Cy301, Ye302, Mg303, G30
Data of one pixel is generated by four pairs of four filters.

Next, a parallel plate 140 is used, and first Cy is used.
The light incident on 301 is controlled to enter Ye 302, and then changed to G304 and Mg 303. By doing this, Cy301, Y for one pixel
Color information transmitted through the four color filters e302, Mg303, and G304 can be obtained. As a result, the number of pixels is doubled vertically and horizontally. Further, as control of the parallel plate 140, an operation of shifting the entire image pickup point by half a pixel in the vertical and horizontal directions and photographing, and Cy30 for each pixel.
It is possible to obtain the number of pixels that is four times the height and width by combining with the shooting operation for obtaining the color information of 1, Ye302, Mg303, and G304.

Next, the operation from photographing to output of image data will be described with reference to the flowcharts of FIGS. 4 (a) and 4 (b). Here, the CCD 103 has 64 effective pixels.
The mode in which the image is 0 × 480 pixels and the parallel plate 140 is not operated is referred to as mode # 1. This mode #
1 is 640 × 480 by performing interpolation from data of four photoelectric conversion elements that are spatially different and are adjacent to each other.
Image data of. In addition, the parallel plate 140
Mode # 2, mode #
3 is assumed. Mode # 2 is a mode for capturing image data that has passed through all color filters from the same spatial point, and mode # 3 is for capturing image data of spatial interpolation points of pixel points captured in mode # 2. Shall be synthesized.

The flow of the operation when the image data is first displayed on the display unit 110 and the overall layout is confirmed and then the image data is stored in the storage unit 116 or output to the print unit 114 will be described below. First of all,
In step S1 of (a), the camera unit is initialized. Next, in step S2, the number of output pixels is set for each output device. For example, the output to the display unit 110 is 320 × 240 pixels of the image shot in mode # 1, and the output to the print unit 114 is 12 shot in mode # 3.
It is assumed that 80 × 960 pixels and 640 × 480 pixels captured in mode # 2 are stored in the storage unit 116. Also,
It is possible to set a plurality of output pixel numbers for one output device, such as setting 640 × 480 pixels captured in mode # 1 as the second setting for the display unit 110. The setting information is stored in the main memory 118.

Next, the start of image capturing is started in step S3.
When it is detected in step S4, a message for the user to select the number of output pixels is displayed on the display unit 110, and the number of output pixels is selected in step S4. In steps S5 and S6, the control information of the lens controller 107 is supplied to the camera controller 111 or the direction of the camera is changed so that the requested image can be captured.

When the display unit 110 is selected as the output device, the parallel plate 140 is not driven and photographing is performed in the # 1 mode. Therefore, in FIG. 4B, the step S7 of driving the parallel plate 140 is not performed, but the image is captured in step S8, and the image is captured once in step S9.
Ends with. In this case, an image of 640 × 480 pixels has been captured for one screen, and this is 320 in step S10.
After reduction to image data of × 240 pixels, step S1
1 is output to the display unit 110. Then, the process returns to the standby state for fetching the next image data in step S3. The reduction processing is performed by the image input control unit 105 in the frame memory 1
The image data is written to or read from the image data file 06.

To describe the control in this case, the control unit 117 sets the control information stored in the main memory 118 in the image input control unit 105. Image input control unit 1
Reference numeral 05 temporarily stores the area selected from the image data in the frame memory 106 according to this setting, and transfers it to the main memory 118 using the address signal 131, the data signal 132, and the control signal 133. After that, the control unit 11
7, the display control unit 109 is selected by the address signal 131, and the image data stored in the main memory 118 is sent to the display unit 110 via the display control unit 109. This mode is effective when trial and error is repeated at a low resolution in order to confirm the entire layout in advance.

Next, a case where image data is stored in the storage section 116 will be described. At this time, the shooting mode is Mode #
Use 2. After performing steps S1 and S2, if the start of capturing is instructed in step S3, first, in step S
In step 4, the output device is selected, and the storage unit 116 is selected.
Next, in steps S5 and S6, the direction of the camera and the zoom are adjusted to start capturing. Next, in steps S7 and S8, the parallel plate 140 is driven to capture the image data transmitted through the respective color filters for each pixel. That is, four times of photographing is performed, image data having color information of Cy301, Ye302, Mg303, and G304 is obtained for each pixel of 640 × 480 pixels, and these are subjected to step S
Synthesize at 10. This combined image data is processed in step S
At 11, the data is output to the storage unit 116. The output method is the display unit 11
This is the same as the method for 0.

The image input controller 105 may synthesize the image data in step S10.
Since the capacities of the frame memory 106 and the main memory 118 increase, the frame memory 106 is the CCD 1
The memory capacity for the number of pixels of 03 is prepared in advance, and the memory capacity for the number of pixels that can be formed by combining the whole is set in the main memory 11
It is advantageous in terms of cost to prepare the image processing unit No. 8 and perform the image combining process by the control unit 117.

Next, the case of outputting the image data to the printer unit 114 will be described. At this time, mode # 3 is used as the shooting mode. When the start of capturing is instructed in step S3 after steps S1 and S2, the output device is first selected in step S4, and the printing unit 114 is selected. Next, in steps S5 and S6, the direction of the camera,
After adjusting the zoom, start capturing. Step S
In 7 and S8, the parallel plate 140 is driven to capture the image data transmitted through the respective color filters for each pixel. This shooting is equivalent to the mode # 2. The same shooting is repeated three more times while shifting the spatial shooting pixel points. The four image data of 640 × 480 pixels thus photographed are combined in step S10 to form 1280 × 9.
The image data has 60 pixels. The combined image data is output to the print unit 114 in step S11. The output method is the same as the method for the display unit 110. By capturing the image data of the plurality of pixels set as described above, the output result of the image suitable for each output device can be obtained.

In the present embodiment, the parallel plate 14 is used as the optical axis displacement means for changing the number of pixels of the image input section.
Although 0 has been used to optically shift the optical axis to apparently increase the number of pixels, other than this, as the optical axis displacement means, silicone oil is sealed between two glass plates,
The same effect can be obtained by using an active prism capable of changing the apex angle of the glass plate. Furthermore, it is possible to increase the number of pixels that can be photographed by displacing the optical axis by a mechanical method in which the CCD 103 is vertically and horizontally displaced with respect to the optical axis 120.

[0031]

As described above, according to the present invention, it is possible to realize an image pickup device capable of obtaining an image having the number of pixels suitable for various output devices.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is a configuration diagram showing a principle of changing the number of pixels by a parallel plate.

FIG. 3 is a configuration diagram showing an example of a color filter.

FIG. 4 is a flowchart showing the operation of the present invention.

FIG. 5 is a block diagram showing a conventional imaging device.

[Explanation of symbols]

 103 CCD 105 image input control unit 106 frame memory 111 camera control unit 117 control unit 118 main memory

Claims (15)

[Claims] 1. An image pickup means in which a large number of photoelectric conversion elements are two-dimensionally arranged on an image pickup surface to output an image signal, an optical axis displacing means for displacing an optical axis of the image pickup means with respect to the image pickup surface, and the optical axis displacing means. An image pickup apparatus including a synthesizing unit that synthesizes a plurality of image signals obtained by the image pickup unit picking up images at a plurality of positions of the optical axis displaced by. 2. The image pickup apparatus according to claim 1, further comprising a reduction unit that reduces an image picked up by the image pickup unit. 3. The image pickup apparatus according to claim 1, wherein an optical means is used as the optical axis displacement means. 4. The image pickup apparatus according to claim 3, wherein a transparent parallel plate arranged in front of the image pickup surface and having a variable inclination angle with respect to the image pickup surface is used as the optical means. 5. An active lens comprising, as the optical means, two transparent plates having variable apex angles and a silicone oil enclosed between the two transparent plates. Imaging device. 6. The image pickup apparatus according to claim 1, wherein a mechanical means is used as the optical axis displacement means. 7. The image pickup apparatus according to claim 6, wherein a means for moving an image pickup surface of the image pickup means is used as the mechanical means. 8. The image pickup apparatus according to claim 1, further comprising control means for controlling a displacement amount and a displacement direction of the optical axis by the optical axis displacement means. 9. The synthesizing means synthesizes a plurality of image signals obtained by sequentially capturing images by the image capturing means while the optical axis displacing means is controlled by the control means, and has a different number of pixels according to a request. The image pickup apparatus according to claim 8, wherein a single image is created. 10. The synthesizing means creates an image having a number of pixels suitable for display on a display device.
An imaging device according to any one of the preceding claims. 11. The synthesizing means creates an image having the number of pixels suitable for printing by a printer.
An imaging device according to any one of the preceding claims. 12. The image pickup apparatus according to claim 1, wherein the synthesizing unit creates an image having a pixel number suitable for image storage. 13. The image pickup device according to claim 4, wherein the parallel flat plate is configured to slightly rotate on one rotation axis. 14. The image pickup device according to claim 4, wherein the parallel plate is configured to make a minute rotation on two rotation axes. 15. The image pickup apparatus according to claim 1, wherein a CCD is used as the image pickup means.