JPH0993497A - Image processing unit and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide the image processing unit in which a processing time is short by receiving an image with high resolution through the picture element shift method even if an image pickup element with a few pixel numbers is employed and binarizing is processing is performed simultaneously with photographing. SOLUTION: An image received by a CCD 103 without driving a parallel flat plate 125 is binarized by a binarization processing section 127 based on a designated threshold level and the result is displayed on a display section 115. A proper threshold level is decided by observing a display screen. Then plural images obtained by binarizing the images received by the CCD 103 by driving the parallel flat plate 125 based on the decided threshold level are produced and they are composited into one image.

Description

Detailed Description of the Invention

[0001]

TECHNICAL FIELD The present invention relates to image processing.
More specifically, the present invention relates to image processing in which a high-resolution image is input by a pixel shift method and at the same time binarization processing is performed while shortening the processing time while using an image input unit having insufficient resolution. It is a thing.

[0002]

2. Description of the Related Art In recent years, as the performance of personal computers has become higher and the performance of high performance output devices such as color printers has become widespread, and more multimedia has been created, the opportunities for inputting, displaying, storing, printing and transmitting images have increased.

There is a scanner as an image input means, which can capture a document or photograph printed on paper at high resolution, but cannot capture a three-dimensional object, takes a long time to input an image, and has poor operability. There are problems such as. Therefore, an image processing system using a VTR camera as an image input means has begun to spread widely.

A block diagram of a conventional image processing apparatus of this type is shown in FIG. In FIG. 5, the overall control unit 510 includes an address signal 519, a data signal 520, and a control signal 521.
By sending signals adaptively to the lines, data is transmitted / received and controlled to / from each controlled block. Address signal 5
Each controlled block is selected by 19. As the data signal 520, data exchanged between the control unit 510 and the controlled unit is output. The control signal 521 includes
Control information is exchanged between the blocks, whereby arbitration is performed so that an error such as a collision of the address signal 519 and the data signal 520 does not occur.

The main memory 514 stores a program for operating the control unit 510, control information of each functional block, and data such as image data, and the contents thereof are changed as necessary.

Although not shown, an input device such as a keyboard or a mouse operates as an address signal 519 and a data signal 5.
20 and the control signal 521 are connected to each other so that the user can successively input data.

The lens is fixed to a frame (not shown), and the light transmitted through the lens portion 501 is an image pickup element C.
An image is formed on a CD (Charge Coupled Device) 503. Photoelectric conversion elements are two-dimensionally arranged in the CCD 503, and charges corresponding to the intensity of light incident on each photoelectric element are accumulated. Further, a color filter array 502 is arranged on the front surface of the CCD 503, and charges corresponding to the intensity of light transmitted through each color filter of the filter array are accumulated in the photoelectric conversion element. As a result, the distribution of each color component is converted into charge information and obtained. Typical color filter arrays include Cy (cyan) and Ye.
There is one in which complementary color filters corresponding to (yellow), Mg (magenta), and Gr (green) are arranged in a checkered pattern.

Further, the lens unit 501 is a lens control unit 50.
The control of 8 controls zooming, AF (autofocus), and the like.

The charge information accumulated in the CCD 503 is C
The charge information of the two-dimensionally arranged photoelectric conversion elements is sequentially output as an analog image signal according to the timing generated by the CD control unit 509. This analog image signal is an analog signal containing charge information of each photoelectric conversion element as voltage information.

The analog image signal is converted into a digital signal by the A / D converter 504 and supplied to the color processor 505. In the color processing unit 505, complementary colors Cy and Y converted into digital signals are obtained.
Y (luminance) / C (color difference) data is generated from the image data of e, Mg, and Gr, and the C signal is separated into U and V data by color difference. This YUV signal is converted into R (red), G (green) and B (blue) data by color space conversion processing. White balance is adjusted based on the RGB signals generated here. Gamma conversion is performed on the RGB data after the white balance adjustment, color space conversion is performed again, and the Y / C data is restored. The conversion at this time is an inverse conversion of the previous color space conversion. Further, as a correction, correction such as linear matrix conversion or clip matrix conversion is performed to perform digital Y conversion.
UV signal.

The human eye is sensitive to changes in brightness,
Since the color difference is not as sensitive as the luminance, it is considered that there is no problem in terms of image quality with respect to the U and V components even if they are dot-sequential. This also has the advantage of leading to a reduction in the amount of memory, and is supplied to the frame memory control unit 506 as image data of the YU / V signal (U and V components are in point order). The number of pixels of the image data output here is C
The number of pixels depends on the arrangement of the photoelectric conversion elements of the CD 503, and is generally 640 (horizontal) × 480 (vertical), which is the same number of pixels as a TV. Each pixel has a luminance and color difference (U or V component) of 8 bi
It is expressed as data of t.

Although the flow of color processing is generally described here, the processing method is not limited to this, and color processing is performed as it is as an analog signal, and A / D conversion is performed at the final stage. It may be.

The frame memory control unit 506 clips image data from the position designated by the overall control unit 510 at a designated size, and stores the image in the frame memory 507 at a designated timing. The frame memory control unit 506 stores and reads the image data with the frame memory 507 by the address signal 516, the image data signal 517, and the frame memory control signal 518.

The image data once stored in the frame memory 507 is transferred to the main memory 514 by the frame memory control unit 506 controlled by the overall control unit 510.

The image data stored in the main memory 514 is processed according to the user's request.

The display control unit 511 is composed of a color space conversion unit, a display memory, a display memory control unit, and a D / A conversion unit, and is displayed from the main memory 514 for display on the display unit 515. The data to be displayed is the display control unit 51.
Forwarded to 1.

When displaying image data on the display unit 515, the image data is first transferred to the display control unit 511.
The display control unit 511 has a memory having a capacity corresponding to the number of pixels displayed on the display unit 515, and stores image data in a designated area in the memory. At this time, the format of the image data processed in the display control unit 511 is expressed in RGB, and therefore the image data supplied in YUV is converted into data expressed in RGB by color space conversion. By storing this data in a predetermined position of the display memory, it becomes possible to combine it with the image displayed on the display unit 515. The data stored in the display memory is read in synchronization with the display control synchronization signal 522, A / D converted, and supplied as an image signal 523 to the display 515 for display.

Similarly, when data is stored in the storage unit 513, the hard disk drive and the magnetic disk D are similarly transferred via the lines of the address signal 519, the data signal 520 and the control signal 521.
It is stored in the storage unit 513 composed of a rive or the like.

LAN as a more important application
Image data may be sent to another terminal in a remote place via a (Local Area Network) or a general public line. For example,
Address signal 519, data signal 520, control signal 52
The image data transferred to the communication unit 512 configured by a FAX device or the like via the line 1 is compression-encoded by the encoding method adapted to the communication unit 512 and transmitted to the communication line 524.

Although a video camera capable of inputting an image as an input device has been described as a constituent element here, other image input devices such as a still camera and a scanner can be used without any problem.

[0021]

As described above, according to the conventional example, it is possible to output image data input from various image input devices to various output devices.

However, when the input image data is transferred to a terminal at a remote place using a communication line, for example, RGB
Sending multi-valued image data represented by each 8 bits is not very practical at this time due to band limitation of the communication line.

Therefore, there is a method of temporarily creating a histogram of each value from multi-valued image data, binarizing it with an appropriate threshold value, and then encoding and transmitting it. However, this method produces a visually clear image. There is a problem that it cannot always be sent.

The number of pixels of a commonly used VTR camera has a resolution of about 640 vertical × 840 horizontal pixels, and when an A4 size original is photographed, its resolution is
It's only 100 dpi. G3 commonly used
The FAX has a resolution of about 200 dpi and has a problem that the resolution is only half.

Therefore, in order to raise the resolution in a pseudo manner, there is a method of using a pixel shift method in which the optical axis incident on the CCD is slightly displaced for photographing, but it takes time to displace and photograph. Moreover, there is a problem that the binarization process takes time due to the increase in the image data.

The present invention has been made under such a circumstance. Even if an image pickup device having a small number of pixels is used, a high resolution image can be inputted by the image pickup by the pixel shift method, and while the image pickup is carried out. It is an object of the present invention to provide an image processing device and an image processing method that can reduce the processing time by performing binarization.

[0027]

In order to achieve the above object, in the present invention, an image processing apparatus is configured as in the following (1) to (3) and an image processing method is configured as in (4). .

(1) An image pickup device, and an optical axis displacement means for minutely displacing the position of the optical axis incident on the photosensitive surface of the image pickup device,
Binarizing means for binarizing a signal of an image photographed by the image pickup element with an appropriate threshold value, display means for displaying an image binarized by the binarizing means, and indicating the threshold value. And a control unit for operating the optical axis displacing unit to control a plurality of images photographed by the image pickup device and binarized by the binarizing unit to be combined into one image. An image processing apparatus equipped with.

(2) The image processing apparatus according to (1) above, wherein the optical axis displacement means uses a parallel plate.

(3) The image processing apparatus according to (1), wherein the optical axis displacement means uses a variable apex angle prism.

(4) An image is fetched by frame reading, a binarized image obtained by binarizing the image with a designated threshold is displayed, and the displayed binarized image is used to set the threshold. Decide, capture the image by field reading,
An image processing method for synthesizing and outputting one image from a plurality of binarized images based on a threshold value determined previously.

[0032]

BEST MODE FOR CARRYING OUT THE INVENTION According to the present invention, the processing time can be shortened by carrying out the binarization processing while raising the resolution in a pseudo manner by the image pickup by the pixel shift method.

The present invention will be described in detail below with reference to an embodiment for inputting a color image. In the embodiment, parallel plate driving and binarization are sequentially performed to form each binarized image.

[0034]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram showing the configuration of an "image processing apparatus" which is an embodiment. The following description is based on this block diagram, focusing on the points different from the conventional example.

A parallel plate 125 is arranged between the lens portion 101 and the CCD 103. The parallel flat plate 125 slightly shifts the optical axis to form an image on the photoelectric conversion element surface (photosensitive surface) of the CCD 103. This makes it possible to change the resolution of the CCD 103, that is, the number of pixels.

This principle is shown in FIG. The parallel plate 201 is rotatable about an axis perpendicular to the plane of the drawing and passing through the center of the parallel plate 201, and is parallel to the photoelectric conversion element surface 202 of the CCD 103 as shown in FIG. It has become. At this time, the optical axis 203 incident on the parallel plate 201 and the optical axis 204 after transmission are on the same straight line.
Next, as shown in FIG. 2B, when the parallel plate 201 is rotated by a slight angle, the optical axis is bent by the parallel plate 201, and the optical axis 204 is displaced in parallel to the optical axis 203.
The light of 05 will enter.

Therefore, even if the number of photoelectric conversion elements arranged in the CCD 103 remains unchanged, the light beam incident on the optical axis 204 is photographed before and after it is changed by rotating the parallel plate 201, and the photographed image is obtained. By combining with, it is possible to obtain an image in which the number of pixels in the vertical direction on the paper surface is increased. By rotating the parallel plate 201 about an axis orthogonal to the above-mentioned axis, it is possible to similarly obtain an image in which the number of pixels in the direction perpendicular to the paper surface is increased.

Next, the relationship between the color filter array 102 in the CCD 103 and the number of pixels will be described.

FIG. 3 shows an example of the color filter array. The figure shows a part of the color filter array,
Cy (cyan) 301, Ye (yellow) 302, Mg
Color filters of (magenta) 303 and Gr (green) 304 are arranged in a checkered pattern. Mg303, Gr
304 is inverted and arranged for each line.

In the case of the conventional example without the parallel plate 125, the addition output of the charges of the upper and lower pairs (301 and 303, 302 and 304) of the lines indicated by 305 is output in the first field, and the line indicated by 306 in the second field. The added charge amount of the pair is output. The data adjacent to each other in the horizontal direction of the addition output is further processed as a pair to perform color processing to obtain color information. Therefore, at this time, Cy301, Ye
Data of one pixel is generated by a pair of four color filters 302, Mg303, Gr304. This read mode is called "frame read mode".

However, in the above-mentioned reading method, the number of pixels corresponding to the photoelectric conversion elements of the CCD 103 can be obtained.
The spatial resolution is reduced. Therefore, in the case of photographing by the pixel shift method, the amount of charges corresponding to the light transmitted through a single color filter is read out in each field by reading line by line 307 and 308. Further, at this time, the parallel plate 125 captures four images in which the optical axes are shifted vertically and horizontally by the half pitch of the color filter and synthesizes them to obtain pixels that are twice as wide and long as the CCD 103. This read mode is called "field read mode".

Up to this point, the method of substantially increasing the number of pixels has been described.

Furthermore, in this application, 2
A binarization processing unit 127 that binarizes the image data in order to transmit the binarized image data using the communication unit 112 is added. The output of the CCD 103 digitally converted by the A / D 104 is supplied to the binarization processing unit 127, which is binarized by comparing with an adaptively determined threshold value, and the result is obtained by the frame memory control unit. To 106. Here, a method of setting a threshold value for performing binarization will be described later.

Next, the overall flow of this embodiment will be described with reference to the flowchart of FIG.

First, the initial setting S1 of the camera section is performed. At this time, the parallel plate 125 is parallel to the photoelectric conversion element surface of the CCD 103.

When the subject is set, the angle of view of the camera is determined, and the photographing is ready, the system is notified in S2 that the image can be captured. The method may be such that the menu screen is displayed and the input is performed by the keyboard or the mouse.

When the start of image capturing is detected in S2, the image is captured in the "frame read mode" described above in S3, and the result is displayed on the display unit 115 in S4. Since the purpose of this display is to determine the threshold value for binarization, it may be displayed in gray scale instead of a color image such as RGB.

Next, in S5, the user inputs an appropriate threshold value, and the result of binarizing the threshold value is displayed on the display unit 115 in S6. At this time, the screen may be different from the display in S4 or may be overwritten. In S7, the user waits for the input of the judgment result as to whether or not the threshold value previously input may be visually viewed.

S is satisfied until the binarized result is satisfied.
Repeat S5, S6 and S7, and when the threshold value is determined, S8
Proceed to. In S8, the user waits for the input of the judgment result as to whether to start over from the beginning.

When both the angle of view and the threshold value are determined, the threshold value is set in S9. Here, in S3, the image data pre-scanned in the "frame reading mode" and S5
From the threshold value determined by
A threshold value corresponding to each color filter is calculated, and the result is set in the binarization processing unit 127.

S10, S13 and S14 are parallel plates 12
5 shows a control for driving 5 times four times to capture an image in which the optical axis is shifted by half pitch.

At this time, in S1, the CCD 103 is operated in the "field read mode" to output the charge amount corresponding to the luminance transmitted through each color filter, and the charge amount is output to the A / D1.
After being converted into a digital amount in 04, the digital amount is supplied to the binarization processing unit 127. The binarization processing unit 127 binarizes the input data with the threshold value corresponding to each color filter set in S9 and supplies it to the frame memory control unit 106 (S12).

The frame memory control unit 106 determines S15.
Then, the image data binarized four times and binarized are rearranged so as to be spatially aligned, and the main memory 1
Transfer to 14 once.

The binarized image data stored in the main memory 114 is transferred to the communication unit 112, compression-encoded, and transferred to a desired place via the communication line 124.

In this embodiment, the parallel plate 140 is used as a means for changing the number of pixels of the image input section, and the optical axis is optically shifted to apparently increase the number of pixels. Also, the same effect can be obtained by enclosing silicon oil between two glass plates and using an active prism (variable apex angle prism) capable of changing the apex angle of the glass plates.

As described above, according to this embodiment,
A high-resolution image can be input even if an image sensor having a small number of pixels is used, and a clear image binarized with an appropriate threshold value visually can be input in a short time.

[0057]

As described above, according to the present invention,
Even if an image sensor having a small number of pixels is used, a high-resolution image can be input by shooting by the pixel shift method, and the processing time can be shortened by performing binarization while shooting.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment.

FIG. 2 is a diagram showing the principle of high resolution using a parallel plate.

FIG. 3 is a diagram showing a color filter pattern in a filter array used in an example.

FIG. 4 is a flowchart showing the operation of the embodiment.

FIG. 5 is a diagram showing a configuration of a conventional example.

[Explanation of symbols]

 101 lens part 103 CCD 110 overall control part 115 display part 125 parallel plate 127 binarization processing part

Claims (4)

[Claims] 1. An image pickup device, an optical axis displacement means for minutely displacing a position of an optical axis incident on a photosensitive surface of the image pickup device, and a signal of an image photographed by the image pickup device is binary with an appropriate threshold value. Binarizing means for digitizing, display means for displaying an image binarized by the binarizing means, instructing means for instructing the threshold value, and the optical axis displacing means for operating the image pickup device. An image processing apparatus comprising: a control unit that controls a plurality of images that have been captured and binarized by the binarization unit to be combined into a single image. 2. The image processing apparatus according to claim 1, wherein the optical axis displacing means uses a parallel plate. 3. The image processing apparatus according to claim 1, wherein the optical axis displacement means uses a variable apex angle prism. 4. An image is captured by frame reading, a binarized image obtained by binarizing the image with a designated threshold value is displayed, and the threshold value is determined by looking at the displayed binarized image. Then, the image processing method is characterized in that an image is fetched by field reading, and one image is synthesized from a plurality of binarized images based on a threshold value previously determined and output.