KR20130129645A - Multiple pinhole camera and method for operating the same - Google Patents

Abstract

The multi-pinhole camera according to an embodiment of the present invention, a pinhole array including a plurality of pinholes (pinhole array), a sensor array including a plurality of optical sensors for sensing the light passing through the pinhole and And a controller configured to generate a reconstructed image based on the detected light. Accordingly, the structure of the camera can be simplified and downsized, and a high quality image can be obtained.

Description

Multiple pinhole camera and method for operating the same}

The present invention relates to a camera and a method of operating the same, and more particularly, to a multiple pinhole camera and a method of operating the same using a plurality of pinholes.

Pinhole cameras are simple cameras that have been around for a long time. However, there was a difficulty in increasing light quantity and resolution.

On the other hand, digital cameras, which have been studied a lot recently, are basically made based on a camera structure made by an analog method long ago using film. Accordingly, it is necessary to use a lens, and there is a limit in miniaturizing a technology for photographing high quality pictures. In addition, in order to increase the amount of light in order to improve the quality of the picture, a large lens is required, and in consideration of the focal length caused by the lens, the camera may become large.

SUMMARY OF THE INVENTION An object of the present invention is to provide a multiple pinhole camera and a method of operation thereof that are simple in structure, small in size, and capable of obtaining high quality images.

The multi-pinhole camera according to an embodiment of the present invention, a pinhole array including a plurality of pinholes (pinhole array), a sensor array including a plurality of optical sensors for sensing the light passing through the pinhole and And a controller configured to generate a reconstructed image based on the detected light.

A method of operating a multiple pinhole camera according to an exemplary embodiment of the present invention includes detecting light passing through a plurality of pinholes, dividing a plurality of images based on the detected light, and synthesizing the plurality of divided images. It includes.

According to the present invention, the structure of the camera can be simplified, reduced in weight and downsized, and a high quality image can be obtained.

1 is a view schematically showing an example of a conventional pinhole camera.
2 to 6 are views referred to for explaining the multiple pinhole camera and its operation method according to an embodiment of the present invention.
7 is a block diagram of a multi-pinhole camera according to an embodiment of the present invention.
8 to 16 are diagrams for explaining various examples of a multiple pinhole camera and an operation method thereof according to an exemplary embodiment of the present invention.
17 to 22 are views referred to compare and explain a multiple pinhole camera and a lens camera according to an embodiment of the present invention.
23 is a flowchart illustrating a method of operating a multiple pinhole camera according to an embodiment of the present invention.

1 is a view schematically showing an example of a conventional pinhole camera.

The pinhole camera 100 is a camera that photographs light through a small hole, a pinhole 110. Thus, the pinhole camera 100 typically does not use a lens.

The conventional pinhole camera 100 usually consists of two boxes.

The pinhole 110 is drilled in the outer box to serve as a light collecting part. The pinhole 110 of the pinhole camera 100 should be close to a perfect circle without distortion of the image.

Films can be placed in the inner box to capture images. The focal length is adjusted by moving the inner box back and forth, and the shutter may be covered by hand or a suitable stopper may be used.

Meanwhile, the pinhole camera 100 may include an optical system such as an additional lens, or may include a component that processes the captured image 120 instead of or in addition to the inner box. In addition, it may be configured in other forms.

On the other hand, since the pinhole camera 100 does not have an aperture, the pinhole camera 100 is adjusted so that imaging is performed only by the exposure time. Therefore, there is a limit to the increase in the amount of light, and there is a limit to the increase in the resolution of the picked-up image.

On the other hand, the image 120 taken by the pinhole camera 100 is photographed inverted up, down, left and right with respect to the subject 150. As shown in FIG. 1, the image 120 formed through the pinhole 110 is turned upside down, left, and right. However, hereinafter, the top, bottom, left and right are shown in an inverted form based on the subject 150 for convenience of description.

2 to 6 are views referred to for explaining the multiple pinhole camera and its operation method according to an embodiment of the present invention.

Referring to the drawings, the multi-pinhole camera 200 according to an embodiment of the present invention, a pinhole array 210 including a plurality of pinholes, a plurality of light sensing the light passing through the pinhole A sensor array 230 including a sensor may include a controller 240 extracting a captured image and generating a reconstructed image based on the detected light.

Light from the subject 250 or other light sources passes through the pinhole array 210, sensed by the sensor array 230, and condensed into the image 220 on the sensor surface.

Meanwhile, as illustrated in FIG. 2B, the image formed on the sensor surface through the pinhole array 210 overlaps the image due to diffraction and diffusion of light passing through the plurality of pinholes provided in the pinhole array 210. Phenomenon occurs. FIG. 2 (a) briefly illustrates an image 220 in which an overlap phenomenon occurs.

On the other hand, the control unit 240 may perform the processing operation of these images, and may perform the extraction operation of the captured image and the generation of the reconstructed image desired by the user.

As shown in FIG. 3A, the sensor array 230 is provided inside the multiple pinhole camera 200 to detect light passing through the pinhole array 210, and to detect the light and / or the detected light. The image of the based subject can be converted into an electrical signal.

On the other hand, since the light passing through the pinhole array 210 can be diffused and spread wider than the size of the pinhole array 210, as shown in FIG. 3 (b), the sensor array 230 is more than the pinhole array 210 It is further desirable to configure the sensor surface on which the sensors are arranged to cover a wide surface.

The optical sensor included in the sensor array 230 may be a charge coupled device (CCD) image sensor.

The CCD sensor serves to receive an image like a film of a film camera, and converts an image of an input object into an electrical signal.

The optical sensor may be configured with other sensors such as a complementary metal-oxide semiconductor (CMOS) sensor.

The image photographed by the camera may be stored in a storage medium through the processing of the controller 240.

4A illustrates the optical sensor data, and the sensor array 230 may be captured in a form of light overlapping, scattered, and cloudy. The controller 240 may extract the captured image as shown in FIG. 4B based on the detected light for output and storage of the image.

5 and 6 illustrate an example of a processing process of the controller 240.

The controller 240 may first extract the captured image 510 including the plurality of images, and may segment the images based on the distance between the pinholes, the number of pinholes, and the size of the sensor region captured by one pinhole. The divided images 521, 522... May have low resolution as an image obtained by dividing the overlapping images one by one.

On the other hand, each of the images of the light passing through the pinhole by the subject (light source) is clear but overlapping, so it must be separated. Since the light passing through the pinhole is not refracted, the number of pinholes and the arrangement of the pinholes included in the pinhole array 210 can be easily distinguished according to the optical path.

Thereafter, the controller 240 may detect and correct an error of each of the divided images 521, 522. Alternatively, the images may be separated from the beginning again from the captured image 510 including the plurality of images. The detection and correction of the error may be performed in such a way as to shift at least a part of the image in a predetermined reference arrangement form based on the number and arrangement of pinholes, for example.

The controller 240 generates the reconstructed image 620 by synthesizing the divided images or the images in which the error is minimized 611, 612, and 613. The divided images have a lower resolution than the image actually captured, but the reconstructed image is equivalent to the resolution of the image captured.

Meanwhile, the reconstructed image 620 may have a higher resolution than the divided images or the images in which the error is minimized 611, 612, and 613, and may have the same resolution as that of the actually captured image.

7 is a block diagram of a multi-pinhole camera according to an embodiment of the present invention.

The multi-pinhole camera according to an embodiment of the present invention includes a pinhole array 210 including a plurality of pinholes, a sensor array 230 including a plurality of optical sensors for detecting light passing through the pinholes, and The controller 240 may generate a reconstructed image based on the detected light.

The optical sensor may be a Charged Coupled Device (CCD) image sensor or a CM0S image sensor capable of converting light into an electrical signal. The captured image can be converted into an electrical image signal by a CCD image sensor or the like.

Alternatively, the multi-pinhole camera according to an embodiment of the present invention may include a transducer 231 in the sensor array 230 or separately that may convert light into an electrical signal or other data.

Meanwhile, the pinhole array 210 may pass light from the same light source to the plurality of pinholes to generate interference and diffraction.

Meanwhile, the pinhole array 210 may be disposed at a position where the first bright interference fringe is generated due to the interference and diffraction. In this case, the size of the first bright interference fringe may be inversely proportional to the number of pinholes included in the pinhole array, and may be proportional to the spacing between the pinholes.

The controller 240 may divide the plurality of captured images overlapped based on the number of pinholes included in the pinhole array and the distance between the pinholes.

The controller 240 may divide the plurality of captured images based on the size of the photosensor region corresponding to one pinhole, and may divide the plurality of captured images for each optical path.

The controller 240 may generate the reconstructed image by synthesizing the divided plurality of captured images.

The controller 240 may include a divider 241 for extracting and dividing a captured image based on an optical path, and a synthesizer 242 for generating a high resolution reconstructed image by synthesizing the divided images.

The controller 240 may further include a segmentation corrector 244 that evaluates the divided images and detects an error, and an adjuster 245 that minimizes noise of the image or rearranges the images according to a predetermined placement criterion.

On the other hand, the control unit 240 may increase the resolution of the plurality of divided images, synthesize the image to generate the reconstructed image, and further increase the resolution or correct the high-resolution image image compensator 243 It may further include.

Meanwhile, the multi-pinhole camera may further include a display 260 for displaying images and allowing a user to preview the image, and further include storage 270 for storing images such as a reconstructed image. can do.

8 to 17 are views referred to for describing various examples of a multiple pinhole camera and an operation method thereof according to an embodiment of the present invention.

According to one embodiment of the invention, the path is taken into account in the image processing process according to the size of the light sensor.

FIG. 8 illustrates a case in which three optical sensors are allocated to one pinhole 211 among a plurality of pinholes included in the pinhole array 210.

As shown in FIG. 8A, light incident along the optical path may stimulate each of the corresponding optical sensors in the sensor array 230, and the light passing through the other path may be excluded from the process because the light passes through the sensor. can do.

That is, the imaging of the captured object may be inferred based on the position of the sensor that reaches the light passing through the pinhole.

In (a) of FIG. 8, the sensor is divided by each optical path (the same light direction) for easy description, but the optical sensors form one array column as shown in (b) of FIG. 8. The plurality of sensor arrays 230 may be provided in a matrix form of N * M.

FIG. 9 illustrates an object viewed through a pinhole, and a rectangle briefly illustrates a size of an image or sensor region corresponding to a pinhole and a circle to one pinhole.

10 to 16 exemplify a case in which an optical path is generated at a ratio in which phases are formed in four optical sensors relative to the size of one pinhole 211, but the present invention is not limited thereto.

In the example in which an optical path is generated at a ratio in which four images are formed in relation to the size of one pinhole 211, the images are left upper quadrant 1010, upper right quadrant 1030, lower left quadrant 1030, and lower right quadrant 1040. Can be separated by).

Referring to FIG. 11 and FIG. 12, when the object of FIG. 9 is photographed, an image captured by the optical sensor may be captured as the image of FIG. 11. The control unit 240 generates four images by separating the images for each quadrant from the image of FIG. 11.

12 (a) is a top left quadrant, FIG. 12 (b) is a top right quadrant, FIG. 12 (c) is a bottom left quadrant, and FIG. 12 (d) is divided images according to light paths corresponding to the bottom right quadrant. .

An area indicated by 1 to 15 in FIG. 12A corresponds to an area indicated by 1 to 15 in FIG. 11. As such, the images corresponding to the upper left quadrant may be generated by dividing the entire image into 2 * 2 matrices, and extracting and synthesizing the images corresponding to the upper left quadrant of each matrix. Also, images corresponding to other quadrants can be generated in the same manner.

Meanwhile, according to the present invention, as shown in FIG. 13, the separated captured images may be combined to generate a high resolution image. FIG. 13A illustrates an image corresponding to an upper left optical path, FIG. 13B illustrates an image corresponding to an upper right optical path, FIG. 13C illustrates an image corresponding to a lower left optical path, and FIG. 13. (D) is an image corresponding to the right-side light path.

The controller 240 increases the left and right resolutions of the four images, respectively. The increase in the left and right resolution may be implemented in a manner of copying, inserting an image of itself according to a cell or other criterion of the images, or in other known methods.

Thereafter, the controller 240 synthesizes the image corresponding to the upper left light path and the image corresponding to the upper right light path and corrects the color.

FIG. 15 doubles the image corresponding to the upper left light path of FIG. 13A and the image corresponding to the upper right light path of FIG. 13B from left to right, and then alternates two images for each line. Illustrate a composite image with alternate insertions.

For example, the color correction may determine many colors of the corresponding cell itself and neighboring cells as its own color.

15 illustrates an image before color correction. The controller 240 determines the colors of eight cells around each of the color corrected cells 1510, 1520, 1530, and 1540, and the cells corrected before the color corrected cells 1510, 1520, 1530, and 1540, and correspond to five or more colors. Alternatively, you can determine the color corrected with the most colors. FIG. 16 illustrates an image comprising cells 1610, 1620, 1630, 1640 after color corrected in this manner.

The controller 240 may generate an upper light path image by synthesizing an image corresponding to an upper left light path and an image corresponding to an upper right light path, and generate a lower light path image in the same manner.

Thereafter, the controller 240 may similarly process the above-described process of increasing the upper and lower resolutions, synthesizing, and color correcting the upper and lower optical path images, respectively, to generate and restore a high resolution image as shown in FIG. 14.

Meanwhile, in the present exemplary embodiment, an example in which the plurality of images is increased and synthesized in the horizontal direction and then the same process is performed in the vertical direction is described. have.

In normal cameras, it is necessary to focus when shooting, which prevents quick shooting. However, in the case of the multi-pinhole camera according to the present invention, since it is not necessary to focus, fast shooting is possible.

In addition, the singular pinhole camera has a problem of increasing the exposure time due to lack of light amount or deteriorating image quality, but the present invention has the effect of increasing the total amount of light amount by configuring multiple pinholes.

That is, the present invention does not need to focus and has an advantage of easy resolution reconstruction.

In addition, when reconstructing the captured image, the distortion due to the absence of the lens is not considered, thereby minimizing the burden of image processing.

In addition, since the lens is not used, the camera can be made smaller and lighter.

17 to 22 are views referred to compare and explain a multiple pinhole camera and a lens camera according to an embodiment of the present invention. Hereinafter, the present invention will be described in more detail by comparing a multi-pinhole camera and a lens camera.

Lens cameras are cameras that capture images using geometrical optical properties, in particular the straightness and focus of light. Lens cameras have the advantage of having a high resolution at the point of focus, but the problem of not having the right focus has a problem of blurring and a problem of increased cost, weight, and structure complexity due to the lens.

On the other hand, as shown in Figure 18 (a), a multi-lens array camera using a plurality of lenses, one light passes through two or more lenses, focusing on a separate position without the optical interference phenomenon. In other words, separate images are formed by the two lenses.

Therefore, there is no interference phenomenon, and the image captured by the multiple lenses is generated without mutual signal interference. Such a multi-lens array camera has a problem of increasing cost, weight, and structure complexity due to a plurality of lenses, and it is difficult to accurately determine an optical path due to a refractive phenomenon.

On the other hand, the pinhole camera is a camera using light interference and diffraction phenomenon.

Interference is a phenomenon in which two waves overlap each other and energy is not evenly distributed in a space, but at a maximum, at a maximum, and at a minimum, at a minimum. To cause interference, two or more waves must remain constant at the same speed, frequency, wavelength, and relative phase.

Diffraction is the phenomenon where light bends or spreads out of an edge.

17 and 18 (b) illustrate the first bright pattern 290 formed on the sensor surface of the pinhole array 210 and the sensor array 230 included in the pinhole camera.

The distance between the pinholes is very small compared to the distance b between the pinholes and the sensor surface 230 where the diffraction pattern is observed, and one light passes through two or more pinholes to generate an interference pattern.

When the difference between the optical path distance of the light passing through the pinhole and the optical path distance of the light passing through the other pinhole is an integer multiple of the wavelength, constructive interference occurs and a bright pattern is formed. On the other hand, destructive interference occurs when the optical path difference of two lights is an integer multiple of half wavelength.

The diffracted light causes multiple bright and dark patterns, and the pinhole camera captures the image produced by the first bright pattern of the diffracted light.

Pinhole cameras have low resolution because they do not collect light as a whole. However, the amount of light can be increased by using a plurality of pinholes.

Meanwhile, the multiple pinhole camera according to the present invention may include a plurality of pinholes, and the sensor array 230 may be disposed at a position where the first bright interference fringe occurs due to the interference and diffraction.

Meanwhile, the size c of the first bright interference fringe may be inversely proportional to the number of pinholes included in the pinhole array, and may be proportional to the spacing between the pinholes.

In addition, the size of the pinhole may be proportional to the product of the width c of the concatenated image 290 and the distance b of the image and the pinhole.

Therefore, the smaller the pinhole width and the larger the number of pinholes, the thinner the pinhole camera can be made thinner and the higher the resolution can be.

19 compares an image photographed with a multi-lens array and an image photographed with multiple pinholes.

When the image of the alphabet letter 'ABCD' is arranged in three stages as shown in (a) of FIG. 19 is photographed with a lens array, optical interference and interference fringes do not exist according to multiple lenses. You can shoot so that images do not overlap.

On the other hand, when the image of the alphabet letter 'ABCD' is arranged in three stages as shown in Fig. 19 (a) with multiple pinholes, the image is always superimposed by the interference fringe as shown in Fig. 19 (c). However, since the optical path can be easily determined, the image can be reconstructed through the image processing method described above.

FIG. 20 illustrates that light from a plurality of light sources A and B is photographed at position C with multiple lens arrays, and FIG. 21 illustrates that light from the plurality of light sources A and B is photographed at position C with multiple pinholes. 22 is a diagram comparing photographed images.

Referring to the drawings, the light of the point light source A and the point light source B can be captured at the same position (C) on the optical sensor through the multiple lens array. This is not due to interference because the light source is different because it is simply an image overlapping chip due to light overlap.

On the other hand, in the multiple pinhole array camera, the point light source A and the point light source B may be gathered at one point C on the image sensor. At this time, an interference fringe is formed at the point C by the interference of the point light sources A and B, and thus an image overlap phenomenon occurs. That is, the image overlap chip phenomenon may occur only when the interference fringe is formed by the optical interference effect.

When the image illustrated in (a) of FIG. 22 is photographed with a multi-lens array camera, only an image overlap occurs as shown in FIG. 22 (b). However, when the image is captured with a multi-pinhole camera, the image is overlapped as illustrated in FIG. And phenomena may occur.

Multi-pinhole camera according to an embodiment of the present invention is photographed by forming a pattern by a light diffraction and interference, a constant interference fringe pattern, the size of the first bright pattern is proportional to the distance between the plurality of pinholes provided, the number of pinholes It is determined in inverse proportion to.

By adjusting the spacing and number between the plurality of pinholes, high resolution can be achieved and the amount of light can be increased. In addition, since the lens is not used, focus-free imaging is possible, and the camera can be made smaller and lighter.

23 is a flowchart illustrating a method of operating a multiple pinhole camera according to an embodiment of the present invention.

According to an embodiment of the present invention, a method of operating a multiple pinhole camera includes detecting a light passing through a pinhole (S2310), dividing a plurality of images based on the detected light (S2320), and the divided plurality of Synthesizing the image may include a step (S2340).

In the dividing step (S2320), the plurality of images may be divided based on the size of the photosensor region corresponding to one pinhole, and the plurality of images may be divided for each optical path.

Meanwhile, the operation method of the multi-pinhole camera according to the embodiment of the present invention may further include converting the resolution of the image (S2330). In particular, the method may further include increasing the resolution of the divided plurality of images.

The increasing of the resolution may include increasing the resolution of the divided plurality of images in a horizontal direction, as described with reference to FIGS. 13 through 16, and synthesizing the images having the increased resolution arranged in the horizontal direction. And correcting the color of the synthesized image, increasing the resolution of the color corrected images in a vertical direction, synthesizing all the images, and correcting the color of the synthesized total image. can do.

Meanwhile, the method may further include a correcting step of rearranging the plurality of divided images according to a preset arrangement.

Meanwhile, the multiple pinhole camera and its operation method according to the present invention may be provided in a mobile phone such as a mobile phone, a smartphone, and an image display device such as a TV or a monitor.

The multi-pinhole camera and its operation method according to the present invention are not limited to the configuration and method of the embodiments described as described above, the embodiments are all or part of each embodiment so that various modifications can be made May be optionally combined.

On the other hand, the operating method of the multi-pinhole camera of the present invention can be implemented as a processor-readable code on a processor-readable recording medium provided in the multi-pinhole camera. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

Claims (16)

A pinhole array including a plurality of pinholes;
A sensor array including a plurality of optical sensors for sensing light passing through the pinholes; And
And a controller configured to generate a reconstructed image based on the detected light. The method of claim 1,
And the pinhole array passes light from the same light source into the plurality of pinholes to generate interference and diffraction. 3. The method of claim 2,
And the sensor array is positioned at a position where a first bright interference fringe occurs due to the interference and diffraction. The method of claim 3,
Wherein the size of the first bright interference fringe is inversely proportional to the number of pinholes included in the pinhole array, and is proportional to the spacing between the pinholes. The method of claim 1,
And the controller divides the plurality of captured images overlapped based on the number of pinholes included in the pinhole array and the interval between the pinholes. The method of claim 5,
The control unit may divide the plurality of captured images based on the size of the photosensor region corresponding to one pinhole. The method of claim 5,
The controller is configured to divide the plurality of captured images for each optical path. The method of claim 5,
The control unit may generate the reconstructed image by synthesizing the plurality of divided captured images. The method of claim 5,
The control unit may increase the resolution of the plurality of divided images and synthesize the generated multi-pinhole camera. The method of claim 1,
The optical sensor is a multiple pinhole camera, characterized in that the CCD image sensor. Sensing light passing through a plurality of pinholes;
Segmenting a plurality of images based on the sensed light; And
Synthesizing the plurality of divided images; and operating the multiple pinhole camera. 12. The method of claim 11,
The dividing step may include: dividing the plurality of images based on the size of an optical sensor region corresponding to one pinhole. 12. The method of claim 11,
In the dividing step, the plurality of images are divided by light paths. 12. The method of claim 11,
And increasing the resolution of the divided plurality of images. 15. The method of claim 14,
The resolution increase step,
Increasing the resolution of the divided plurality of images in a horizontal direction;
Synthesizing the images of which the resolution is increased are arranged in a horizontal direction;
Correcting a color of the synthesized image;
Increasing the resolution of the color corrected images in a vertical direction;
Synthesizing the entire images;
Correcting the color of the synthesized overall image; a method of operating a multiple pinhole camera, comprising: a. 12. The method of claim 11,
And rearranging the plurality of divided images according to a preset arrangement.

Images