CN101098399A - Device and method for restoring high-resolution images - Google Patents

Abstract

A device and method for restoring high-resolution images are provided. The device includes: a camera module including a plurality of lenses and a plurality of sub-image sensors corresponding to the plurality of lenses, each of the plurality of sub-image sensors including a color filter having a single color; an original image generation module receiving a plurality of an original color separation image; an intermediate image generation module rearranging pixel information of pixels at the same position of a plurality of original color separation images provided by the original image generation module, and generating an intermediate having a resolution higher than that of each original color separation image an image; and a final image generating module that performs demosaicing on the intermediate image, performs sharpening of the demosaiced intermediate image, and generates a final image.

Description

Device and method for restoring high-resolution images

This application claims Korean Patent Application No. 10-2006-0057660 filed with the Korean Intellectual Property Office on June 26, 2006 and Korean Patent Application No. 10-2006-0105348 filed with the Korean Intellectual Property Office on October 27, 2006 priority of a patent application, which is hereby incorporated by reference in its entirety.

technical field

The present invention relates to an apparatus and method for restoring high-resolution images, and more particularly, to a method for recovering high-resolution images from small cameras installed in small digital devices such as mobile phones, personal digital assistants (PDAs), MP3 players, etc. A device and method for restoring a high-resolution image from an image obtained by a module.

Background technique

Like most digital devices, digital cameras have opened up new worlds for people. As an alternative to traditional film-based cameras, digital cameras have the advantage of being easy to operate, and compared to those of professional photographers, ordinary users can take pictures and view them immediately after capture without developing and printing. photo. In addition, since digital cameras record captured images as digital files in the camera's memory, it is possible to upload photographs taken with high picture quality to personal computers (PCs) as permanent or semi-permanent digital image files, for example, when necessary Anytime for storage, image processing and/or printing.

In addition, digital cameras have been miniaturized and become more portable, enabling them to be embedded in small digital devices such as mobile phones, personal digital assistants (PDAs), MP3 players, and the like. As a result, taking pictures and enjoying photos has become a part of daily affairs, and whether a digital camera can be embedded in a small digital device has become very important when purchasing a digital device.

Recently, digital devices have become more miniaturized, and in addition to requiring personalized devices and convenience, consumers have begun to demand smaller and lighter digital device products.

Therefore, it is obvious that in order to make a compact digital device with an embedded digital camera smaller and lighter, the embedded digital camera itself should be made smaller and lighter.

FIG. 1 schematically shows the operating principle of a digital camera incorporated into a conventional compact camera module.

Referring to FIG. 1 , images of a predetermined object 101 captured by a user through a lens 101 a of diameter D _a and a lens 101 b of diameter D _b are formed as an image A and an image B on an image sensor 102 a and an image sensor 102 b, respectively.

Although a lens 101b having a relatively large diameter _Db is advantageous from a resolution point of view, a problem may arise that a camera module incorporated into a small digital device becomes bulky due to a relatively long focal length _fb , and in This is an undesirable feature of installing a camera module into a small digital device. That is, the large lens size and long focal length make it difficult to realize a compact and lightweight digital camera.

In contrast, a lens 101 _a with a relatively small diameter Da has a short focal length _fa to form the image A of the object 101 . Therefore, although the camera module can be miniaturized, an image with high resolution, which is one of the most important features in a digital camera, cannot be obtained. Such camera modules cannot fully satisfy consumers' demand for high-resolution pictures.

In order to solve such a problem, many inventions have been proposed, for example, Korean Patent Laid-open Application No. 2003-0084343, but the problem has not been solved yet.

Contents of the invention

Accordingly, an aspect of the present invention is to provide an apparatus and method for restoring a high-pixel image capable of realizing a small digital device by reducing the size of a digital camera installed in the small digital device.

Another aspect of the present invention is to provide an apparatus and method for restoring a high-pixel image capable of reducing the size of a digital camera installed in a small digital device.

Another aspect of the present invention is to provide an apparatus and method for restoring a high-pixel image capable of easily correcting optical deviation and inconsistency in sensitivity in expressing the high-pixel image.

Another aspect of the present invention is to provide an apparatus and method for restoring a high-pixel image, which can simultaneously achieve high-sensitivity image perception by using color filters with different transmittances without changing the structure of the image sensor. measurement and low-sensitivity image sensing.

Another aspect of the present invention is to provide an apparatus and method for restoring a high-pixel image in a digital camera module having a built-in digital camera, wherein more options are available when designing a compact digital device mounted with a digital camera.

Additional aspects and/or advantages of the present invention will be set forth in part in the following description, and some will be clear from the description, or can be learned through practice of the present invention.

Therefore, an aspect of the present invention is to provide an apparatus for restoring a high-pixel image, the apparatus including: a camera module including a plurality of lenses and a plurality of sub-image sensors corresponding to the plurality of lenses, each of the plurality of sub-image sensors Both include a color filter with a single color; the original image generation module receives a plurality of original color separation images; the intermediate image generation module rearranges the pixels on the same position of the multiple original color separation images provided by the original image generation module information, and generate an intermediate image having a resolution higher than that of each original color separation image; and a final image generating module that performs demosaicing on the intermediate image, performs sharpening of the demosaiced intermediate image, and generates a final image.

According to another aspect of the present invention, there is provided a device for restoring a high-pixel image, the device comprising: a camera module including a plurality of lenses and a plurality of sub-image sensors corresponding to the plurality of lenses, each of the plurality of sub-image sensors Both include a color filter, which is divided into a plurality of color regions with different colors; an original image generation module, which divides a plurality of original color separation images into a plurality of pixel groups; an intermediate image generation module, which will be divided into pixel information of a pixel at the same position in a plurality of original color separation images of a plurality of pixel groups is mapped to each pixel of a pixel group corresponding to the same position, and a pixel having a resolution higher than that of each original color separation image is generated an intermediate image; and a final image generating module that restores the intermediate image using a predetermined interpolation algorithm, performs sharpening of the received intermediate image, and generates a final image.

According to another aspect of the present invention, there is provided a device for restoring high-pixel images, the device comprising: a camera module including a plurality of color lenses and a plurality of sub-image sensors corresponding to the plurality of color lenses, wherein the plurality of Each of the color lenses has a single color, and obtains a plurality of original color separation images through the plurality of sub-image sensors; the original image generation module receives a plurality of original color separation images; the intermediate image generation module rearranges the original image generation modules The pixel information of the pixels on the same position of the plurality of original color separation images provided, and generating an intermediate image having a resolution higher than that of each original color separation image; and a final image generation module, performing demosaicing on the intermediate image, performing The demosaiced intermediate image is sharpened and a final image is produced.

According to another aspect of the present invention, there is provided a device for restoring high-pixel images, the device comprising: a camera module including a plurality of color lenses and a plurality of sub-image sensors corresponding to the plurality of color lenses, wherein the plurality of Each of the color lenses has a single color, and obtains a plurality of original color separation images through the plurality of sub-image sensors; the original image generation module divides the plurality of original color separation images into a plurality of pixel groups; the intermediate image generation module, Mapping pixel information of pixels at the same position in a plurality of original color separation images divided into a plurality of pixel groups to each pixel of the pixel group corresponding to the same position, and generating an intermediate image; and a final image generation module, The intermediate image is restored using a predetermined interpolation algorithm, sharpening the restored intermediate image is performed, and a final image is generated.

According to another aspect of the present invention, there is provided a method of restoring a high-pixel image in a camera module, the camera module including a plurality of lenses and a plurality of sub-image sensors corresponding to the plurality of lenses, each of the plurality of lenses Including having a color filter of monochrome, the method includes: obtaining a plurality of raw images through each of a plurality of sub-image sensors; receiving the plurality of raw images, and generating a plurality of raw color-separated images; rearranging the raw images from the camera module The pixel information of the pixels on the same position of a plurality of original color separation images provided by the image generation module, and generate an intermediate image having a resolution higher than that of each original color separation image; The intermediate images of the mosaic are sharpened, and a final image is produced from the sharpened images.

According to another aspect of the present invention, there is provided a method of restoring a high-pixel image in a camera module, the camera module including a plurality of lenses and a plurality of sub-image sensors corresponding to the plurality of lenses, each of the plurality of lenses Including a color filter having a single color, the method includes: obtaining a plurality of original color separation images through each of a plurality of sub-image sensors; dividing the obtained plurality of original color separation images into a plurality of pixel groups; The pixel information of the pixels at the same position in the plurality of original color separation images of the plurality of pixel groups is mapped to each pixel of the pixel group corresponding to the same position, and an intermediate image is generated; and using a predetermined interpolation algorithm to restore the intermediate image, performing The interpolated intermediate images are sharpened, and a final image is produced from the sharpened images.

According to another aspect of the present invention, there is provided a method for restoring a high-pixel image in a camera module, the camera module including a plurality of color lenses and a plurality of sub-image sensors corresponding to the plurality of color lenses, the plurality of color lenses each including a color filter having a single color, the method comprising: obtaining a plurality of raw color separation images by each of the plurality of sub-image sensors; receiving the plurality of raw color separation images; rearranging the raw image generated from the camera module The module provides pixel information of pixels at the same position in multiple original color separation images, and generates an intermediate image having a resolution higher than that of each original color separation image; and performs demosaicing on the intermediate image, performing demosaicing The intermediate image is sharpened, and a final image is generated from the sharpened image.

According to another aspect of the present invention, there is provided a method for restoring a high-pixel image in a camera module, the camera module including a plurality of color lenses and a plurality of sub-image sensors corresponding to the plurality of color lenses, the plurality of color lenses Each has a single color, and the method includes: obtaining a plurality of original color separation images through each of the sub-image sensors; dividing the plurality of original color separation images into a plurality of pixel groups; The pixel information of the pixels at the same position in the plurality of original color separation images is mapped to each pixel of the pixel group corresponding to the same position, and an intermediate image is generated; and the intermediate image is restored using a predetermined interpolation algorithm, and the interpolated intermediate image is executed. sharpen and produce the final image.

Description of drawings

The above-mentioned and other characteristics and advantages of the present invention will become clearer by following detailed description of preferred embodiments of the present invention in conjunction with the accompanying drawings, wherein:

FIG. 1 is a diagram illustrating the operating principle of a digital camera installed in a conventional small digital device;

FIG. 2A is a diagram showing a basic structure of a conventional digital camera;

2B is a cross-sectional view of a unit pixel forming the image sensor unit shown in FIG. 2A;

Fig. 3 is a block diagram of a device (300) for restoring a high pixel image according to an embodiment of the present invention;

4A and 4B are diagrams illustrating the structure of a digital camera module according to an embodiment of the present invention;

4C is a cross-sectional view of a unit pixel forming an image sensor unit shown in FIG. 4A;

5A and 5B are diagrams illustrating a method of painting a color filter according to an embodiment of the present invention;

6 is a diagram illustrating a process of correcting a positional deviation and an inconsistency of sensitivity of an original image by an original image generation module according to an embodiment of the present invention;

7A and 7B are diagrams illustrating a process of generating an intermediate image according to an embodiment of the present invention;

8 is a diagram illustrating a process of restoring a high-pixel image by an image generation module according to an embodiment of the present invention;

FIG. 9 is a diagram illustrating a structure of a digital camera module according to an embodiment of the present invention;

10 is a flowchart showing a method for restoring a high-pixel image using the first image generation module shown in FIG. 7B with the camera module having the structure shown in FIG. 4A; and

FIG. 11 is a flowchart illustrating a method of restoring a high-pixel image using the second image generation module illustrated in FIG. 8 with the camera module having the structure illustrated in FIG. 4A .

Detailed ways

Reference will now be made in detail to embodiments of the invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like parts throughout. The embodiments are described below in order to explain the present invention by referring to the figures.

Generally, the larger the number of unit pixels, the clearer and more orderly the obtained image, so the performance of a digital camera can be determined by the number of unit pixels.

In addition to the number of unit pixels, the light intensity of the camera lens also affects the performance of the digital camera. The light intensity of a camera lens is known as the F-number or aperture value. The F-number, which is an indication of the amount of light reaching the image sensor of the digital camera per unit area, is obtained by dividing the focal length f of the lens by the diameter D (ie, f/D). The larger the F-number, the smaller the amount of light reaching the image sensor of the digital camera per unit area. The smaller the F-number is, the larger the amount of light reaches per unit area of the image sensor of the digital camera, thereby obtaining a bright image with high resolution. As described above, the F-number is closely related to the resolution of the obtained image and the amount of light per unit area reaching the image sensor of the digital camera.

Assume that two digital cameras with different lens sizes, focal lengths, and unit pixels have the same F-number. Since the amount of light per unit area reaching the respective image sensors of the two digital cameras is the same, images obtained from the respective digital cameras will have the same light intensity.

Based on the above principles, embodiments of the present invention propose a method for restoring high-pixel images that can be applied to a digital camera module that has multiple lenses capable of displaying high-pixel images while reducing lens diameter and focal length .

FIG. 2A is a diagram showing a basic structure of a conventional digital camera. Referring to FIG. 2, a conventional digital camera mainly includes: a lens 201 having a diameter _D2 that concentrates light reflected from a predetermined object; and an image sensor 202 that generates electrical image signals corresponding to pixel levels in response to the light concentrated by the lens 201. The focal length of lens 201 is also shown.

A color filter array according to a Bayer pattern is included in the image sensor 202 to realize light received at the lens 201 as an original full-color image. FIG. 2A shows a top view of image sensor 202 .

The Bayer pattern originates from the principle that digital images are inevitably realized by points even if the images actually existing in nature are not composed of points.

To form a digital image from dots, the luminance (or brightness) and chrominance (or color) components are collected and distributed on a two-dimensional panel to receive red (R), green (G) and blue (B) colors points for each brightness of the composition.

In the Bayer pattern array, the most sensitive G color component of the human eye is 50%, and each of the R color and B color components is 25%, thus forming a two-dimensional matrix, which is called a Bayer pattern color filter.

The Bayer pattern color filter recognizes only the color components allocated thereto from the respective R, G, and B color components forming the matrix, does not recognize the full color components, and inserts the unrecognized color components to reproduce the full color components.

2B is a cross-sectional view of a unit pixel used to form the image sensor of FIG. 2A

Referring to FIG. 2B showing portions forming unit pixels 202 a to 202 d of the image sensor ( 202 of FIG. 2A ), the Bayer pattern color filter 203 is included in the image sensor 202 . Other features of a conventional device are shown in Figure 2B, but will not be discussed here, as such discussion is not necessary to understand the present application.

FIG. 3 is a block diagram of an apparatus 300 for restoring a high-pixel image according to an embodiment of the present invention.

The device 300 includes: a camera module 301, which concentrates incident light and generates a plurality of color-separated images; an image generation module 302, which generates a final image based on the color-separated images provided by the camera module 301; and a display module 303, which displays the generated image Module 302 provides the final image.

A structure of a digital camera module having a plurality of lenses capable of applying the method of restoring a high-pixel image according to an embodiment of the present invention will be described with reference to FIGS. 4A and 4B .

4A to 4C are diagrams illustrating the structure of a digital camera module having a plurality of lenses capable of applying a method of restoring a high-pixel image according to an embodiment of the present invention.

The camera module 301 includes a plurality of lenses 401 and a plurality of sub-image sensors 402 corresponding to the plurality of lenses 401 . Here, a plurality of sub-image sensors 402 have a color filter having an area painted with a single color, and an original image of each color is obtained from each sub-image sensor 402 .

The digital camera module includes: a plurality of lenses 401 that concentrate incident light reflected from a predetermined object, the plurality of lenses 401 having the same diameter; and a plurality of sub-image sensors 402a to 402d that generate electrical image signals in response to light reflected from the object . That is, a color filter similar to that of the related art, which has a space divided into a plurality of color regions, is included in the plurality of sub-image sensors 402a to 402d so as to concentrate the color passing through the lens 401. Light is realized as native full color.

4B is a side view of the digital camera module shown in FIG. 4A, where it is assumed that the focal length _f3 ranging from the plurality of lenses 401 (401a-401d) having the same diameter _D3 to the plurality of object image forming sensors 402a-402d is the same .

Therefore, a plurality of lenses 401 (401a-401d) having the same diameter _D3 are arranged on the same plane.

Regarding the lens arrangement mode, although the illustrated embodiment has shown that a plurality of lenses 401 ( 401 a - 401 d ) are symmetrically arranged in the up-down and left-right directions, the present invention is not limited thereto. A plurality of lenses 401 (401a-401d) may be linearly arranged in a horizontal direction or a vertical direction. Where an odd number of lenses is available, lenses may be arranged radially around a central lens, and various other lens arrangements may be employed.

The plurality of lenses 401 (401a-401d) may concentrate light reflected from a predetermined object at a position specified by the lens or at a position of a lens other than the predetermined lens among the plurality of lenses shifted by a predetermined number of pixels based on the position of the predetermined lens. .

Hereinafter, for convenience of explanation, an embodiment in which four lenses are arranged in a 2×2 (width×height) matrix will be explained by way of example.

Based on the above-mentioned principle of assuming that two digital cameras with different lens sizes, focal lengths, and unit pixels have the same F-number, since the amount of light per unit area reaching the respective image sensors of the two digital cameras is the same, it is obtained from the respective digital cameras The images will have the same light intensity. In this embodiment, a plurality of lenses 401 (401a-401d) each smaller than the lens 201 shown in FIG. 2 are used, and each has a number of unit pixels smaller than the image sensor unit 202 shown in FIG. Fewer multiple sub-image sensors 402a to 402d. In addition, the focal length _f3 of FIG. 4B is shorter than the focal length f2 of FIG. ₂ .

Assume that the image sensor 202 shown in FIG. 2 forms four million pixels, and each of the plurality of sub-image sensors 402 a to 402 d shown in FIG. 4A forms one million pixels, respectively.

Accordingly, the plurality of sub-image sensors 402a to 402d shown in FIG. 4A correspond to four image sensors each having one million pixels. Four lenses 401a-401d, each having the same diameter _D3 , produce images on sub-image sensors 402a-402d corresponding to lenses 401a-401d, respectively.

Each of the color filters in the sub-image sensors 402a to 402d is quartered so that four regions fit the size of the sub-image sensors 402a to 402d, and then each region is painted with a single color.

FIG. 4C is a cross-sectional view of unit pixels 402d-1 to 402d-4 used to form the sub-image sensors 402a to 402d of FIG. 4A. Since the unit pixels 402d-1 to 402d-4 exist in the same sub-image sensor 402d, the same color filter 403 is included in the area of the unit pixels 402d-1 to 402d-4.

Here, the color filter 403 is divided into a first filter area and a second filter area according to the transmittance of the color painted thereon. High-sensitivity sensing and low-sensitivity sensing are realized simultaneously by changing the transmittance of the color painted on the first filter area and the second filter area so that the amount of light passing through the first filter area and the second filter area is different. Among various color areas, a color area having the highest transmittance is grouped into a first filter area, and other color areas are grouped into a second filter area.

Hereinafter, for convenience of explanation, an embodiment in which a plurality of color regions and sub-image sensors are denoted by the same reference numerals will be explained.

For example, the respective color regions (that is, green (G) color region, red (R) color region, blue (B) color region and gray (Gr) color regions) are denoted by the same reference numerals as those of the sub-image sensors, ie, 402a, 402b, 402c, and 402d, respectively. The gray (Gr) color area 402d having the highest transmittance is grouped into the second filtering area, and the other color areas (ie, the green (G) color area 402a, the red (R) color area 402b and the blue (B) color area 402c ) are grouped as the first filter area.

In another embodiment, a color filter of a color other than gray may be formed in the second filter region. For example, a color filter of any one of white (W), yellow (Y), cyan, and magenta may be formed. However, the color of the color filter formed in the second filtering area is not limited to these examples, and a color filter of any color having a transmittance higher than that of the color filter formed in the filtering area of the first filter All can be considered to be included in the scope of the present invention.

For the transmittance of the color filter shown in Fig. 4A, among blue (B) color filter, green (G) color filter, red (R) color filter and gray (Gr) color filter, gray (Gr ) color filter has the highest transmittance.

As such, if the transmittance of the color corresponding to the second filter area 402d is higher than the transmittance of the color corresponding to the first filter areas 402a to 402c, the amount of light passing through the first filter area and the second filter area is different.

This means that the amount of light reaching the respective sub-image sensors 402a to 402d matching the respective color regions differs, and a high-sensitivity sensing function and a low-sensitivity sensing function can be simultaneously realized in the respective sub-image sensors 402a to 402d.

The color filter described above may be coated by a photo-lithography method or an inkjet method which will be shown with reference to FIGS. 5A and 5B , respectively.

The original white sensor ( 501 ) without color is painted in green ( 502 ) with monochromatic blue (B), green (G), red (R) and gray (Gr) painted according to the photoprinting method. Then, except that one quarter (1/4) of the image sensor area painted green belongs to the green mode, the image sensor area painted green has three quarters (3/4) of the image sensor area (503) painted green. /4) is removed.

The removed image sensor is painted red (504). Then, two-quarters (2/4) of the image sensor area painted red is removed.

The resulting sub-image sensor corresponds to an image sensor having image sensor areas painted one-quarter green and one-quarter red, respectively, covering two quarters (2/4) of the green area of the sub-image sensor It is painted blue and gray (506-508).

Current photoprinting methods are easy to perform compared to Bayer pattern color filtering processes.

Figure 5B shows the inkjet method.

The operation starts with the white sensor 509 . According to the inkjet method, first, a partition wall forming operation (510) is performed in which the same number of partition walls (for example, 4) as the lenses in the illustrated embodiment are formed in the image sensor, and the generated 4 partition walls are The regions are painted with desired colorinks, ie, green ink, red ink, blue ink, and gray ink, respectively (511-514).

The ink-jet method is a relatively simple process and can be advantageous in saving ink usage, which ultimately reduces the manufacturing cost of the sensor.

The color filters applied in the above manner are respectively matched to the plurality of lenses and the plurality of sub-image sensors.

For example, referring to FIG. 4A , in the case where a green (G) color filter, a red (R) color filter, a blue (B) color filter, and a gray (Gr) color filter are included in the sub-image sensors 402a to 402d, The incident light reflected from the subject through the first lens 401a among the four lenses 401 (401a-401d) forms a green (G) ) color image. Incident light reflected from a subject through the second lens 401b of the four lenses 401 (401a-401d) forms a red (R) color filter through a red (R) color filter included in the sub-image sensor 402b matched to the second lens 401b. )image.

Similarly, the blue (B) color filter included in the sub-image sensor 402c matched with the third lens 401c and the gray (Gr) color filter included in the sub-image sensor 402d matched with the fourth lens 401d are respectively passed. , the incident light reflected from the object through the third lens 401c and the fourth lens 401d among the four lenses 401 (401a-401d) form a blue (B) color image and a gray (Gr) color image.

In other words, incident rays reflected from the subject through the four lenses 401a-401d form images with colors of corresponding color filters included in the respective sub-image sensors 402a-402d, ie four images of the same size but different colors.

Meanwhile, the image generating module 302 receives a plurality of color-separated images from the camera module 301 and generates a final image based on the color-separated images provided from the camera module 301 .

To this end, the image generation module 302 includes: an original image generation module 302a, an intermediate image generation module 302b and a final image generation module 302c.

In the image generation module 302 according to the current embodiment of the present invention, the original image generation module 302 a receives an input of a plurality of original color separation images provided from the camera module 301 .

That is to say, as shown in FIG. 4A, the original image generation module 302a receives the green image obtained by the sub-image sensor 402a including the green filter, the red image obtained by the sub-image sensor 402b including the red filter, and the image obtained by the sub-image sensor 402b including the blue filter. The blue image obtained by sub-image sensor 402c and the gray image obtained by sub-image sensor 402d including a gray filter.

Here, the green image, red image, and blue image provide color information required to generate a final image by the final image generation module 302c, which will be described later. Instead, gray images provide the luminance information needed to produce the final image.

In an optional embodiment of the present invention, when the obtained original images are not arranged in the specified positions of the corresponding sub-image sensors 402a to 402d, the original image generating module 302a corrects the positions of the original images. In addition, when the obtained raw images have inconsistent sensitivities, the raw image generation module 302a corrects the sensitivity levels of raw images other than the raw image with the lowest sensitivity level based on the sensitivity of the raw image with the lowest sensitivity level.

FIG. 6 is a diagram illustrating a process of correcting a positional deviation and an inconsistency of sensitivity of an original image by the original image generating module ( 302 a ) according to an embodiment of the present invention.

Light rays incident on a predetermined object through the four lenses 401a to 401d arranged at fixed positions are sent through color filters included in the respective sub-image sensors 402a to 402d matched with the lenses 401a to 401d to form a corresponding color image. Then, the original image generation module 302a checks the positions of the respective images formed in the sub image sensors 402a to 402d.

Referring to FIG. 6 , respective images formed at positions in the sub-image sensors 402 a to 402 d are denoted by reference numerals 601 to 604 .

Reference numeral 601 denotes an image formed on the sub-image sensor 402a. In the embodiment of the present invention, unless the positions are fixed, the four lenses 401a to 401d are shifted by a predetermined number of pixels, and the image at the center of the sub-image sensor 402a can be considered to be at the standard position, that is, as indicated by the dotted rectangle 605 display, specify the location of the image.

Reference numeral 602 denotes an image formed on the sub-image sensor 402b shifted left by one pixel from a designated position. Reference numeral 603 denotes an image formed on the sub-image sensor 402c shifted down by one pixel from a specified position. Reference numeral 604 denotes an image formed on the sub-image sensor 402d that is diagonally shifted by one pixel from the designated position, that is, shifted by one pixel in both right and downward directions from the designated image.

As shown in FIG. 6, the images of the respective sub-image sensors 402a to 402d may deviate from their assigned positions, presumably due to optical misalignment. Such optical deviation usually causes the images of the respective sub-image sensors 402a to 402d to deviate from their assigned positions, making it difficult to obtain neat and clear images.

According to an embodiment of the present invention, the original image generation module 302a can correct the deviating image position caused by the optical deviation by means of software.

In detail, the original image generation module 302a corrects the deviated image position due to optical deviation in the following manner. For example, for image 602, once the image has shifted one pixel to the left from the specified location, shift the image to the right by one pixel. In image 603, the image is shifted up by one pixel. In image 602, the image is shifted left by one pixel and up by one pixel. Alternatively, in image 604, the image may be shifted up one pixel, then left one pixel.

In addition, the original image generation module 302a corrects the inconsistency of sensitivity, which will be described with reference to FIG. 6 .

Assuming that the sensitivity levels of the images in 601, 602, 603, and 604 are 10, 9, 8, and 7 respectively, the original image generation module 302a is based on the sensitivity of the image 604 with a sensitivity level of 7, by adjusting Sensitivity levels of images other than image 604 of level (ie, 7) are used to correct for inconsistent sensitivity levels.

Through the above processing, the image of each color is supplied to the intermediate image generation module 302b.

The intermediate image generation module 302b rearranges pixel information of pixels at the same positions of the respective original images provided by the original image generation module 302a, thereby being able to generate an intermediate image having a higher resolution than that of the original image for each color.

Here, the term "intermediate image" used to distinguish each image formed on the sub-image sensor means an image having a higher resolution than that of the original image obtained by rearranging pieces of pixel information of pixels at the same position in each original image. image. However, the term "intermediate image" is not necessarily a final image with a relatively high resolution. The final image may be produced by performing demosaicing or sharpening on the intermediate image. Meanwhile, the generated intermediate image may have the same resolution as that of the image sensor 402 in which the plurality of sub-image sensors 402a to 402d are arranged.

For example, if each of the plurality of sub-image sensors 402a to 402d has a resolution of 4×4, the intermediate image may have a resolution of 8×8, which is the same as the resolution of the image sensor 402 . The intermediate image is then provided to the final image generation module 302c. The final image generating module 302c generates a final image by performing demosaicing on the intermediate image provided by the intermediate image generating module 302b and sharpening the demosaiced image.

The display module 303 displays the final image provided by the final image generation module 302c.

The display module 303 can be implemented in the form of, for example, a flat panel display or a touch screen, but is not limited to these forms.

7A and 7B are diagrams illustrating a process of generating an intermediate image according to an embodiment of the present invention.

Before explaining the method of restoring a high-pixel image, it is assumed that in the display device 300 shown in FIG. The position is offset by a predetermined number of pixels.

Here, shifting the lens by a predetermined number of pixels may include shifting the lens based on a moving distance and a moving direction.

As shown in FIG. 7A, when using the camera module 301 to photograph a predetermined object 701 including four rectangles of the same size, it is assumed that the lens 401a is arranged at a fixed position to capture a rectangle A (701a), and based on the position of the lens 401a, the remaining lenses are 401b to 401d are shifted by a predetermined number of pixels to capture rectangles B, C, and D (701b, 701c, and 701d).

In shifting to the predetermined position, based on the position of the lens 401a of the photographing rectangle A (701a), the lens of the photographing rectangle B (701b) is shifted to the right to the predetermined position. Based on the position of the shot 401a of the shot rectangle A (701a), the shot of the shot rectangle C (701c) is moved down to a predetermined position.

In addition, based on the position of the shot 401a of the shot rectangle A (701a), the shot of the shot rectangle D (701d) is shifted diagonally (rightward and downward method) to a predetermined position.

Rectangles A ( 701 a ), B ( 701 b ), C ( 701 c ), and D ( 701 d ) respectively correspond to images formed on the respective sub-image sensors 402 a to 402 d through the lenses 401 a to 401 d.

Assuming that a plurality of sub-image sensors 402a to 402d matched with corresponding lenses form one million pixels respectively, the captured images of rectangles A (701a), B (701b), C (701c) and D (701d) are arranged according to the shape of the object, to form a single image indicated by the symbol "+" in Figure 7A. In this way, an image 703 having the same resolution as that of the entire image 704 of the subject captured with a camera capable of four-megapixel resolution imaging can be formed.

FIG. 7B illustrates a method of generating an intermediate image according to an embodiment of the present invention.

Based on the same principle as described above with reference to FIG. 7A, among the four lenses 401a to 401d of the camera module 301 that photographs a predetermined object 705, one lens 401a is arranged at a fixed position, and the other lenses 401b to 401d are arranged one by one from the position of the lens 401a. pixel offset.

In other words, based on the position-fixed lens 401a, the lenses 401b to 401d are shifted one pixel to the right, one pixel down and one pixel diagonally (i.e., one pixel in both the right and down directions ).

Using the above-mentioned camera module 301, the intermediate image generation module 302b (see FIG. 3 ) of the image generation module 302 rearranges the pixel information of the pixels at the same position of each original image 706 provided by the original image generation module 302a, thereby being able to generate The high-resolution intermediate image 707 of the original image for each color.

The final image generation module 302c performs demosaicing on the intermediate image 707 generated by the intermediate image generation module 302b, and sharpens the demosaiced intermediate image, thereby restoring the original image to a final image with a higher resolution.

For simplicity, the image generating module 302 described with reference to FIG. 7A will now be referred to as a first image generating module.

FIG. 8 is a diagram illustrating a process of restoring a high-pixel image by an image generation module according to another embodiment of the present invention. The camera module used in the present embodiment is the same as described above, and thus its explanation will not be repeated.

At the same time, the original image generation module 302a of the image generation module 302 divides the original color-separated image provided by the camera module 301 into a plurality of pixel groups. The intermediate image generation module 302b generates a first intermediate image 802 from the original image, which has the same resolution as the sub-image sensors 402a to 402d shown in FIG. 8 .

Here, the first intermediate image 802 may be divided into a plurality of pixel groups 803, 804, and 805, each pixel group being formed by 2×2 virtual pixels (width×height).

In each of the plurality of pixel groups 803, 804, and 805, the pixels may be divided into main pixels 803a, 804a, and 805a to which color and brightness information is mapped, and pixels located near and not adjacent to the main pixels 803a, 804a, and 805a. Sub-pixels 803b, 804b and 805b with information.

The positions of the main pixels 803 a , 804 a and 805 a may be set at a plurality of positions of each pixel group 803 , 804 and 805 .

For example, in each pixel group 803, 804, and 805 formed of 2×2 pixels as shown in FIG. The position of the main pixel. As another example, positions corresponding to the first row and the second column in each pixel group (ie, 803b, 804b, and 805b) may be determined as the position of the main pixel.

If the first intermediate image 802 is generated in the above manner, the intermediate image generation module 302b maps the pixel information of the pixels at the same position in each original color separation image to the main pixels of the pixel group corresponding to the same position.

For example, the intermediate image generating module 302b maps the pixel information of the pixels on the first row and the first column of each original color separation image to the main pixel of the pixel group 803 located in the first row and the first column of the first intermediate image 802 on 803a.

Similarly, the intermediate image generation module 302b maps the pixel information of the pixels on the first row and the first column of each color separation image to the main pixel 804a of the pixel group 804 located in the first row and the second column of the first intermediate image 802 superior.

In addition, the intermediate image generation module 302b obtains luminance information based on the color information in the pixel information of the pixels at the same position in each original color separation image, and maps the obtained luminance information to the main pixels 803a to 803a to the respective pixel groups 803 to 805 805a on.

In addition, the intermediate image generation module 302b maps the obtained brightness information to the main pixels of the pixel group located in the first row and the first column of the first intermediate image 802 .

8, it can be seen that the green (G) color information, red (R) color information and blue (B) color information provided by the sub-image sensors 402a to 402c and the gray (Gr) color information provided by the sub-image sensor 402d are Mapped to the main pixel of each pixel group. Although not shown in FIG. 8 , luminance information (Y) detected from three pieces of color information can also be mapped onto the main pixels 803 a to 805 a of the respective pixel groups 803 to 805 .

For example, it can be seen that: the green (G) color information of the pixels located in the first row and the first column of the green image, the red (R) color information of the pixels located in the first row and the first column of the red image, the red (R) color information of the pixels located in the blue image The blue (B) color information of the pixels in the first row and the first column of the color image, the luminance information of the pixels in the first row and the first column of the gray image, and the luminance information detected based on the three pieces of color information are mapped to the first On the main pixel 803a of the pixel group 803 .

Similarly, it can be seen that: the green (G) color information of the pixels located in the first row and the second column of the green image, the red (R) color information of the pixels located in the first row and the second column of the red image, and the red (R) color information of the pixels located in the blue image The blue (B) color information of the pixels in the first row and second column of the color image, the brightness information of the pixels located in the first row and second column of the gray image, and the brightness information detected based on the three pieces of color information that have been provided are mapped onto the main pixel 804a of the second pixel group 804 .

Therefore, the intermediate image generation module 302b generates a second intermediate image 806 in which three pieces of color information and two pieces of brightness information are mapped onto main pixels 803a , 804a and 805a of each pixel group 803 , 804 and 805 .

Thereafter, the intermediate image generation module 302b performs interpolation on the second intermediate image 806 using an interpolation method.

That is, the intermediate image generation module 302b obtains pixel information recorded in each sub-pixel 803b, 804b, and 805b based on information of the main pixels 803a, 804a, and 805a shown in FIG. 8 .

The intermediate image generation module 302b can interpolate the second intermediate image 806 according to various algorithms.

For example, pixel information recorded in each sub-pixel of the second intermediate image 806 may be calculated from information grasped from main pixels adjacent to the sub-pixel.

More specifically, in FIG. 8, the pixel information recorded in the sub-pixel 803b located between the main pixel 803a of the first pixel group 803 and the main pixel 804a of the second pixel group 804 can be determined as two main pixels The average 807 of the pixel information of 803a and 804a.

Likewise, the pixel information recorded in the sub-pixel 804b located between the main pixel 804a of the second pixel group 804 and the main pixel 805a of the third pixel group 805 can be determined as the average of the pixel information of the two main pixels 804a and 805a Value 808.

If the interpolation of the second intermediate image 806 is performed in this way, the final image generation module 302c performs sharpening of the interpolated second intermediate image 806 . As a result, a final image with a high resolution (ie, the resolution of the sub-image sensor×4) is produced from the color-separated image with a low resolution (ie, the resolution of the sub-image sensor) obtained by each sub-image sensor 809.

For simplicity, the image generating module 302 described with reference to FIG. 8 will be referred to as a second image generating module. FIG. 9 is a diagram showing a digital camera module of a second embodiment of the present invention. The digital camera module according to the second embodiment of the present invention has the same structure as that of the digital camera module according to the first embodiment of the present invention shown in FIG. 4A except for the following characteristics. That is, the camera module according to the second embodiment includes a plurality of lenses 901a to 901d having different colors. Here, the plurality of lens 901a values 901d may be divided into a first group and a second group according to transmittance. The lenses included in the second group may have a color having higher transmittance than the lenses included in the first group.

Specifically, for example, the first group may include a first lens 901a having green, a second lens 901b having red, and a third lens 901c having blue among four lenses, and the fourth lens included in the second group 901d may have a color of higher transmittance than green, red, and blue, ie, gray.

Therefore, when the plurality of lenses 901a to 901d have different colors, no color separation filter layer is formed in the plurality of sub image sensors 902a to 902d.

In addition, the image sensor 902 is divided into a plurality of sub image sensors 902a to 902d respectively corresponding to the plurality of lenses 901a to 901d, and a color separation image is obtained by using the plurality of sub image sensors 902a to 902d. As shown in FIG. 3 , the color-separated image is provided from the camera module 301 through the image generation module 302 , and a final image is generated based on the color-separated image, and then displayed on the display module 303 . The image generation module 302 that generates the final image is the same as that described above with reference to FIGS. 3 to 8 , and a detailed explanation thereof will not be given.

Next, a method of restoring a high-pixel image according to an embodiment of the present invention will be described with reference to FIGS. 10 and 11 .

FIG. 10 is a flowchart illustrating a method of restoring a high-pixel image using the first image generation module illustrated in FIG. 7B with the camera module having the structure illustrated in FIG. 4A .

For convenience of explanation, it is assumed that a red filter, a green filter, a blue filter and a gray filter are respectively formed in the plurality of sub-image sensors 402a to 402d forming the image sensor 402 as shown in FIG. 7B. Also, assuming that the image sensor 402 is formed with 8×8 pixels (width×height), each of the plurality of sub image sensors 402 a to 402 d forming the image sensor 402 is formed with 4×4 pixels (width×height).

First, in operation S1001, light reflected from a predetermined object 705 is concentrated through four lenses 401a to 401d.

In operation S1002, light rays concentrated through the lenses 401a to 401d are transmitted through color filters included in the respective sub-image sensors 402a to 402d matched with the lenses 401a to 401d.

As a result, a plurality of color separation images are obtained through the respective sub-image sensors 402a to 402d in operation S1003. Here, an image obtained by each of the sub-image sensors 402 a to 402 d has a resolution of one quarter (1/4) of the resolution of the image sensor 402 . That is, since the resolution of the image sensor 402 is 8×8, each of the plurality of color separation images obtained by the respective sub-image sensors 402 a to 402 d has a resolution of 4×4.

After operation S1003, in operation S1004, the image generation module 302 checks whether the original color separation images 706 obtained in operation S1003 are located at their designated positions.

As a result of the check, if it is determined that the obtained original color separation images 706 deviate from their designated positions, then in operation S1005, the image generation module 302 corrects the positions of the original color separation images 706 so that the original color separation images 706 are normally positioned to their specify the location.

If it is determined that the obtained original color separation images 706 are located at their designated positions, in operation S1006, the image generating module 302 checks whether the sensitivity of the original color separation images 706 obtained in operation S1003 is consistent.

As a result of the check, if it is determined that the obtained original color separation images 706 have inconsistent sensitivities, the image generation module 302 corrects the inconsistency in sensitivity based on the sensitivity of the original color separation images having the lowest sensitivity level in operation S1007.

If it is determined that the obtained original color separation image 706 has the same sensitivity, then in operation S1008, the image generation module 302 generates a plurality of original images based on the obtained original color separation image 706, and rearranges the pixels of the pixels at the same position in each original image information, so that an intermediate image 707 having a resolution higher than that of the original image for each color can be generated.

Thereafter, in operation S1009, the intermediate image is demosaiced, and in operation S1010, the demosaiced intermediate image is sharpened to generate a final image.

Next, the final image generated by the image generating module 302 is displayed through the display module 303 in operation S1011.

FIG. 11 is a flowchart illustrating a method of restoring a high-pixel image using the second image generation module illustrated in FIG. 8 with the camera module having the structure illustrated in FIG. 4A .

First, the operation of providing a plurality of color-separated images by the camera module 301 having the structure shown in FIG. 4A is the same as operations S1001 to S1007 shown in FIG. 10 . In operation S1101, as shown in FIG. 8, the original image generation module 302a divides a plurality of original color separation images into a plurality of pixel groups 303, 304 and 305, each pixel group consists of 2×2 virtual pixels (width×height) form.

Then, the intermediate image generation module 302b generates the first intermediate image 802 having the same resolution as the image sensor 402 shown in FIG. 8 in operation S1102.

Next, in operation S1103, the intermediate image generating module 302b maps the pixel information of the pixels at the same position in each original color separation image to the main pixel of the pixel group corresponding to the same position.

The intermediate image generating module 302b generates a second intermediate image 806 in which three pieces of color information and two pieces of brightness information are mapped onto main pixels 803a, 804a, and 805a of each pixel group 803, 804, and 805 in operation S1104.

In operation S1105, the intermediate image generation module 302b interpolates the second intermediate image 806 using an interpolation method.

After interpolating the second intermediate image 806, the final image generation module 302c performs sharpening of the interpolated second intermediate image 806 in operation S1106.

As a result, in operation S1107, a final image 809 with a high resolution (ie, the resolution of the sub-image sensor×4) is generated from the color-separated image with a low resolution (ie, the resolution of the sub-image sensor), and then The final image 809 is displayed by the display module 303 .

From operation S1001 to operation S1007, in addition to concentrating light through a color lens instead of a color filter, and obtaining a plurality of original color separation images through a sub-image sensor, the camera module with the structure shown in FIG. 10 is used. The shown high pixel image restoration method is essentially the same as the high pixel image restoration method using the first image generation module shown in FIG. 7B. Similarly, as described above, since the operation of providing a plurality of color-separated images by the camera module 301 having the structure shown in FIG. 4A is the same as operations S1001 to S1007 shown in FIG. The high pixel image restoration method shown in Figure 11 of the module is essentially the same as the high pixel image restoration method using the second image generation module shown in Figure 8 .

In a word, using the digital camera according to the embodiment of the present invention can realize the same image brightness as that of the image sensor shown in FIG. 2A due to the same F-number. The digital camera includes four lenses and four image sensors. Each of the four image sensors has a relatively smaller lens size and a relatively shorter focal length than the lens shown in FIG. A small number of pixels, ie, one million pixels. In addition, as shown in FIG. 2A , using 4 images taken by the sub-image sensors shown in FIG. 4A each capable of 1-megapixel resolution imaging, it is possible to restore the image with the image sensor capable of 4-megapixel resolution through a predetermined process. Pixel resolution imaging cameras capture images of the same resolution as images.

Since the digital camera according to the embodiment of the present invention has a relatively small lens and a relatively short focal length, it is possible to achieve a miniaturized and lightweight design of the digital camera while maintaining the same resolution.

According to the above method and device for restoring a high-pixel image, one or more of the following effects can be obtained.

The method and apparatus provide the advantage of being able to obtain high pixel images using a miniaturized camera module.

Since the size of a digital camera mounted on a small digital device is reduced, it is also possible to miniaturize the digital device.

In addition, the present invention can restore high-pixel images while reducing the size of a digital camera mounted on a compact digital device.

In addition, the apparatus and method for restoring a high-pixel image according to the present invention can easily correct optical deviation and inconsistency in sensitivity while expressing a high-pixel image.

By using color filters with different transmittances, the apparatus and method for restoring a high-pixel image according to the present invention can simultaneously realize high-sensitivity image sensing and low-sensitivity image sensing without changing the structure of an image sensor unit.

In addition, the apparatus and method for restoring high-pixel images according to the present invention allow more options to be available when designing a small digital device mounted with a digital camera.

While certain embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the claims and its equivalents.

Claims (44)

1. A device for restoring high-resolution images, comprising: a camera module including a plurality of lenses and a plurality of sub-image sensors corresponding to the plurality of lenses, each of the plurality of sub-image sensors including a color filter having a single color; The original image generation module receives a plurality of original color separation images; an intermediate image generation module rearranging pixel information of pixels at the same position of a plurality of original color separation images provided by the original image generation module, and generating an intermediate image having a resolution higher than that of each original color separation image; and The final image generating module performs demosaicing on the intermediate image, clears the demosaiced intermediate image, and generates a final image. 2. The apparatus of claim 1, wherein the plurality of lenses concentrates light reflected from a predetermined object at a designated position of the lens or in a plurality of lenses shifted by a predetermined number of pixels based on a position of the predetermined lens The location of shots other than the intended shots. 3. The apparatus of claim 1, wherein when the obtained original color separation image is not arranged at a designated position of the corresponding sub-image sensor, the original image generation module corrects the position of the original image. 4. The device as claimed in claim 1, wherein when the obtained original color separation images have inconsistent sensitivities, the original image generation module corrects the original image except for the one with the lowest sensitivity level based on the sensitivity of the original image with the lowest sensitivity level. Sensitivity level for raw images other than raw. 5. The apparatus of claim 1, wherein the color filter is divided into a first filter area and a second filter area according to transmittance on which the color is applied. 6. A device for restoring high-resolution images, comprising: A camera module including a plurality of lenses and a plurality of sub-image sensors corresponding to the plurality of lenses, each of the plurality of sub-image sensors including a color filter divided into a plurality of color regions having different colors ; The original image generation module divides multiple original color separation images into multiple pixel groups; The intermediate image generation module maps the pixel information of the pixels at the same position in the plurality of original color separation images divided into a plurality of pixel groups to each pixel of the pixel group corresponding to the same position, and generates a pixel with a value higher than each an intermediate image at the resolution of the original color separation images; and The final image generation module restores the intermediate image using a predetermined interpolation algorithm, performs sharpening of the received intermediate image, and generates a final image. 7. The apparatus of claim 6, wherein when the obtained original color-separated image is not arranged at the specified position of the corresponding sub-image sensor, the original image generation module corrects the position of the original image. 8. The device as claimed in claim 6, wherein when the sensitivities of the obtained original color-separated images are inconsistent, the original image generation module corrects the original image except the one with the lowest sensitivity level based on the sensitivity of the original image with the lowest sensitivity level. Sensitivity level for raw images other than raw. 9. The apparatus of claim 6, wherein the color filter is divided into a first filter area and a second filter area according to the transmittance of the color painted thereon, and the transmittance of the color painted on the second filter area The rate is higher than the transmittance of the color painted on the first filter area. 10. The apparatus of claim 6, wherein each of the plurality of pixel groups includes a plurality of pixels corresponding to an arrangement pattern of the color filters. 11. The device of claim 6, wherein the pixel information includes brightness information provided by a sub-image sensor corresponding to the first filter area and color information provided by a sub-image sensor corresponding to the second filter area. 12. A device for restoring high-resolution images, comprising: A camera module comprising a plurality of color lenses and a plurality of sub-image sensors corresponding to the plurality of color lenses, wherein each of the plurality of color lenses has a single color, and a plurality of original color separation image; The original image generation module receives a plurality of original color separation images; an intermediate image generation module rearranging pixel information of pixels at the same position of a plurality of original color separation images provided by the original image generation module, and generating an intermediate image having a resolution higher than that of each original color separation image; and The final image generating module performs demosaicing on the intermediate image, clears the demosaiced intermediate image, and generates a final image. 13. The apparatus of claim 12, wherein the plurality of lenses concentrates light reflected from a predetermined object at a designated position of the lens or in a plurality of lenses shifted by a predetermined number of pixels based on a position of the predetermined lens The location of shots other than the intended shots. 14. The apparatus of claim 12, wherein when the obtained original color-separated image is not arranged at a designated position of the corresponding sub-image sensor, the original image generation module corrects the position of the original image. 15. The device as claimed in claim 12, wherein, when the obtained raw color separation images have inconsistent sensitivities, the raw image generating module corrects the color except for the raw image with the lowest sensitivity class based on the sensitivity Sensitivity level for raw images other than raw. 16. The apparatus as claimed in claim 12, wherein each of the plurality of color lenses is divided into a first lens area and a second lens area according to a transmittance on which the color is painted, and painted on the second lens area The transmittance of the color on is higher than the transmittance of the color painted on the first lens area. 17. A device for restoring high-resolution images, comprising: A camera module comprising a plurality of color lenses and a plurality of sub-image sensors corresponding to the plurality of color lenses, wherein each of the plurality of color lenses has a single color, and a plurality of original color separation image; The original image generation module divides multiple original color separation images into multiple pixel groups; an intermediate image generating module, which maps pixel information of pixels at the same position in a plurality of original color separation images divided into a plurality of pixel groups to each pixel of the pixel group corresponding to the same position, and generates an intermediate image; and The final image generation module restores the intermediate image using a predetermined interpolation algorithm, performs sharpening of the received intermediate image, and generates a final image. 18. The apparatus of claim 17, wherein when the obtained original color-separated image is not arranged at the specified position of the corresponding sub-image sensor, the original image generation module corrects the position of the original image. 19. The apparatus as claimed in claim 17, wherein, when the obtained raw color separation images have inconsistent sensitivities, the raw image generating module corrects the color except for the raw image with the lowest sensitivity class based on the sensitivity of the raw image with the lowest sensitivity class. Sensitivity level for raw images other than raw. 20. The apparatus of claim 17, wherein each of the color lenses is divided into a first lens area and a second lens area according to a transmittance on which the color is painted, and the color painted on the second lens area The transmittance of the color is higher than the transmittance of the color painted on the first lens area. 21. The apparatus of claim 17, wherein each of the plurality of pixel groups includes a plurality of pixels corresponding to an arrangement pattern of the color filters. 22. The device of claim 20, wherein the pixel information includes brightness information provided by a sub-image sensor matching the first lens area and color information provided by the sub-image sensor matching the second lens area. 23. A method of restoring a high-pixel image in a camera module, the camera module comprising a plurality of lenses and a plurality of sub-image sensors corresponding to the plurality of lenses, each of the plurality of lenses comprising a monochromatic A color filter, the method comprising: obtaining a plurality of original images through each of the plurality of sub-image sensors; receiving a plurality of original images, and generating a plurality of original color-separated images; rearranging pixel information of pixels at the same position of a plurality of original color separation images provided from the original image generation module of the camera module, and generating an intermediate image having a resolution higher than that of each original color separation image; and Demosaicing is performed on the intermediate image, sharpening the demosaiced intermediate image is performed, and a final image is generated from the sharpened image. 24. The method of claim 23, further comprising concentrating the light reflected from the predetermined object on a specified position of the lens or on a plurality of lenses except the predetermined lens shifted by a predetermined number of pixels based on the position of the predetermined lens. outside the lens position. 25. The method of claim 24, wherein when the obtained original color separation images are not arranged at the specified positions of the corresponding sub-image sensors, generating the plurality of original color separation images comprises correcting the positions of the original images. 26. The method of claim 23, wherein when the obtained original color separation images have inconsistent sensitivities, generating a plurality of original color separation images comprises correcting the sensitivity of the original image with the lowest sensitivity level except Sensitivity level of the original image other than the original image of the sensitivity level. 27. The method of claim 23, further comprising dividing the color filter into a first filter area and a second filter area according to transmittance on which the color is applied. 28. A method of recovering a high pixel image in a camera module comprising a plurality of lenses and a plurality of sub-image sensors corresponding to the plurality of lenses, each of the plurality of lenses comprising a color filter with monochrome device, the method includes: obtaining a plurality of original color-separated images through each of the sub-image sensors; dividing the obtained multiple original color separation images into multiple pixel groups; mapping pixel information of pixels at the same position in a plurality of original color separation images divided into a plurality of pixel groups to respective pixels of pixel groups corresponding to the same position, and generating an intermediate image; and The intermediate image is restored using a predetermined interpolation algorithm, sharpening the interpolated intermediate image is performed, and a final image is generated from the sharpened image. 29. The method of claim 28, wherein when the obtained original color separation images are not arranged at the designated positions of the corresponding sub-image sensors, generating the plurality of original color separation images comprises correcting the positions of the original images. 30. The method according to claim 28, further comprising: generating a plurality of original color separation images from the received color separation images, wherein when the obtained original color separation images have inconsistent sensitivities, generating the plurality of original color separation images comprises The sensitivity levels of the original images other than the original image with the lowest sensitivity level are corrected based on the sensitivity of the original image with the lowest sensitivity level. 31. The method of claim 28, further comprising dividing the color filter into a first filter area and a second filter area according to the transmittance on which the color is applied. 32. The method of claim 28, wherein each of the plurality of pixel groups includes a plurality of pixels corresponding to an arrangement pattern of the color filters. 33. The method of claim 31, wherein the pixel information includes brightness information provided by the sub-image sensor matching the first filtering area and color information provided by the sub-image sensor matching the second filtering area. 34. A method of restoring a high-pixel image in a camera module, the camera module comprising a plurality of color lenses and a plurality of sub-image sensors corresponding to the plurality of color lenses, each of the plurality of color lenses including a monochromatic The color filter, the method includes: Obtaining a plurality of original color-separated images through each of the plurality of sub-image sensors; Receive multiple raw color separation images; rearranging pixel information of pixels at the same position of a plurality of original color separation images provided from the original image generation module of the camera module, and generating an intermediate image having a resolution higher than that of each original color separation image; and Demosaicing is performed on the intermediate image, sharpening the demosaiced intermediate image is performed, and a final image is generated from the sharpened image. 35. The method of claim 34, further comprising: using a plurality of lenses to concentrate light reflected from a predetermined object at a specified position of the lens or in a plurality of lenses shifted by a predetermined number of pixels based on the position of the predetermined lens The location of shots other than the intended shots. 36. The method of claim 34, wherein when the obtained original color separation images are not arranged at the designated positions of the corresponding sub-image sensors, generating the plurality of original color separation images comprises correcting the positions of the original images. 37. The method according to claim 34, further comprising: generating a plurality of original color separation images from the received color separation images, wherein when the obtained original color separation images have inconsistent sensitivities, generating the plurality of original color separation images comprises The sensitivity levels of the original images other than the original image with the lowest sensitivity level are corrected based on the sensitivity of the original image with the lowest sensitivity level. 38. The method as claimed in claim 34, further comprising: dividing each of the plurality of color lenses into a first lens area and a second lens area according to the transmittance on which the color is painted, and painting the second lens area The transmittance of the color on the area is higher than the transmittance of the color painted on the first lens area. 39. A method of restoring a high-pixel image in a camera module, the camera module comprising a plurality of color lenses and a plurality of sub-image sensors corresponding to the plurality of color lenses, each of the plurality of color lenses having a single color, The methods include: Obtain multiple original color separation images through multiple sub-image sensors; dividing a plurality of original color separation images into a plurality of pixel groups; mapping pixel information of pixels at the same position in a plurality of original color separation images divided into a plurality of pixel groups to respective pixels of pixel groups corresponding to the same position, and generating an intermediate image; and The intermediate image is restored using a predetermined interpolation algorithm, sharpening the interpolated intermediate image is performed, and a final image is generated. 40. The method as claimed in claim 39, further comprising: generating a plurality of original color separation images from the received color separation images, wherein when the obtained original color separation images are not arranged in the designated positions of the corresponding sub-image sensors, generating The plurality of raw color separation images includes correcting the position of the raw image. 41. The method of claim 39, wherein when the obtained original color separation images have inconsistent sensitivities, generating a plurality of original color separation images comprises correcting the sensitivity of the original image with the lowest sensitivity level except for those with the lowest sensitivity level. Sensitivity level of the original image other than the original image of the sensitivity level. 42. The method as claimed in claim 39, further comprising: dividing each of the plurality of color lenses into a first lens area and a second lens area according to the transmittance on which the color is painted, and painting on the second lens The transmittance of the color on the area is higher than the transmittance of the color painted on the first lens area. 43. The method of claim 39, wherein each of the plurality of pixel groups includes a plurality of pixels corresponding to an arrangement pattern of color filters of the camera module. 44. The method of claim 42, wherein the pixel information includes brightness information provided by a sub-image sensor matching a first lens area and color information provided by a sub-image sensor matching a second lens area.

Images