JP2004228662A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique by which the intensity of the light of not less than four wavelength bands can be obtained by one exposure. <P>SOLUTION: An image pickup sensor 3 is provided with a plurality of light-receiving elements P which are two-dimensionally arranged, and a filter for transmitting the light of a specific wavelength band is attached on each of the light-receiving elements P. This filter includes four kinds of filters i.e. a B filter for transmitting the light of a blue wavelength band, a G1 filter for transmitting the light of a green wavelength band being comparatively short, G2 filter for transmitting the light of a green wavelength band being comparatively long, and an R filter for transmitting the light of a red wavelength band. In this way, the intensity of lights of four wavelength bands can be obtained by one exposure. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a technique for acquiring light intensities in a plurality of wavelength bands.
[0002]
[Prior art]
Generally, an imaging apparatus such as a digital still camera and a digital video camera includes an imaging sensor that obtains the intensity of light reflected from a subject in three wavelength bands respectively corresponding to three primary colors to obtain color information of the subject. I have to. FIG. 1 is a diagram showing a conventional general imaging sensor 300. The imaging sensor 300 has a plurality of light receiving pixels P arranged two-dimensionally, and each of the light receiving pixels P is provided with one of filters that transmit light in a wavelength band corresponding to three primary colors. As shown in the figure, the G filters that transmit light in the green wavelength band are arranged in a checkered pattern (one of the checkered patterns), and the B filters that transmit light in the blue wavelength band and the light that transmits the red wavelength band. The R filters are arranged evenly in the remaining area (the other area of the checkered pattern). This RGB arrangement is called a “Bayer arrangement”. By employing such an imaging sensor, the imaging apparatus can obtain the light intensities (three primary colors) in three wavelength bands as color information of the subject (for example, see Patent Document 1).
[0003]
On the other hand, in the past, attempts have been made to obtain the spectral reflectance of an object from an image obtained by an imaging apparatus in order to solve the problem of metamerism or to analyze the components of an object. I have. The spectral reflectance of the subject can be obtained by obtaining the spectral distribution of the reflected light from discrete color information of the reflected light from the subject, and removing the spectral distribution of the illumination light from the spectral distribution of the reflected light. As a method of obtaining a spectral distribution from discrete color information of reflected light, spline interpolation processing, estimation by principal component analysis, Winner estimation, and the like are known.
[0004]
However, the spectral distribution of the reflected light cannot be obtained with high accuracy only by the intensity of light in three wavelength bands obtained by a general imaging device. For this reason, a multi-band camera, which is an imaging device capable of acquiring the intensity of light in four or more wavelength bands, has been proposed. Conventionally proposed methods of a multi-band camera include the following.
[0005]
1. A color wheel type in which four or more filters having different spectral transmission characteristics are arranged in a disk shape, and these filters are sequentially arranged in the path of incident light to perform imaging.
[0006]
2. A tunable filter system that uses an interferometer or an optical element that transmits only a specific wavelength band due to the polarization phenomenon of liquid crystal, and sequentially changes the transmitted wavelength band to perform imaging.
[0007]
3. A grating system that separates incident light into different wavelength bands by diffraction and performs two-dimensional imaging by scanning.
[0008]
In addition, there is Patent Document 2 as prior art document information related to this application.
[0009]
[Patent Document 1]
U.S. Pat. No. 3,971,065
[Patent Document 2]
JP 2001-61109 A
[0010]
[Problems to be solved by the invention]
However, conventionally proposed multi-band cameras require switching and scanning of wavelength bands, and in each case, it is not possible to obtain the intensity of light in all the target wavelength bands with a single exposure. There is a problem that it takes a long time and an object or the like that moves at a relatively high speed cannot be a subject.
[0011]
The present invention has been made in view of the above problems, and has as its object to provide a technique capable of obtaining light intensities in four or more wavelength bands by one exposure.
[0012]
[Means for Solving the Problems]
In order to solve the above problem, an invention according to claim 1 is an imaging apparatus, wherein a first type filter that transmits light in a green wavelength band and a second type filter that transmits light in a wavelength band shorter than the first type filter. A type filter, and an imaging sensor having a two-dimensional array of a plurality of light receiving pixels each having a third type filter that transmits light in a wavelength band longer than the first type filter, The first type filter is attached to the light receiving pixels corresponding to one area of the checkered pattern in the array of the plurality of light receiving pixels, and the second and third type filters correspond to the other area of the checkered pattern. And at least one of the first to third type filters includes two or more types of filters having different wavelength bands of transmitted light. It is characterized in.
[0013]
According to a second aspect of the present invention, in the imaging device according to the first aspect, the image sensor outputs an image having a plurality of pixels corresponding to any one of four or more colors, and the four or more types are output. Are classified into any one of three groups corresponding to the three primary colors. For each of the groups, the value of the pixel corresponding to the one primary color is determined from the value of the pixel corresponding to the color included in the group. Derivation means for deriving a value is further provided.
[0014]
According to a third aspect of the present invention, in the imaging device according to the first aspect, the image sensor outputs a first image having a plurality of pixels respectively corresponding to any one of four or more colors. Each kind of color is classified into any one of three groups corresponding to the three primary colors, and for each of the groups, from the pixel values corresponding to the colors included in the group, one second image And a compression unit that compresses a third image composed of the three generated second images to generate one compressed image.
[0015]
The invention according to claim 4 is an imaging apparatus, wherein the imaging sensor has a predetermined number of four or more imaging regions formed of a plurality of light receiving pixels arranged in a plane, and the imaging sensor includes a plurality of imaging regions. A microlens array having the predetermined number of microlenses for forming a light image of a subject in a plane, and the predetermined number of filters corresponding to the predetermined number of microlenses in a plane. Wherein the predetermined number of filters have different wavelength bands of transmitted light.
[0016]
The invention according to claim 5 is an imaging apparatus, wherein the imaging sensor includes a plurality of imaging regions including a plurality of light receiving pixels arranged in a plane, and an optical image of a subject is provided on the plurality of imaging regions. A microlens array having a plurality of planarly arranged microlenses, each of which forms an image, a filter array having a plurality of planarly arranged filters respectively corresponding to the plurality of microlenses, and A rearrangement unit that rearranges pixels included in the image of the subject obtained in each of the image regions to generate one rearranged image, wherein the plurality of filters are arranged in a wavelength band of transmitted light. Are different from each other in four or more types.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0018]
<1. First Embodiment>
<1-1. Configuration>
FIG. 2 is a diagram illustrating a basic configuration of the imaging device according to the embodiment of the present invention. The imaging device 1 is configured as, for example, a digital still camera, a digital video camera, or a colorimeter. Regardless of the configuration of the imaging device 1, the basic configuration of the imaging device 1 is the same. That is, the imaging apparatus 1 mainly includes an imaging unit 40, an A / D converter 41, an image processing unit 42, an overall control unit 50, a card slot 43, an operation input unit 44, and a data output terminal 45.
[0019]
The imaging unit 40 includes the lens 2 and the imaging sensor 3 that photoelectrically converts a light image of a subject formed by the lens 2 and outputs an image signal (hereinafter, simply referred to as an “image”). The image output from the image sensor 3 is converted into a digital signal by the A / D converter 41, and a predetermined process is performed by the image processing unit 42 as needed. The image thus obtained is recorded on a memory card 91 inserted in the card slot 43 or transmitted to an external device 92 connected via the data output terminal 45.
[0020]
The operation input unit 44 indicates an operation member for receiving various commands such as an instruction to start imaging from a user. The imaging device 1 performs a predetermined operation in response to the operation of the operation input unit 44.
[0021]
The overall control unit 50 has a control function for controlling each unit of the above-described imaging apparatus 1 and a data processing function for performing various data processing, and includes a microcomputer. More specifically, the overall control unit 50 includes a CPU 51 as its main body, a ROM 52 for storing a control program and the like, a RAM 53 as a work area, and the like. Various functions of the overall control unit 50 are realized by the CPU 51 executing a control program stored in the ROM 52.
[0022]
The imaging sensor 3 included in the imaging unit 40 includes a plurality of light receiving pixels each having a filter (color filter) that passes light in a specific wavelength band, two-dimensionally arranged. Each of the light receiving pixels photoelectrically converts the light transmitted through the filter attached thereto and outputs a signal charge corresponding to the intensity thereof. In this embodiment, there are four types of filters having different wavelength bands of transmitted light. Thereby, the imaging device 1 is a multi-band camera capable of acquiring the intensities of light in four wavelength bands.
[0023]
FIG. 3 is a diagram illustrating an example of spectral transmission characteristics of four types of filters attached to light receiving pixels. Each of the four types of filters has a spectral transmission characteristic of transmitting light in a visible wavelength band (about 380 to about 780 nm). As shown in the figure, among the four types of filters, there are a B filter, a G1 filter, a G2 filter, and an R filter, starting from the one that transmits short wavelength bands. The B filter transmits light in the blue wavelength band, the G1 filter transmits light in a relatively short wavelength band of the green wavelength band, the G2 filter transmits light in a relatively long wavelength band in the green wavelength band, The R filter transmits light in the red wavelength band. That is, in the present embodiment, two types of filters, a G1 filter and a G2 filter, are included as filters of a type that transmits light in the green wavelength band (hereinafter, referred to as “G-type filters”). The peaks of the spectral transmission characteristic lines of the G1 filter and the G2 filter are arranged, for example, at positions where the peaks of the spectral transmission characteristic lines of the B filter and the R filter are equally divided into three. Hereinafter, the color of light transmitted by the R filter is R, the color of light transmitted by the G1 filter is G1, the color of light transmitted by the G2 filter is G2, and the color of light transmitted by the B filter is B. Also, let G be the color of light transmitted by a general G filter that transmits light in the green wavelength band. Here, R, G, and B are three primary colors of light.
[0024]
FIG. 4 is a diagram showing an example of the arrangement of these four types of filters in the image sensor 3. As shown in FIG. 4, the G type filters (G1 filter and G2 filter) are attached to the light receiving pixels P corresponding to one area of the checkered pattern in the array of the plurality of light receiving pixels P. Then, the G1 filter and the G2 filter are alternately arranged in the vertical direction. On the other hand, the B filter and the R filter are attached to the light receiving pixels P corresponding to the other area of the checkered pattern (the area where the G type filter is not arranged). The B filters and the R filters are alternately arranged in the vertical direction. That is, the G1 filter and the G2 filter are evenly arranged at the positions of the G filters in the Bayer array (see FIG. 1), and the B and R filters are arranged at the same positions as in the Bayer array. As a result, the four types of filters are uniformly arranged on the entire light receiving surface of the image sensor 3.
[0025]
These four types of filters are formed by repeating the selective formation of a dyed layer (a resin film of gelatin or the like) using photolithography technology four times on light-receiving pixels made of silicon or the like. The spectral transmission characteristics of each filter are adjusted by the dye that dyes the dyed layer and the thickness of the dyed layer. Note that the spectral transmission characteristics of the filter are not limited to those shown in FIG. 3, and may be characteristics according to the purpose of the imaging device 1. For example, if the imaging device 1 is to image an object that emits (or reflects) light having a bright line spectrum such as a fluorescent lamp, one filter of the image sensor 3 mainly transmits the bright line spectrum. It may have a spectral transmission characteristic. Further, in the case of the imaging device 1 used for a specific purpose, it is preferable to adjust the spectral transmission characteristics of the filter in consideration of changes in characteristics due to an environment in which the imaging device is used and aging.
[0026]
The signal output from each light receiving pixel of the image sensor 3 indicates the intensity of light in any of the four wavelength bands shown in FIG. 3 according to the type of filter attached to the light receiving pixel from which the signal was output. It will be. Therefore, the imaging device 1 can obtain light intensities of four wavelength bands by one exposure.
[0027]
<1-2. Image processing>
The signal output from each light receiving pixel of the imaging sensor 3 is a value of each pixel in an image output from the imaging sensor 3 (hereinafter, referred to as a “sensor output image”). Therefore, the value of each pixel included in the sensor output image indicates the intensity of any one of the four wavelength bands, that is, a value corresponding to any one of the colors B, G1, G2, and R. As a result, the sensor output image is in a state having four types of color values. The arrangement of the colors corresponding to each pixel in the sensor output image is the same as the arrangement of the filters shown in FIG.
[0028]
The sensor output image is preferably subjected to appropriate processing according to the use at the output destination. For this reason, in the imaging device 1 of the present embodiment, the image processing unit 42 performs image processing on the sensor output image according to the output destination of the sensor output image. Hereinafter, the processing content of the image processing unit 42 will be described. In the following description, in the image, the pixel corresponding to R corresponds to the R pixel, the pixel corresponding to B corresponds to the B pixel, the pixel corresponding to G corresponds to the G pixel, the pixel corresponding to G1 corresponds to the G1 pixel, and G2. The pixels are respectively referred to as G2 pixels.
[0029]
<1-2-1. Operation>
First, a case will be described in which a computer is connected as the external device 92, and the computer uses a sensor output image for calculation for obtaining the spectral distribution of light from a subject. In this case, after the sensor output image is converted into a digital signal by the A / D converter 41, the image is not directly processed by the image processing unit 42, but is directly output to the computer as the external device 92 via the data output terminal 45. Is done. By outputting a sensor output image without performing image processing in this manner, an image including light intensities (four colors) in four wavelength bands without data deterioration due to image processing can be output. By using such an image including the light intensities in the four wavelength bands, it is possible to improve the calculation accuracy when calculating the spectral distribution of light from a subject in a computer.
[0030]
<1-2-2. Display>
Next, a case where a color display device is connected as the external device 92 and a sensor output image is displayed on the color display device will be described. In general, a color display device is configured to display an image in which the color information of each pixel is represented by three primary colors of RGB, and thus the sensor output image cannot be used as it is for display on the color display device. For this reason, when displaying the sensor output image on the color display device, a normal image in which each pixel has the values of the three primary colors of RGB is generated in the image processing unit 42 from the sensor output image.
[0031]
FIG. 5 is a diagram conceptually illustrating a method of generating the normal image D3 from the sensor output image D1. In generating the normal image D3, first, from the sensor output image D1, an image in which each pixel has any one of the three primary colors of RGB and the arrangement of the three primary colors is a Bayer array (hereinafter, referred to as a “Bayer image”). ) D2 is generated, and the generated Bayer image D2 is subjected to interpolation of missing pixel values (a process of generating missing color information in one pixel) to generate a normal image D3.
[0032]
In generating the Bayer image D2, R, G1, G2, and B are divided into three groups corresponding to the three primary colors of RGB, and the value of the pixel corresponding to one primary color of the Bayer image is obtained for each group. Specifically, R among R, G1, G2, and B is classified into the group of primary colors R. Therefore, the value of the R pixel in the sensor output image D1 is set to the value of the R pixel in the Bayer image D2. Similarly, since B among R, G1, G2, and B is classified into the group of primary colors B, the value of the B pixel in the sensor output image D1 is the value of the B pixel in the Bayer image D2. Further, G1 and G2 among R, G1, G2 and B are classified into the group of primary colors G. Therefore, the value of the G pixel of the Bayer image D2 is derived from the values of the G1 pixel and the G2 pixel of the sensor output image D1.
[0033]
The value of the G pixel of the Bayer image D2 at the same position as the G1 pixel of the sensor output image D1 is obtained by convolving the 3 × 3 pixels centered on the G1 pixel of the sensor output image D1 using the filter mask shown in FIG. It is derived by performing an operation. That is, the value of the G pixel is derived by adding the result of multiplying the value of the G1 pixel by the weight α and the result of multiplying the average value of the values of the four G2 pixels in the vicinity by the weight β. . The G pixel value of the Bayer image D2 at the same position as the G2 pixel of the sensor output image D1 is calculated using the filter mask shown in FIG. 7 for 3 × 3 pixels centered on the G2 pixel of the sensor output image D1. It is derived by performing a convolution operation. That is, the value of the G pixel is derived by adding the result of multiplying the value of the G2 pixel by the weight β and the result of multiplying the average value of the values of the four G1 pixels in the vicinity by the weight α. . Here, the weights α and β satisfy the following equation (1).
[0034]
α + β = 1 (1)
The weights α and β can be determined based on the relationship between the spectral transmission characteristics of the G1 filter and the G2 filter and the spectral transmission characteristics of a general G filter. For example, when the wavelength bands of the light transmitted by the G1 filter and the G2 filter are distributed symmetrically with respect to the center position of the wavelength band of the light transmitted by the general G filter, α = β = 0.5. do it. If the G1 filter has a spectral transmission characteristic that is closer to a general G filter than the G2 filter, α is relatively large, and conversely, the spectral transmission characteristic that the G2 filter approximates a general G filter. , Β may be made relatively large.
[0035]
When the Bayer image D2 is generated in this way, interpolation of the missing pixel value is performed. That is, the value of the R pixel in the Bayer image D2 is the value of R for all pixels, the value of the G pixel is the value of G for all pixels, and the value of the B pixel is B for all pixels. Required by Thereby, the normal image D3 in which each pixel has the values of the three primary colors of RGB is generated. Note that the interpolation processing of the missing pixel value is equivalent to generating three color component images composed of only one color value.
[0036]
In this manner, R, G1, G2, and B are divided into three groups corresponding to the three primary colors, and the values of the pixels corresponding to the one primary color of the Bayer image are derived for each group. A normal image D3 in which color information is expressed by three primary colors can be generated from the sensor output image D1. Thus, a sensor output image that cannot be displayed as it is can be used for display on a color display device.
[0037]
When the color display device requires a television signal, after generating the normal image D3 as described above, the RGB values of each pixel are determined by a predetermined method such as the NTSC system, the PAL system, and the SECAM system. May be converted into a television signal of the following format. The conversion from the RGB values to the television signal can be easily performed by a known method.
[0038]
<1-2-3. Printing>
Next, a case where a color printing device is connected as the external device 92 and the sensor output image is printed by the color printing device will be described. Generally, a color printing apparatus prints an image in which the color information of each pixel is represented by three colors of CMY (cyan, magenta, yellow) or four colors of CMYK (cyan, magenta, yellow, black). Be composed. For this reason, the sensor output image cannot be used as it is for printing in a color printing apparatus.
[0039]
Therefore, in this case, the image processing unit 42 first generates a normal image in which the color information of each pixel is represented by the three primary colors of RGB by the same method as that used for display, and further generates the generated normal image. Are converted to CMY values or CMYK values. Conversion from RGB values to CMY values or CMYK values can be easily performed by a known method. Through such processing, the sensor output image can be used for printing in the color printing apparatus.
[0040]
By the way, when the sensor output image is used for display or printing, in order to make the image more beautiful, sharpening processing for enhancing the edge of the subject, gradation correction processing for adjusting contrast, and color correction for adjusting hue and saturation are performed. Image processing such as processing may be performed. Of these image processes, the sharpening process is preferably performed on the sensor output image at a stage before the normal image is generated.
[0041]
In the sharpening process, one pixel of the sensor output image is set as a target pixel, and convolution operation using the filter mask of FIG. 8 is performed on 5 × 5 pixels centered on the target pixel to obtain the target pixel. A new value for the pixel is determined. By performing such a calculation using all the pixels of the sensor output image as the target pixels, the edge of the subject in the sensor output image is emphasized. In the sensor output image, there are four pixels corresponding to the same color as the target pixel and having the same horizontal or vertical direction in a 5 × 5 pixel region centered on the target pixel. This is the same even when a pixel corresponding to any color becomes a target pixel. Therefore, by performing the calculation using the filter mask shown in FIG. 8 on the sensor output image, the calculation can be performed between pixels corresponding to the same color. That is, it is possible to obtain a calculation result that is not affected by the values of the pixels corresponding to other colors. Note that the gradation correction processing and the color correction processing may be performed on any of the sensor output image and the normal image because the processing result of the target pixel is not affected by the values of the peripheral pixels.
[0042]
<1-2-4. Record>
Next, a case where a sensor output image is recorded on the memory card 91 will be described. In general, when recording an image (Bayer image) output from an image sensor in which a filter array has a Bayer array, interpolation of missing pixel values is performed to obtain a normal image in which each pixel has three primary color values of RGB. And performs a compression process and records it as a compressed image. That is, a color component image for each of the three primary colors of RGB is generated, and a normal image composed of the three color component images is compressed.
[0043]
Similarly, a color component image for each of four colors B, G1, G2, and R is generated from a sensor output image obtained from the imaging sensor 3 of the present embodiment, and is configured from the four color component images. When a compressed image is generated by compressing an image, the data amount of the compressed image is increased by the increase in the number of colors. On the other hand, a normal image in which color information is expressed in three primary colors is generated by the same method as used for display, and when this normal image is compressed, the values of G1 and G2 among the four color values Will be lost.
[0044]
For this reason, in the imaging apparatus 1 of the present embodiment, R, G1, G2, and B are divided into three groups corresponding to the three primary colors of RGB, and one pixel is provided for each group while maintaining the pixel values. A color component image is generated. FIG. 9 is a diagram conceptually illustrating a method of generating three color component images from a sensor output image. As shown in the figure, one color component image D4r composed of only the value of R is generated from the value of the R pixel of the sensor output image D1, and composed only of the value of B from the value of the B pixel of the sensor output image D1. One color component image D4b is generated. Further, one color component image D4g is generated from the values of the G1 pixel and the G2 pixel of the sensor output image D1.
[0045]
In generating the color component image D4g, the value of the pixel not corresponding to G1 or G2 is derived based on the values of both the G1 pixel and the G2 pixel existing around the pixel. At the same time, the values of the G1 pixel and the G2 pixel are not changed and are maintained as they are in the color component image D4g. That is, the color component image D4g has both values of G1 and G2. Accordingly, a compressed image obtained by compressing the image D4 including the three color component images D4r, D4g, and D4b includes four color values of R, G1, G2, and B. Further, since this compressed image is obtained by compressing an image composed of three color component images, the data amount is substantially equal to that of a compressed image obtained by compressing a normal image, and the compressed image is composed of four color component images. The amount of data can be suppressed as compared with a case where a compressed image is generated by compressing an image to be compressed.
[0046]
When the compressed image obtained in this manner is used, an expansion process is performed on the compressed image to restore the image D4, and G1 and G2 pixels are respectively extracted from the color component image D4g included in the image D4. By extracting, the values of the four colors R, G1, G2, and B can be used.
[0047]
<1-3. Modification>
This embodiment is not limited to the above description, and various modifications are possible. Hereinafter, such modified examples will be described. In the following, the color of light transmitted by a filter with a symbol attached to its name is indicated by the symbol attached to the filter. For example, the color of light transmitted by the X filter is indicated by X.
[0048]
<1-3-1. Multiple B or R type filters>
In the above embodiment, the G-type filter that transmits light in the green wavelength band includes two types of filters, the G1 filter and the G2 filter. However, a filter that transmits light in the blue wavelength band (hereinafter, referred to as a G-type filter). A filter that transmits light in the red wavelength band (hereinafter, referred to as an “R-type filter”) may include two or more types of filters.
[0049]
FIG. 10 shows a B1 filter that transmits light in a relatively short wavelength band among blue wavelength bands, a B2 filter that transmits light in a relatively long wavelength band among blue wavelength bands, and a filter attached to a light receiving pixel. FIG. 9 is a diagram showing an example of the arrangement of filters in the image sensor 3 when there are four types of filters, a G filter and an R filter. In this case, the B type filter includes two types of filters, a B1 filter and a B2 filter. As a result, the image sensor 3 outputs a sensor output image including a plurality of pixels respectively corresponding to any one of the four colors B1, B2, G, and R, and obtains light intensities in four wavelength bands.
[0050]
FIG. 11 shows an R1 filter that transmits light of a relatively short wavelength band in the red wavelength band and an R2 filter that transmits light of a relatively long wavelength band in the red wavelength band, as filters attached to the light receiving pixels. FIG. 9 is a diagram illustrating an example of the arrangement of filters in the image sensor 3 when there are four types of filters, a G filter and a B filter. In this case, the R type filter includes two types of filters, an R1 filter and an R2 filter. As a result, the image sensor 3 outputs a sensor output image including a plurality of pixels respectively corresponding to any of the four colors B, G, R1, and R2, and obtains light intensities in four wavelength bands.
[0051]
In any case, the G-type filter is attached to the light-receiving pixels P corresponding to one of the areas of the checkered pattern in the array of the plurality of light-receiving pixels P, and the B-type filter and the R-type filter have the checkered pattern. The light receiving pixels P corresponding to the other area (the area where the G type filter is not disposed) are evenly applied. Since the human eye has high sensitivity to light in the green wavelength band, the G-type filter is arranged in one of the checkered areas as described above, and the number of arrangements is larger than that of the B-type filter and the R-type filter. This can enhance the resolution of the acquired image.
[0052]
Also, when generating a Bayer image from a sensor output image obtained from the image sensor 3 shown in FIG. 10 or 11 for use in display or the like, similarly to the above, three groups corresponding to the three primary colors of RGB are used. , And the values of the pixels corresponding to one primary color of the Bayer image may be obtained for each group. For example, the value of the B pixel of the Bayer image at the same position as the B1 pixel of the sensor output image obtained from the imaging sensor 3 shown in FIG. Can be derived by performing a convolution operation using the filter mask shown in FIG. Further, the value of the B pixel of the Bayer image at the same position as the B2 pixel of the sensor output image can be derived by using the filter mask shown in FIG. 12 in which α and β are interchanged.
[0053]
<1-3-2.6 types of filters>
Further, the B-type filter, the G-type filter, and the R-type filter may include two types of filters, respectively, and the intensity of light in six wavelength bands may be obtained by a total of six types of filters.
[0054]
FIG. 13 is a diagram illustrating an example of the spectral transmission characteristics of the six types of filters in such a case. As shown in the figure, as the six types of filters, there are a B1 filter, a B2 filter, a G1 filter, a G2 filter, an R1 filter, and an R2 filter, starting from a filter that transmits a short wavelength band. Thus, two types of filters are included as a B type filter, two types of filters are included as a G type filter, and two types of filters are included as an R type filter.
[0055]
FIG. 14 is a diagram illustrating an example of the arrangement of these six types of filters in the image sensor 3. Also in this case, the G type filter is attached to the light receiving pixels P corresponding to one area of the checkered pattern in the array of the plurality of light receiving pixels P, and the B type filter and the R type filter are applied to the other area of the checkered pattern. (The area where the G type filter is not disposed) is evenly applied to the light receiving pixels P corresponding to the area. As a result, the image sensor 3 outputs a sensor output image including a plurality of pixels respectively corresponding to any of the six colors B1, B2, G1, G2, R1, and R2, and the intensity of light in the six wavelength bands is reduced. can get.
[0056]
Also, when generating a Bayer image from the sensor output image obtained from the image sensor 3, similarly to the above, the six colors are divided into three groups corresponding to the three primary colors of RGB, and each group is divided into three colors. What is necessary is just to obtain the value of the pixel corresponding to one primary color of the Bayer image. The primary color B group is divided into B1 and B2, the primary color G group is divided into G1 and G2, and the primary color R group is divided into R1 and R2. Then, a filter mask as shown in FIG. 6 may be used to derive the G pixel value of the Bayer image, and a filter mask as shown in FIG. 12 may be used to derive the B pixel and R pixel values of the Bayer image.
[0057]
<1-3-3.8 types of filters>
Also, the B-type filter and the R-type filter each include two types of filters, and the G-type filter includes four types of filters so that a total of eight types of filters can obtain light intensities in eight wavelength bands. It may be.
[0058]
FIG. 15 is a diagram illustrating an example of the spectral transmission characteristics of the eight types of filters in such a case. As shown in the figure, as the eight types of filters, there are a B1 filter, a B2 filter, a G1 filter, a G2 filter, a G3 filter, a G4 filter, an R1 filter, and an R2 filter, starting from a filter that transmits a short wavelength band. Accordingly, two types of filters are included as the B type filter, four types of filters are included as the G type filter, and two types of filters are included as the R type filter.
[0059]
FIG. 16 is a diagram illustrating an example of the arrangement of these eight types of filters in the image sensor 3. Also in this case, the G type filter is attached to the light receiving pixels P corresponding to one of the areas of the checkered pattern in the array of the plurality of light receiving pixels P. Then, in this region, the G1 to G4 filters are respectively arranged evenly. Further, the B-type filter and the R-type filter are evenly applied to the light receiving pixels P corresponding to the other area of the checkered pattern (the area where the G-type filter is not arranged). As a result, the image sensor 3 outputs a sensor output image including a plurality of pixels respectively corresponding to any one of the eight colors B1, B2, G1, G2, G3, G4, R1, and R2. Light intensity is obtained.
[0060]
As the number of wavelength bands that can be obtained in this way increases, the calculation accuracy when calculating the spectral distribution of light from a subject or the like in a computer or the like can be improved. The number of wavelength bands that can be acquired matches the number of types of filters, but the number of types of filters may be any number as long as it is four or more. In order to obtain a highly accurate spectral distribution, it is necessary to obtain light intensities in four or more wavelength bands.
[0061]
Also, when generating a Bayer image from the sensor output image obtained from the imaging sensor 3, similarly to the above, the eight colors are divided into three groups corresponding to the three primary colors of RGB, and each group is divided into eight colors. What is necessary is just to obtain the value of the pixel corresponding to one primary color of the Bayer image. The primary color B group is divided into B1 and B2, the primary color G group is divided into G1, G2, G3 and G4, and the primary color R group is divided into R1 and R2.
[0062]
To derive the values of the B pixel and the R pixel of the Bayer image, a filter mask as shown in FIG. 12 may be used. On the other hand, for deriving the value of the G pixel of the Bayer image, for example, a filter mask shown in FIG. 17 can be used. More specifically, for example, the value of the G pixel of the Bayer image at the same position as the G1 pixel of the sensor output image is calculated using the filter mask shown in FIG. 17 for 5 × 5 pixels centered on the G1 pixel of the sensor output image. Is derived by a convolution operation using. That is, the value of the G pixel is obtained by adding the weight α to the value of the G1 pixel. ₁ , The values of the four G2 pixels, two G3 pixels, and two G4 pixels in the vicinity are weighted by α. ₂ , Α ₃ , Α ₄ Is obtained by adding the average value of the result of multiplying by. Where the weight α ₁ , Α ₂ , Α ₃ , Α ₄ Satisfies the following expression (2).
[0063]
α ₁ + Α ₂ + Α ₃ + Α ₄ = 1 ... (2)
These weights α ₁ ~ Α ₄ Can be determined by the relationship between the spectral transmission characteristics of each of the G1 to G4 filters and the spectral transmission characteristics of a general G filter. That is, α is set so that the weight of a pixel to which a filter whose spectral transmission characteristic approximates to that of the G filter is added becomes larger. ₁ ~ Α ₄ Is determined. The value of the G pixel of the Bayer image at the same position as each of the G2 pixel, G3 pixel, and G4 pixel is calculated by the weight α of the filter mask shown in FIG. ₁ ~ Α ₄ May be derived using an appropriately modified one.
[0064]
<1-3-4. Near ultraviolet or near infrared wavelength band>
Although the filters of the above embodiments all transmit light in the visible wavelength band, the light receiving pixels made of silicon have wavelengths from near ultraviolet (about 300 nm) to near infrared (about 1100 nm). Since the light can be subjected to photoelectric conversion, the plurality of types of filters may include a filter that transmits light in a near-ultraviolet or near-infrared wavelength band.
[0065]
FIG. 18 is a diagram illustrating an example of spectral transmission characteristics of four types of filters including a UV filter that transmits light in a near ultraviolet wavelength band. As shown in the figure, the four types of filters include a B filter, a G filter, and an R filter that transmit light in the visible wavelength band, and a UV filter that transmits light in the near ultraviolet wavelength band.
[0066]
FIG. 19 is a diagram illustrating an example of the arrangement of these four types of filters in the image sensor 3. UV filters are classified into filters of a type that transmits light in a shorter wavelength band than G-type filters (hereinafter, referred to as “short-wavelength type filters”). This short wavelength type filter includes two types of filters, a UV filter and a B filter. The G type filter is attached to the light receiving pixels P corresponding to one area of the checkered pattern in the array of the plurality of light receiving pixels P, and the short wavelength type filter and the R type filter are used for the other area of the checkered pattern (the G type filter). Are uniformly applied to the light-receiving pixels P corresponding to the area where is not arranged. Then, the UV filter and the B filter are evenly arranged in a region where the short wavelength type filter is arranged.
[0067]
From such an imaging sensor 3, light intensities in four wavelength bands including a near ultraviolet wavelength band and wavelength bands respectively corresponding to the three primary colors of RGB are obtained. That is, in the image sensor 3, information of the near-ultraviolet wavelength band of the light from the subject is obtained together with the color information of the subject in one exposure.
[0068]
FIG. 20 is a diagram illustrating an example of spectral transmission characteristics of four types of filters including an IR filter that transmits light in a near-infrared wavelength band. As shown in the figure, there are four types of filters, a B filter, a G filter, and an R filter that transmit light in the visible wavelength band, and an IR filter that transmits light in the near infrared wavelength band.
[0069]
FIG. 21 is a diagram showing an example of arrangement of these four types of filters in the image sensor 3. IR filters are classified as filters of a type that transmits light in a longer wavelength band than G-type filters (hereinafter, referred to as “long-wavelength type filters”). The long wavelength type filter includes two types of filters, an IR filter and an R filter. The G type filter is attached to the light receiving pixels P corresponding to one area of the checkered pattern in the arrangement of the plurality of light receiving pixels P, and the long wavelength type filter and the B type filter are used for the other area of the checkered pattern (the G type filter). Are uniformly applied to the light-receiving pixels P corresponding to the area where is not arranged. Then, the IR filter and the R filter are evenly arranged in a region where the long wavelength type filter is arranged.
[0070]
From such an image sensor 3, light intensities in four wavelength bands including a near-infrared wavelength band and wavelength bands respectively corresponding to the three primary colors of RGB are obtained. That is, in the image sensor 3, information of the near-infrared wavelength band of the light from the subject is obtained together with the color information of the subject in one exposure.
[0071]
The near-ultraviolet wavelength band is a wavelength band in which insects and the like have visibility, so that if light intensity in the near-ultraviolet wavelength band is obtained, various analyzes can be performed on subjects such as plants. Also, light in the near-ultraviolet wavelength band is often scattered by the subject, and is therefore suitable for detecting wrinkles and spots on the face. On the other hand, light in the near-infrared wavelength band is less likely to be scattered or absorbed by the subject and easily penetrates the living body. It can be performed.
[0072]
<1-3-5. Reproduction of color matching function>
If there are four types of filters as filters applied to the light receiving pixels of the image sensor 3, it is possible to reproduce a color matching function representing the sensitivity of the human eye. FIG. 22 is a diagram illustrating a color matching function. As shown in the figure, the color matching function is represented by three curves of r, g, and b. The curve r has a portion rn that protrudes to the negative side.
[0073]
Therefore, in order to reproduce such a color matching function, as shown in FIG. 23, an R1 filter having a spectral transmission characteristic that matches the characteristic of the positive portion of the curve r of the color matching function, and a curve of the color matching function An R2 filter having a spectral transmission characteristic whose polarity is opposite to the characteristic of the negative part rn of r, a G filter having a spectral transmission characteristic matching the characteristic of the curve g of the color matching function, and a characteristic of a curve b of the color matching function B filters having the same spectral transmission characteristics are set. These four types of filters may be arranged on the image sensor 3 as in FIG. 11, for example.
[0074]
The value obtained from the light-receiving pixel with the G filter is the primary color G value, the value obtained from the light-receiving pixel with the B filter is the primary color B value, and the value obtained from the light-receiving pixel with the R1 filter is obtained. The value obtained by subtracting the value obtained by the light receiving pixel provided with the R2 filter from the value is defined as the value of the primary color R. By doing so, the color matching function is substantially reproduced, and the values of the three primary colors corresponding to the values felt by the human eye can be obtained.
[0075]
<1-3-6. Honeycomb arrangement>
Further, the imaging sensor 3 may be a honeycomb sensor in which the arrangement of the light receiving pixels is a honeycomb arrangement. In the honeycomb sensor, light receiving pixels are arranged in a honeycomb shape (adjacent light receiving pixels are shifted by half a pixel). FIG. 24 is a diagram illustrating an example of the arrangement of four types of filters, a B filter, a G1 filter, a G2 filter, and an R filter, with respect to the honeycomb sensor 3a. The arrangement of the filter shown in FIG. 24 corresponds to the arrangement of the filter shown in FIG. 4 rotated by 45 degrees.
[0076]
In such a honeycomb sensor 3a, a wiring portion between the light receiving pixels P is not required, so that a space is created, and each light receiving pixel can be enlarged as compared with the imaging sensor 3 described above. It becomes.
[0077]
When the types of filters having different wavelength bands of transmitted light are increased, the bandwidth of light transmitted by each filter becomes narrower, so that the sensitivity of a light receiving pixel provided with such a filter may be reduced. There is. When the honeycomb sensor 3a is employed as the imaging sensor 3, the size of each light receiving pixel can be increased, so that such a decrease in the sensitivity of the light receiving pixel can be prevented.
[0078]
<2. Second Embodiment>
Next, a second embodiment of the present invention will be described. In the first embodiment, the filter is arranged for each light receiving pixel of the image sensor, but in the present embodiment, the filter is arranged for each unit including a plurality of light receiving pixels. . The configuration of the imaging device 1 of the present embodiment is almost the same as that shown in FIG. 2, but the configuration of the imaging unit 40 is different.
[0079]
FIG. 25 is an exploded perspective view illustrating a schematic configuration of the imaging unit 40 according to the present embodiment. Light from the subject enters from the upper part of FIG. As shown in the drawing, the imaging unit 40 includes a microlens array 20 having a total of four horizontal 2 × vertical 2 microlenses 21 arranged in a plane, and a total of four horizontal 2 × vertical 2 filters 61. A filter array 60 having a plurality of grids 71 arranged horizontally and 2 × vertically 2 arranged in a plane, and an image sensor 30 having a plurality of light receiving pixels arranged in a plane. Are provided in this order from the light incident side.
[0080]
The light-receiving surface composed of a plurality of light-receiving pixels of the image sensor 30 is divided into a total of four image areas 31 (horizontal 2 × vertical 2), and four micro lenses 21 are provided for each of the four image areas 31. Is formed. Further, the four filters 61 are arranged corresponding to the four microlenses 21, respectively, and transmit only light of a specific wavelength band among the light that has passed through the microlenses 21. Furthermore, the four gratings 71 are arranged corresponding to each of the four microlenses 21 so as to prevent interference between light passing through one microlens 21 and light passing through the other microlenses 21. Function.
[0081]
That is, one microlens 21, one filter 61, and one grating 71 are associated with one imaging region 31, and these are one unit as shown by a dashed line in the drawing. U. The light image of the subject formed in the image forming area 31 in the same unit U by the microlens 21 is photoelectrically converted by a plurality of light receiving pixels included in the image forming area 31 into one image. Therefore, an image of one subject is obtained for each unit U, and images of four subjects are obtained by the entire imaging sensor 30. The acquired four images are converted into digital signals in the A / D converter 41, subjected to predetermined processing in the image processing unit 42 as necessary, and stored in the memory card 91 or transmitted to the external device 92. .
[0082]
Since the focal length of the microlens is shorter than the focal length of a normal lens, the imaging device 1 can be downsized by configuring the imaging unit 40 according to the present embodiment. Further, when a plurality of types of filters are formed for each light receiving pixel as in the first embodiment, the number of manufacturing steps of the image sensor may increase, and the manufacturing cost of the imaging apparatus may increase. According to the configuration of the imaging unit 40 of the embodiment, the filter may be arranged for each unit, so that the size of the filter can be relatively large. This makes it relatively easy to manufacture the filter, thereby reducing the manufacturing cost of the imaging device.
[0083]
The wavelength band of the light transmitted by the filter in each unit U corresponds to the color of a pixel in an image obtained in the same unit U. The four filters 61 forming the filter array 60 have different wavelength bands of transmitted light. That is, in this embodiment, four types of filters are employed. As these four types of filters, a B filter, a G1 filter, a G2 filter, and an R filter having the spectral transmission characteristics shown in FIG. 3 are used. FIG. 26 is a diagram illustrating an example of the arrangement of four types of filters 61 in each unit U in the filter array 60.
[0084]
With such a configuration, the imaging apparatus 1 can obtain four images having different colors for the same subject by one exposure. That is, the imaging apparatus 1 can obtain the light intensities of the four wavelength bands by one exposure. Furthermore, the imaging device 1 can obtain the intensity of four wavelength bands with respect to light from the same point of the subject. In other words, four color component images B, G1, G2, and R can be obtained without performing interpolation of the missing pixel value, and no false color or moire caused by the interpolation of the missing pixel value occurs.
[0085]
In the present embodiment, the arrangement of the filters in filter array 60 is not particularly limited, and four types of filters may be arranged in any manner. The four types of filters may transmit any wavelength band as long as the wavelength bands of transmitted light are different from each other.
[0086]
For example, the four types of filters may include a filter that transmits light in the near ultraviolet or near infrared wavelength band. FIG. 27 is a diagram illustrating an arrangement example of the filter 61 in the filter array 60 when the UV filter, the B filter, the G filter, and the R filter having the spectral transmission characteristics illustrated in FIG. 19 are employed. FIG. 28 is a diagram illustrating an example of the arrangement of the filters 61 in the filter array 60 when the B filter, the G filter, the R filter, and the IR filter having the spectral transmission characteristics illustrated in FIG. 20 are employed. With such a configuration, the image sensor 30 can obtain information on the near-ultraviolet or near-infrared wavelength band of the light from the subject together with the color information of the subject in one exposure, as described above. Various analyzes can be performed on the subject.
[0087]
Further, the four types of filters may include a transparent filter (hereinafter, referred to as “W filter”) that transmits light in all wavelength bands. FIG. 29 is a diagram illustrating an example of the arrangement of the filters 61 in the filter array 60 in such a case. With such a configuration, the image sensor 30 can obtain pure color component information about light from the subject together with color information of the subject in one exposure. At the same time, since the W filter transmits light in all wavelength bands, the sensitivity of the light receiving pixels included in the unit U in which the W filter is arranged can be improved. Therefore, the imaging apparatus 1 employing the filter array 60 shown in FIG. 29 obtains an image of the subject by using the unit U in which the W filter is arranged, even when the brightness of the subject is extremely low. be able to. For this reason, the imaging device 1 can be used for a surveillance camera, a vehicle-mounted camera, or the like that is required to be used day and night.
[0088]
Further, the number of units U is not limited to four, and may be any number as long as it is four or more. FIG. 30 is a diagram illustrating an example of the arrangement of the filters 61 in the filter array 60 when the number of units U is six. Even in such a case, the six filters 61 have different wavelength bands of transmitted light, and in the example of FIG. 30, six types of filters having the spectral transmission characteristics shown in FIG. 13 are employed. . When the number of units U is increased in this manner, the number of wavelength bands that can be obtained by one exposure can be increased, so that the calculation accuracy when calculating the spectral distribution of light from a subject in a computer or the like can be improved. Can be improved.
[0089]
In addition, as shown in FIG. 31, the visible wavelength band is divided into, for example, 16, and 16 types of filters (F1 to F16 filters) capable of transmitting light in the respective divided wavelength bands are prepared, as shown in FIG. Alternatively, F1 to F16 filters may be arranged in 16 units U, respectively. In the imaging apparatus 1 employing the filter array 60 shown in FIG. 32, light intensity in 16 wavelength bands can be obtained by one exposure, and can be used as a highly accurate spectrophotometer. When the imaging device 1 is used as a spectrocolorimeter, it is preferable that the number of wavelength bands that can be obtained by one exposure is 16 or more.
[0090]
<3. Third Embodiment>
Next, a third embodiment of the present invention will be described. Although the configuration of the imaging apparatus 1 of the present embodiment is almost the same as that of the second embodiment, in the present embodiment, a plurality of images acquired for each unit U are rearranged and one An arrangement image is generated.
[0091]
FIG. 33 is an exploded perspective view illustrating a schematic configuration of the imaging unit 40 according to the present embodiment. The imaging unit 40 according to the present embodiment includes a microlens array 20 having a plurality of microlenses 21 arranged in a plane and a plurality of filters 61 arranged in a plane, almost in the same manner as the second embodiment. Filter array 60, a partition 70 having a plurality of grids 71 arranged in a plane, and an image sensor 30 having a plurality of light receiving pixels arranged in a plane from the light incident side in this order. ing. The light receiving surface of the image sensor 30 including a plurality of light receiving pixels is divided into a plurality of imaging regions 31. One micro lens 21, one filter 61, and one grating 71 are associated with this imaging region 31, and these form one unit U. However, the imaging unit 40 according to the present embodiment includes a large number of units U, specifically, a plurality of units U arranged M in the horizontal direction and N in the straight direction.
[0092]
FIG. 34 is a diagram showing a relationship between a plurality of units U and a plurality of light receiving pixels P. The upper part shows the arrangement of the units U (that is, the imaging regions 31) in the image sensor 30, and the lower part shows the arrangement in one unit U. 2 shows the arrangement of the light receiving pixels P in FIG. As shown in the drawing, the light receiving surface of the image sensor 30 is divided into horizontal M units × vertical N units U, and one unit U includes m horizontal × n vertical light receiving pixels P.
[0093]
In each unit U, an image of a subject composed of horizontal m × vertical n pixels (hereinafter, referred to as “unit image”) is obtained. The unit image is converted into a digital signal by the A / D converter 41 and then input to the image processing unit 42. Then, the image processing unit 42 rearranges the pixels included in the unit image obtained in each unit U, and generates a rearranged image including N × n pixels in the horizontal direction and M × m pixels in the vertical direction.
[0094]
Since there is a parallax between the plurality of microlenses 21 forming the microlens array 20, light images of the subject are formed at slightly different positions between the imaging regions 31 due to the parallax. Therefore, the positions of the included subject images are slightly different between the plurality of unit images. In the present embodiment, by utilizing such a difference in the position of the subject image between the unit images, the pixels of the plurality of unit images are rearranged to obtain a single high-resolution rearranged image. ing.
[0095]
Here, the horizontal coordinate of each unit U on the image sensor 30 is I, the vertical coordinate is J, the horizontal coordinate of the light receiving pixel P in the unit U is i, and the vertical coordinate is j, The acquired coordinates (I, J) of the unit U are used to identify each unit image, and the coordinates (i, j) of the acquired light receiving pixel P are used to identify each pixel in the unit image. . In FIG. 34, each coordinate (I, J) is shown in each unit U, and each coordinate (i, j) is shown in each light receiving pixel P.
[0096]
Then, assuming that the horizontal coordinate of each pixel in the rearranged image is x and the vertical coordinate is y, the coordinates at which the pixel (i, j) in the unit image (I, J) is to be arranged in the rearrangement processing. (X, y) is expressed by the following equations (3) and (4).
[0097]
x = M (i-1) + I (3)
y = N (j-1) + J (4)
In the rearrangement process, the coordinates (x, y) obtained by Expressions (3) and (4) are set as target coordinates of each pixel, and all pixels of all unit images are arranged at the respective target coordinates. An arrangement image is generated.
[0098]
In the present embodiment, four types of filters having different wavelength bands of transmitted light are employed as the plurality of filters 61 forming the filter array 60. As these four types of filters, a B filter, a G1 filter, a G2 filter, and an R filter having the spectral transmission characteristics shown in FIG. 3 are used.
[0099]
FIG. 35 is a diagram showing an example of the arrangement of four types of filters in each unit U (that is, each microlens 21) in the filter array 60. As shown in FIG. 35, the G type filters (G1 filter and G2 filter) are arranged in the unit U corresponding to one area of the checkered pattern in the arrangement of the units U. Then, the G1 filter and the G2 filter are alternately arranged in the vertical direction. On the other hand, the B filter and the R filter are arranged in the unit U corresponding to the other area of the checkered pattern (the area where the G type filter is not arranged). The B filters and the R filters are alternately arranged in the vertical direction. That is, the arrangement of the filters is the same as the arrangement of the filters shown in FIG.
[0100]
The color of the pixel in each unit image corresponds to the wavelength band of light transmitted by the filter 61 in the obtained unit U. That is, a unit image of only B pixels is provided from the unit U in which the B filter is provided, a unit image of only G1 pixels is provided from the unit U in which the G1 filter is provided, and only a G2 pixel is provided from the unit U in which the G2 filter is provided. From the unit U in which the unit image and the R filter are arranged, a unit image of only R pixels is obtained. Therefore, also in the imaging device 1 of the present embodiment, it is possible to obtain light intensities of four wavelength bands by one exposure.
[0101]
When the above rearrangement processing is performed on these unit images, the arrangement of the colors corresponding to the pixels in the rearranged image becomes the same as the arrangement of the four types of filters in the filter array 60. Therefore, when the filter array 60 shown in FIG. 35 is adopted, an image equivalent to the sensor output image obtained from the imaging sensor 3 shown in FIG. 4 is obtained as a rearranged image. From this, the same image processing as that of the first embodiment can be performed on this rearranged image.
[0102]
In the present embodiment, the number of types of filters may be any number as long as it is four or more, and the filter may transmit any wavelength band. Good. Therefore, as the arrangement of the filters in the filter array 60, any of the arrangements shown in FIG. 10, FIG. 11, FIG. 14, FIG. 16, FIG. 19, and FIG.
[0103]
◎ Note that the specific embodiments described above include inventions having the following configurations.
[0104]
(1) In the imaging device according to claim 5,
Each of the plurality of filters is a first type filter that transmits light in a green wavelength band, a second type filter that transmits light in a wavelength band shorter than the first type filter, and longer than the first type filter. Divided into any of the third type filters that transmit light in the wavelength band,
The first type filter is disposed with respect to the microlenses corresponding to one area of a checkered pattern in the array of the plurality of microlenses,
The imaging apparatus according to claim 1, wherein the second and third type filters are respectively arranged evenly with respect to the microlenses corresponding to the other area of the checkered pattern.
[0105]
According to this, since the human eye has high sensitivity to light in the green wavelength band, the first type filter that transmits light in the green wavelength band is arranged in one of the checkered areas, and By increasing the number of arrangements compared to the third type filter, it is possible to enhance the resolution of the acquired image.
[0106]
(2) In the imaging device according to claim 1 or (1),
The second type filter includes a filter that transmits a near ultraviolet wavelength band.
[0107]
According to this, since the intensity of light in the near ultraviolet wavelength band can be obtained, various analyzes can be performed on the subject.
[0108]
(3) In the imaging device according to claim 1 or (1),
The third type filter includes a filter that transmits a near-infrared wavelength band.
[0109]
According to this, since the intensity of light in the near-infrared wavelength band can be obtained, various analyzes can be performed on the subject.
[0110]
(4) In the imaging device according to claim 4,
The imaging apparatus according to claim 1, wherein the predetermined number of filters includes a filter that transmits near-ultraviolet rays or near-infrared rays.
[0111]
According to this, since the intensity of light in the near ultraviolet or near infrared wavelength band can be obtained, various analyzes can be performed on the subject.
[0112]
【The invention's effect】
As described above, according to the first aspect of the invention, it is possible to obtain light intensities of four or more wavelength bands by one exposure. In addition, since the human eye has high sensitivity to light in the green wavelength band, the first type filter that transmits light in the green wavelength band is arranged in one of the checkered areas, and the second and third types of filters are arranged. By increasing the number of arrangements compared to the filters, it is possible to enhance the resolution of the acquired image.
[0113]
According to the second aspect of the present invention, a value of a pixel corresponding to one primary color is derived for each of three groups corresponding to three primary colors, so that a plurality of pixels corresponding to any one of four or more types of colors are provided. Can be used for display on a general color display device and printing on a color printing device.
[0114]
According to the third aspect of the present invention, a third image composed of four or more second images is generated by compressing a third image composed of three second images to generate one compressed image. The data amount of the compressed image can be reduced as compared with the case where one compressed image is generated by compression.
[0115]
According to the fourth aspect of the present invention, the intensity of light in four or more wavelength bands from the same point from the subject can be obtained by one exposure. Further, the size of the imaging device can be reduced.
[0116]
According to the fifth aspect of the present invention, it is possible to obtain light intensities of four or more wavelength bands by one exposure. Further, the size of the imaging device can be reduced.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating an imaging sensor in which a filter array is a Bayer array.
FIG. 2 is a diagram illustrating a basic configuration of an imaging device.
FIG. 3 is a diagram illustrating an example of a spectral transmission characteristic of a filter.
FIG. 4 is a diagram illustrating an example of an arrangement of filters.
FIG. 5 is a diagram illustrating a method of generating a normal image from a sensor output image.
FIG. 6 is a diagram illustrating an example of a filter mask used for calculation.
FIG. 7 is a diagram illustrating an example of a filter mask used for calculation.
FIG. 8 is a diagram illustrating an example of a filter mask used for calculation.
FIG. 9 is a diagram illustrating a method of generating three color component images from a sensor output image.
FIG. 10 is a diagram illustrating an example of an arrangement of filters.
FIG. 11 is a diagram illustrating an example of an arrangement of filters.
FIG. 12 is a diagram illustrating an example of a filter mask used for calculation.
FIG. 13 is a diagram illustrating an example of a spectral transmission characteristic of a filter.
FIG. 14 is a diagram illustrating an example of an arrangement of filters.
FIG. 15 is a diagram illustrating an example of a spectral transmission characteristic of a filter.
FIG. 16 is a diagram illustrating an example of an arrangement of filters.
FIG. 17 is a diagram illustrating an example of a filter mask used for calculation.
FIG. 18 is a diagram illustrating an example of a spectral transmission characteristic of a filter.
FIG. 19 is a diagram illustrating an example of an arrangement of filters.
FIG. 20 is a diagram illustrating an example of spectral transmission characteristics of a filter.
FIG. 21 is a diagram illustrating an example of an arrangement of filters.
FIG. 22 is a diagram illustrating a color matching function.
FIG. 23 is a diagram illustrating an example of a spectral transmission characteristic of a filter.
FIG. 24 is a diagram illustrating an example of an arrangement of a filter with respect to a honeycomb sensor.
FIG. 25 is a diagram illustrating a schematic configuration of an imaging unit according to the second embodiment.
FIG. 26 is a diagram illustrating an example of the arrangement of filters in a filter array.
FIG. 27 is a diagram illustrating an example of an arrangement of filters in a filter array.
FIG. 28 is a diagram illustrating an example of an arrangement of filters in a filter array.
FIG. 29 is a diagram illustrating an example of an arrangement of filters in a filter array.
FIG. 30 is a diagram illustrating an example of an arrangement of filters in a filter array.
FIG. 31 is a diagram illustrating an example of a spectral transmission characteristic of a filter.
FIG. 32 is a diagram showing an example of the arrangement of filters in a filter array.
FIG. 33 is a diagram illustrating a schematic configuration of an imaging unit according to a third embodiment.
FIG. 34 is a diagram showing a relationship between a plurality of units and a plurality of light receiving pixels.
FIG. 35 is a diagram illustrating an example of the arrangement of filters in a filter array.
[Explanation of symbols]
1 Imaging device
2 lens
3 Image sensor
20 micro lens array
30 Image sensor
60 filter array
70 Partition
U unit
P light receiving pixel

Claims (5)

An imaging device,
A first type filter transmitting light in a green wavelength band, a second type filter transmitting light in a wavelength band shorter than the first type filter, and transmitting light in a wavelength band longer than the first type filter. An image sensor having a plurality of light receiving pixels to which any of the third type filters are respectively arranged in a two-dimensional array;
With
The first type filter is attached to the light receiving pixels corresponding to one area of a checkered pattern in the array of the plurality of light receiving pixels,
The second and third type filters are equally applied to the light receiving pixels corresponding to the other area of the checkered pattern, respectively.
At least one of the first to third type filters includes two or more types of filters having different wavelength bands of transmitted light. The imaging device according to claim 1,
The image sensor outputs an image having a plurality of pixels respectively corresponding to any of four or more colors,
Each of the four or more colors is classified into one of three groups corresponding to three primary colors.
Deriving means for deriving, for each group, a value of a pixel corresponding to one primary color from a value of a pixel corresponding to a color included in the group;
An imaging device, further comprising: The imaging device according to claim 1,
The image sensor outputs a first image having a plurality of pixels corresponding to any of four or more colors,
Each of the four or more colors is classified into one of three groups corresponding to three primary colors.
Generating means for generating, for each group, one second image from values of pixels corresponding to colors included in the group;
Compression means for compressing a third image composed of the three generated second images to generate one compressed image;
An imaging device, further comprising: An imaging device,
An image sensor having a predetermined number of four or more image forming regions composed of a plurality of light receiving pixels arranged in a plane,
A microlens array having the predetermined number of microlenses for forming a light image of a subject on the predetermined number of image formation areas in a plane array,
A filter array having the predetermined number of filters respectively corresponding to the predetermined number of microlenses arranged in a plane,
With
The imaging apparatus according to claim 1, wherein the predetermined number of filters have different wavelength bands of transmitted light. An imaging device,
An imaging sensor having a plurality of imaging regions consisting of a plurality of light receiving pixels arranged in a plane,
A microlens array having a plurality of microlenses that respectively form a plurality of microlenses for forming light images of a subject on the plurality of imaging regions,
A filter array having a plurality of filters respectively corresponding to the plurality of microlenses arranged in a plane,
A rearrangement unit configured to rearrange pixels included in an image of a subject obtained in each of the plurality of imaging regions to generate one rearranged image;
With
The image pickup apparatus, wherein the plurality of filters include four or more types having different wavelength bands of transmitted light.

Images