JP5067885B2 - Image input device and personal authentication device - Google Patents

Description

  The present invention relates to a personal authentication device that performs personal authentication using information on a subject inside a living body, and an image input device suitable as a subject image input unit of this type of personal authentication device.

  In recent years, personal recognition devices have been installed in various information devices such as mobile phones and laptop computers. As information devices are becoming smaller and lighter, personal authentication devices mounted on the information devices are required to be further downsized.

  For example, Patent Documents 1 and 2 describe a personal authentication device using a finger vein pattern. These personal authentication devices use a monocular optical system, and subject distance and imaging distance are limited based on the optical imaging relationship, so there is a limit to downsizing and thinning.

  Patent Document 3 describes an image input device in which a micro lens array, a color filter, a light receiving element array, and the like are combined.

JP 2004-272821 A Japanese Patent Laying-Open No. 2005-092375 Japanese Patent No. 3705766 Japanese Patent No. 3773563

  In the case of a personal recognition device that uses information about a subject inside a finger such as a finger vein pattern, it is necessary to bring the finger into close contact with the image input unit of the device. An image input device used for a personal authentication device has a special feature of capturing an image of a very close subject.

  In addition, when using a finger fingerprint, necessary fingerprint pattern information can be acquired by imaging a narrow area of the finger. However, when using a finger vein, in order to acquire the necessary vein pattern information, It is necessary to image a considerably wide area. In order to observe an image of a subject in a very close state, the same magnification imaging by a microlens array is generally used. This is necessary and causes an increase in device cost. This is because the image pickup area of a general-purpose inexpensive CMOS image pickup element or CCD image pickup element is limited, and thus it is necessary to specially manufacture an image pickup element having a large image pickup area. Therefore, it is desirable that an inexpensive general-purpose image sensor can be used.

  The present invention has been made in view of the above points, and an object of the present invention is to be able to input an image of a relatively wide area of a very close subject using an inexpensive general-purpose image sensor. An object of the present invention is to provide a thin and small image input device suitable as an image input means of an authentication device and to provide a personal authentication device using such an image input device.

The invention according to claim 1 is an image input device for inputting an image of a subject inside a living body,
A lens array in which a plurality of lenses are arrayed;
A light shielding member for preventing crosstalk on the image plane of light rays passing through each lens of the lens array;
A flat plate member that contacts the living body and defines the position of the living body in the lens optical axis direction of the lens array;
An imaging that has an imaging surface and captures a compound eye image that is a set of reduced images of a subject inside a living body, the position of which is defined by the flat plate member, which is substantially imaged by a plurality of lenses of the lens array on the imaging surface. Means,
Processing means for reconstructing a single image from a compound eye image captured by the imaging means,
The light shielding member is disposed between the lens array and the imaging surface and has a rectangular hole corresponding to each lens of the lens array,
The size of the rectangular hole is g × h (where g and h are the length of each side of the rectangular hole cross section in a plane substantially orthogonal to the lens optical axis of the lens array), The distance from the plane to the subject is a, the distance from the substantially main plane of each lens of the lens array to the imaging surface is b, and the size of the imaging surface is x × y (where x and y are rectangles) The length of each side of the imaging surface), and the size of the field of view at the position of the subject is u × v (where u and v are the length of each side of the field of view that is a rectangle)
Relationship of g × a / b + x ≧ u and h × a / b + y ≧ v
Or
Relationship of h × a / b + y ≧ u and g × a / b + x ≧ v
It is characterized by satisfying.

According to a second aspect of the present invention, in the image input device according to the first aspect of the present invention, the height of the light shielding member in the lens optical axis direction is substantially matched with the distance b.

According to a third aspect of the present invention, in the image input device according to the first or second aspect of the present invention , a gas or liquid layer is interposed between the flat plate member and the lens array.

According to a fourth aspect of the present invention, in the image input device according to the first or second aspect of the present invention, a vacuum layer is interposed between the flat plate member and the lens lens array.

According to a fifth aspect of the present invention, in the image input device according to any one of the first to fourth aspects, the lens array is paired with a surface facing the subject and a surface facing the imaging surface. The lens formed on the surface facing the subject and the lens formed on the surface facing the imaging surface each have a negative power in the direction of the subject. Features.

A sixth aspect of the present invention is the image input device according to any one of the first to fifth aspects, further comprising a light source for irradiating light to a living body contacting the flat plate member, the flat plate The member has a transmittance with respect to light irradiated by the light source.

The invention described in claim 7
The image input device according to any one of claims 1 to 6 ,
Authentication processing means for performing authentication processing using a subject image input by the image input device;
It is a personal authentication apparatus characterized by having.

The image input apparatus according to the first aspect of the present invention can capture a subject area larger than the imaging surface of the imaging means and input the image of a very close subject using a thin optical system. For this reason, for example, even in the application of inputting a finger vein pattern image for personal authentication, an inexpensive general-purpose image sensor having an imaging surface size that is not so large can be used as an imaging unit, and the apparatus cost can be reduced accordingly. .

In the image input device according to the second aspect of the present invention, the image plane distance b (distance from the substantially main plane of each lens of the lens array to the imaging surface) is defined by the height of the light shielding member in the lens optical axis direction. It is also possible to prevent crosstalk of light rays in a plane parallel to the imaging surface.

In the image input device according to the third or fourth aspect of the present invention, by providing a gas or liquid layer such as air or a vacuum layer provided between the lens array and the transparent flat plate, a space corresponding to these layers is formed. Compared to the case of filling with a flat plate member or lens array, the volume of the flat plate member or lens array can be reduced to reduce the material cost thereof, and the field of view is widened by refraction at the boundary between the layer, the lens array, and the flat plate member. be able to.

In the image input device according to any one of the first to fourth aspects of the invention, if the back focus of the lens is shortened, a wide field of view can be imaged due to a decrease in optical magnification. It is steep and difficult to process. In the image input device according to the fifth aspect of the invention, since the power of the lens of the lens array is distributed to two, the power of one lens surface can be reduced, and the lens shape that is relatively easy to process in resin molding or the like Thus, a wide field of view can be secured. Also, by providing negative power in the subject direction on both sides, it is possible to suppress differences in image formation position and spread due to the angle at which light rays enter the lens, thus suppressing image distortion even when the field of view is expanded. An image can be input.

The image input device according to the sixth aspect of the present invention has a higher contrast and a higher S / N ratio regardless of the usage environment by imaging the subject in a state where the living body is irradiated with light such as near infrared light. A high subject image can be input.

According to the seventh aspect of the invention, it is possible to realize an inexpensive, thin, and small personal authentication device that performs high-precision personal authentication using a blood vessel pattern image such as a finger.

  As described above, according to the present invention, a thin and small-sized image input device suitable as an image input unit of a personal authentication device can be realized at low cost, and a thin and highly accurate authentication using such an image input device. It is possible to obtain an effect that a small personal authentication device can be realized at low cost.

  Embodiments of the present invention will be described with reference to the drawings. The image input apparatus of the present invention is suitable for the purpose of inputting an image of a subject inside a living body, but here, it is assumed that a blood vessel inside a finger is imaged as a subject and the input image is used for personal authentication. explain.

  FIG. 1 is an apparatus configuration diagram for explaining an embodiment of the present invention. In FIG. 1, reference numeral 1 denotes a human finger, schematically showing a state viewed from the fingertip direction. Reference numeral 2 schematically represents a blood vessel existing inside the finger 1, and this is a subject.

  Reference numeral 3 denotes a lens array for forming a subject image, in which a plurality of aspherical single lenses are two-dimensionally arranged in a plane substantially orthogonal to the optical axis. In addition, as each lens which comprises the lens array 3, a single side | surface or both surfaces may be a spherical lens, an aspherical lens may be used for both surfaces, and a diffractive lens like a Fresnel lens can also be used. Reference numeral 4 denotes a light shielding member for preventing crosstalk on the image plane of light rays passing through each lens of the lens array 3 and suppressing noise light such as ghost and flare.

  2a represents an observation area formed by one lens constituting the lens array 3, and 2b represents an observation area shared by adjacent lenses in the observation area (the observation area and the common area will be described later).

  FIG. 2 is a view of the imaging optical system including the lens array 3 and the light shielding member 4 as viewed from above, that is, from the subject side. As shown in FIG. 2, the light shielding member 4 is a flat plate with a rectangular hole, and the length of one side of the rectangular hole substantially matches the effective diameter of each lens of the lens array 3 or the lens It is larger than the diameter. The rectangular hole extends to the imaging surface, and the size of the rectangular hole is the size of an image (single-eye image) by one lens.

  Since each lens constituting the lens array 3 is circular, the surface on the imaging side of the lens array 3 is used to prevent light from the subject from entering the imaging surface through a gap between the effective range of the lens and the rectangular hole. A film 5 for reflecting light incident from the object side is formed on the portion other than the effective diameter of the lens. This film 5 can be formed by depositing a metal thin film (for example, a chromium thin film) on the lens array or printing an opaque resin on the lens array. The light shielding member 4 and the film 5 remove flare light caused by crosstalk between adjacent lenses.

  The lens array 3 is manufactured by using a transparent resin or glass as a material and processing methods such as a reflow method, an area gradation mask method, and a polishing method, or a molding method using a mold manufactured by these processing methods. be able to.

  The light shielding member 4 is made by making a hole in a flat plate made of resin, glass, metal or the like by etching, drilling, laser processing or the like. When etching or laser processing is used, the height of the light shielding member 4 in the lens optical axis direction may be limited. In such a case, the thickness of the light shielding member is reduced by overlapping the lens optical axis direction and bonding. You just need to secure it. By using an opaque material for the light shielding member 4 or coating a transparent material, light transmission and reflection can be suppressed.

  Reference numeral 6 denotes an image pickup device for picking up a compound eye image of a subject formed by the lens array 3, and the pixels 6a are two-dimensionally arranged in an array. Here, it is assumed that a general CMOS image sensor is used as the image sensor 6. However, a CCD image sensor or the like can be used as the image sensor 6.

  Some CMOS image sensors and the like are provided with a cover glass for protecting the imaging surface. Here, it is assumed that the image sensor 6 is not provided with a cover glass. An image sensor provided with a cover glass may be used. In that case, it is necessary to design the shape and position of the lens array 3 in consideration of the influence of light refraction by the cover glass. An optical low-pass filter for preventing aliasing is provided in the vicinity of the imaging surface, and there is also an imaging device, but here, it is assumed that no low-pass filter is provided for super-resolution processing described later.

  In order to prevent damage to the imaging surface due to contact with the light shielding member 4, the light shielding member 4 is held by the housing 7 together with the lens array 3 while being slightly lifted from the imaging surface. However, when the imaging surface is protected by the cover glass, the light shielding member 4 may be disposed in contact with the cover glass surface.

  In the configuration of FIG. 1, the space where the imaging surface of the imaging device 6 exists is sealed by the lens array 3 and the housing 7, so that dust or the like does not enter from the outside and adhere to the imaging surface. Since the light shielding member 4 is slightly lifted from the imaging surface, in a plane parallel to the imaging surface so that light does not pass through the space between the light shielding member bottom surface and the imaging surface and crosstalk of light rays does not occur. The lens pitch of the lens array 3 is set.

  An LED 8 for irradiating the finger 1 with light is provided so that a clear subject image with high contrast can be captured. As the LED 8, an LED that emits light having a wavelength in the near infrared band having a low absorption rate for a living body (or light having a wavelength having a transmittance in a living tissue such as a red band) is used. Here, as seen in FIG. 2, a plurality of LEDs 8 are installed so as to surround the periphery of the lens array.

  9 is a drive part of LED8. The driving unit 9 may cause the LED 8 to emit / extinguish light in conjunction with the ON / OFF of the power supply of the apparatus. However, it is not desirable from the viewpoint of safety that the LED 8 always emits light when the apparatus power is on. Therefore, it is desirable that a stitch for detecting the finger 1 is provided and that the LED 8 is caused to emit light by the drive unit 9 only during a period when the finger is detected by this switch.

  Reference numeral 10 denotes a flat plate for bringing the finger 1 into contact and defining the position in the lens optical axis direction. Here, the flat plate 10 is a transparent member made of a transparent resin material, but is not limited thereto. The flat plate 10 should just be a flat plate member which consists of materials, such as glass and resin, which have the transmittance | permeability with respect to the near-infrared light etc. which are irradiated to a finger | toe at least. On the lower surface of the flat plate 10, that is, the surface facing the imaging surface, an optical thin film that functions as a band-pass filter that passes light in the vicinity of the wavelength of light irradiated on the finger 1 is deposited. With such a configuration, it is possible to capture a subject image using only light that has passed through the inside of the finger and obtain a high-quality image with high contrast.

  Here, an air layer 11 is interposed between the flat plate 10 and the lens array 3. The air layer 11 may be a layer filled with another gas or liquid, or a vacuum layer. Further, the flat plate 10 and the lens array 3 can be brought into close contact with each other without interposing the air layer 11. When the air layer 11 is provided between the flat plate 10 and the lens array 3, depending on the thickness of the transparent flat plate 10, the thickness of the air layer 11, and the thickness of the lens array 3, the main plane of each lens constituting the lens array 3 The distance to the subject, that is, the subject distance is defined. When the flat plate 10 and the lens array 3 are brought into close contact with each other, the subject distance is defined by the thickness of the flat plate 10 and the thickness of the lens array 3. The same material as the lens array 3 may be used for the flat plate 10.

  As shown in FIG. 1, a distance from a substantially main plane of each lens constituting the lens array 3 to a subject is defined as a (subject distance), and a distance from a substantially main plane of each lens constituting the lens array 3 to the imaging surface. If the distance is b (image plane distance), the optical magnification of the lens is determined by the ratio of a and b.

  A person who wants to receive authentication or a person who wants to register personal data brings the finger 1 into contact with the flat plate 10 and the housing 7. This finger 1 is irradiated with near infrared light emitted from the LED 8. Although the irradiated near-infrared light is transmitted and scattered inside the living body, it is known that the near-infrared light has a transmittance to the living body but is absorbed by reduced hemoglobin in blood. Therefore, a blood vessel pattern image having a dark blood vessel portion is formed on the imaging surface of the imaging device 6 as a compound eye image by the lens array 3.

  In FIG. 1, the light emitted from the LED 8 is emitted directly upward, but toward the center of the finger 1 or toward the side of the finger 1 so as to obtain a blood vessel pattern image with appropriate contrast. May be emitted obliquely upward. In order to efficiently irradiate the finger 1 with light emitted from the LED 8, a lens may be provided in the optical path between the finger 1 and the LED 8 as shown in FIG.

  Further, depending on the use environment of the apparatus, it is not impossible to take an image using an external light source without providing the LED 8 as a light source for illumination. However, if near-infrared light is irradiated by the LED 8, it is desirable to provide the LED 8 because a clear blood vessel pattern image with high contrast can be taken regardless of the use environment.

  An aspect of a compound eye image formed on the imaging surface will be described with reference to FIG. Here, for convenience, a portrait image as shown in FIG. For this subject, a compound eye image as shown in (b) is formed on the imaging surface. In (b), reference numeral 12 denotes an image by individual lenses constituting the lens array 3, that is, a single-eye image. A collection of single-eye images having the same number as the number of lenses is a compound-eye image. Reference numeral 13 denotes a shadow by the light shielding member 4, and this area is an invalid area that does not contribute to reconstruction from a compound eye image to a single image.

Now, the observation areas 2a of the lenses of the lens array 3 are shifted from each other. In addition, a shared area 2b is created in which the observation areas of adjacent lenses overlap. The size s of the observation area by each lens of the lens array 3, the shift (parallax) Δ of the observation area between adjacent lenses, and the size w of the shared area are the lens diameter d and the distance e from the lens edge to the light shielding wall. From the lens pitch p, it is determined by the following equations (1) to (3).
s = a · (d + 2 · e) / b (1)
Δ = a · p / b (2)
w = s− (p + 2 · e) (3)

Here, the subject size to be imaged is u, the size of the rectangular hole of the light shielding member 4 is g (the length of one side of the rectangular hole cross section in a plane substantially orthogonal to the lens optical axis), and the size of the imaging surface of the imaging element 6 Is the sum of the field of view of the lens corresponding to the outermost part of the imaging surface and the imaging surface size x among the plurality of lenses constituting the lens array 3 (the lens located on the leftmost side of the lens array in FIG. 1) If the sum of the half of the field of view and the half of the field of view of the rightmost lens and the imaging surface size x) is larger than the subject size u, the size is obtained using the imaging device 6 having an imaging surface smaller than the size u. It is possible to image subjects up to u. Therefore, the required subject size and apparatus thickness are ensured by determining the rectangular hole size g and the image plane distance b of the light shielding member so as to satisfy the relationship of the following expression (4).
g × a / b + x ≧ u (4)

  Here, the image plane distance b is substantially defined by the height of the light shielding member 4 in the lens optical axis direction (the height is substantially matched with b). In this case, the imaging relationship of the lens does not necessarily hold, and the position of the finger can be defined based on the above equation (4) within a range where the lens cutoff frequency does not fall below the required subject frequency. Next, the position (distance a) of the flat plate 10 and the height of the light shielding member 4 in the lens optical axis direction are determined. Since the imaging relationship does not necessarily hold, the contrast of the image picked up decreases due to an increase in wavefront aberration. However, as will be described later, the filtering is performed using the point spread function (PSF) or optical transfer function (OTF) of the lens. Can compensate.

  It should be noted that the skin of the finger is interposed between the blood vessel serving as the subject and the lens array, and the thickness of the skin varies between individuals, so the distance from the lens array 3 to the subject, that is, the subject distance changes. The relationship (4) may be considered assuming that the object is closest to the lens array and the optical magnification increases.

The above description relates only to the horizontal direction (the width direction of the finger 1) in FIG. 1, but the following is also considered if the depth direction, that is, the length direction of the finger 1 is also considered. The rectangular hole size of the light shielding member 4 is g × h (g, h: the length of each side of the rectangular hole cross section in a plane substantially orthogonal to the lens optical axis), the imaging surface size of the imaging element 6 is xx y, and the imaging is performed. In the case where the size of the subject to be (the visual field size at the subject position of the image input device) is u × v,
g × a / b + x ≧ u and h × a / b + y ≧ v (5)
Or h × a / b + y ≧ u and g × a / b + x ≧ v (6)
The position of the flat plate 10, the rectangular hole size g × h, and the image plane distance b may be set so as to satisfy the above.

  As shown in FIG. 1, when the air layer 11 is provided between the flat plate 10 and the lens array 3, the field of view of the lens is enlarged as compared with the case where no air layer is provided. This will be described with reference to FIG.

  In FIG. 4, (a) shows the state of light rays when there is no air layer and the space from the lens to the subject is filled with the lens material. (B) has shown the mode of the light ray in case there exists the air layer 11 (solid line). It can be seen that the field of view s observed in (b) is larger than the field of view s ′ of the lens observed in (a) due to the effect of refraction in the air layer. As described above, when the air layer 11 is provided, it is possible to enlarge the subject size that can be imaged, which is determined by the sum of the field of view of the lens corresponding to the outermost part of the imaging surface and the imaging surface size x. Further, the distance a from the substantially main plane of the lens to ensure the same field of view to the subject can be shortened, and the image input device can be made thinner by that amount. In addition, when the subject distance a is constant, the viewing angle of each lens of the lens array can be reduced, so that a reduction in image quality can be suppressed accordingly.

  Next, a processing system for a compound eye image captured by the image sensor 6 will be described. The present embodiment includes an image input unit 100, a preprocessing unit 101, a parallax detection processing unit 101, a single image reconstruction unit 104, a correction processing unit 105, and a PSF pattern storage unit 106 as elements related to image input. Furthermore, a registration processing unit 107, a registration data storage unit 108, and an authentication processing unit 109 are provided as elements directly related to personal authentication.

  The image input unit 100 captures compound eye image data captured by the image sensor 6. In the following, data of a compound eye image or a single eye image is simply referred to as a compound eye image or a single eye image.

  The pre-processing unit 101 detects a region corresponding to the shadow of the light blocking member 4 in the captured compound eye image, and performs processing for removing the region as an invalid region. Since the shadow area of the light shielding member is the darkest and the area shape is also a lattice shape, the shadow area can be easily detected by binarizing the compound eye image using an appropriate threshold value. Note that the same preprocessing may be performed during the processing of the parallax detection processing unit 103.

  Since the thickness of the finger skin varies among individuals, the subject distance varies. Even if the finger of the same person is used, the distance between the subject and the flat plate 10 varies depending on whether the finger is strongly pressed against the flat plate 10 or lightly pressed, so that the subject distance varies. The size of the observation area and the shared area also changes due to the change in the subject distance. The size of the observation area and the shared area can be obtained as a relative parallax between adjacent single-eye images (or between adjacent single-eye images) in a compound eye image.

  The parallax detection processing unit 103 is a unit that performs processing for detecting such parallax. In the parallax detection processing unit 103, first, two single-eye images including a blood vessel pattern to be used for parallax estimation are extracted from the compound eye image from which the shadow area of the light shielding member is removed. Since the blood vessel pattern is dark, the blood vessel pattern can be easily extracted by binarizing the single eye image. Since the brightness of the shadow portion of the light shielding member and the brightness of the blood vessel pattern are often different, the binarization for extracting the blood vessel pattern differs from the binarization threshold for removing the light shielding member shadow region. A threshold may be used. In addition, since the brightness may be different for each individual image, an average luminance value is obtained for each individual image, and the average luminance value itself or a value proportional to the average luminance value is used as the threshold for binarization. It may be. Two single-eye images for parallax estimation may have no shared area if they are separated from each other, so it is better to extract two single-eye images with adjacent lenses as much as possible. If two adjacent single-eye images including a blood vessel pattern are not found, two non-adjacent single-eye images may be extracted.

  The parallax detection processing unit 103 performs parallax estimation using the extracted single-eye image. For example, the parallax estimation is performed by shifting the calculation of calculating the sum of squares by taking a luminance deviation between one eye image shifted in the x and y directions and the other eye (reference side). It is possible to use a method in which the amount of shift in the x and y directions in which the sum of squares of the luminance deviation is minimized is used as the parallax in the x and y directions by repeatedly changing the amount gradually. Further, for example, estimation processing as described in paragraphs (0087) to (0093) of Patent Document 4 can be applied to parallax estimation.

  If the lens pitch error of the lens array 3 is sufficiently small with respect to the estimated parallax, the positional relationship between the parallax obtained from the two single-eye images by the estimation calculation and the single-eye image defined by the lens pitch of the lens array Based on the above, it is possible to calculate the relative parallax for all the single eye images constituting the compound eye image. When etching or a similar processing method is used for processing the lens array 3, a stage feed error at the time of producing a mask used for etching leads to a lens pitch error of the lens array, but the effective lens area of the lens array is an imaging area of the image sensor 6. Since the lens pitch error of the lens array can be considered to be sufficiently small, the parallax of all single-eye images is calculated based on the parallax estimated between the two single-eye images. There is no problem with estimation. If there is a relatively large pitch error factor during lens array processing, such as expansion or contraction in plastic molding, a single individual eye image is extracted, and all individual eyes including this reference single eye image and blood vessel pattern are extracted. The above-described parallax estimation calculation is performed on the image, and for a single-eye image that does not include a blood vessel pattern, the parallax is calculated based on the parallax and the positional relationship of the single-eye image in a close region, thereby Find the relative parallax of the image. Needless to say, the smaller the number of parallax estimation calculations, the shorter the processing time for parallax detection.

  The single image reconstruction processing unit 104 performs processing for reconstructing a single image from a compound eye image by using the parallax between single-eye images obtained by the parallax detection processing unit 103. Various known methods can be used to reconstruct a single image from a compound eye image. For example, an operation for preparing a reconstructed image space on a memory and rearranging the pixel brightness of a single-eye image to a position determined according to the position and parallax of the single-eye image on the reconstructed image space is performed. A method of reconstructing a single image in the reconstructed image space can be used by repeating for all pixels of a single-eye image. Also, for example, super-resolution processing as described in paragraphs (0094) to (0128) of Patent Document 4 can be used (however, a single-eye image is reconstructed from a low-resolution image and a compound-eye image) Replace the image with a high-resolution image) Such super-resolution processing uses a plurality of low-resolution images and their relative parallax to restore frequency components that exceed the Nyquist frequency of the image sensor in the low-resolution images. By reducing the optical magnification accompanying the reduction in thickness of the system, a single image with improved resolution can be reconstructed with respect to a single-eye image with reduced resolution.

  In the method of Patent Document 4, a relative positional shift between a subject and a camera in a plane orthogonal to the optical system is detected in a plurality of images acquired by a monocular optical system, and reconfiguration is performed using the detected positional deviation. However, when applied to the present invention, a plurality of single-eye images constituting a compound eye image using a lens array are each handled as an image. A single eye image by each lens of the lens array has a different positional relationship with the subject and has a relative shift, and thus can be handled in the same manner as an image acquired by the method of Patent Document 4. Further, Patent Document 4 describes a case where the parallax is smaller than one pixel of the image sensor, but in the present invention, since the subject exists close to the imaging optical system, the parallax between adjacent single-eye images is 1 There may be a pixel larger than the pixel, that is, a pixel that does not share the subject image between adjacent single-eye images. The process of Patent Document 4 is based on the premise that all pixels of a plurality of images share a subject image, and pixels that do not share the subject image using the estimated parallax because noise is generated in pixels that do not share the subject image. Therefore, it is necessary not to perform wideband interpolation or weight integration for pixels that are not shared.

  When the parallax increases and the area where the subject image is shared between the single-eye images becomes small, the super-resolution effect is not expected, and the single-eye image joining processing (described below) is performed, or the estimated size of the parallax Depending on the situation, the super-resolution processing and the joining processing may be selected. Accordingly, it is possible to reduce the memory for the processing and the calculation time compared to the case where the medium resolution processing is applied to the entire surface.

  The single-eye image joining process is a process of joining regions that share adjacent single-eye images in an overlapping manner, as schematically shown in FIG. The upper part of FIG. 5 shows a compound eye image, and the middle part shows two single-eye images (1) and (2) extracted from the compound eye image and inverted vertically and horizontally. The area surrounded by the squares of the single-eye images (1) and (2) represents the shared area, and the single-eye image (1) and the single-eye image (2) are connected so that the shared areas substantially overlap. The bottom row shows the image after stitching.

  More specifically, since the area shared between adjacent single-eye images changes according to the subject distance, the shared area between adjacent single-eye images depends on the detected parallax or the subject distance calculated from the parallax. Is calculated. Of the two adjacent single-eye images, the common area in one single-eye image is valid and the common area in the other single-eye image is invalid, and the effective areas are joined together. When the effective areas of all the single-eye images are connected, a single image of the subject is reconstructed.

  In order to anticipate the effect of super-resolution processing, it is necessary to remove the band limitation such as the low-pass filter attached to the image sensor and the cut-off frequency of each lens constituting the lens array 3, and as described above, the image sensor 6 Are used without a low-pass filter. In addition, it is necessary to design the optical system so that the cut-off frequency is higher than the target spatial frequency (frequency higher than the Nyquist frequency of the image sensor). Such a consideration is not necessary when the super-resolution effect is not expected.

  An optical system that is made thin by using a lens array has a short focal length and a small number of surfaces, and therefore has a low degree of design freedom. Furthermore, when the subject distance is short and fluctuates, as in the present embodiment applied to the application for imaging a blood vessel pattern of a finger for personal authentication, an image in which distortion and defocus are suppressed over the entire eye image. It is generally difficult to obtain. If a single image is reconstructed from a single-eye image having distortion or the like, there is a risk that a discontinuous portion may be generated, thereby hindering image quality.

  In order to suppress the MTF difference due to the image distortion and the incident angle of the light beam to the lens, and to correct the decrease in the MTF due to the difference suppression, an apparatus in consideration of an optical system manufacturing error or an optical system manufacturing error. It is advisable to store the PSF pattern obtained in advance at the stage of evaluation and inspection after manufacture, and to execute the deconvolution operation using this PSF pattern for each individual eye image before reconstruction or a reconstructed single image. Distortion and defocus can be suppressed over a single eye image or the entire reconstructed single image. If the difference in MTF cannot be suppressed, a PSF pattern is obtained and stored for each subject distance or parallax or for each light incident angle, and a PSF pattern corresponding to the subject distance or parallax or the light incident angle is used. It is advisable to execute a deconvolution operation. The deconvolution operation may be performed on each single-eye image before reconstruction or on a single image after reconstruction. When deconvolution calculation is performed on each single-eye image, a PSF pattern that is band-limited appropriately with the Nyquist frequency of the image sensor is used. When performing a convolution operation on the reconstructed image, a PSF pattern that is band-limited appropriately with respect to the band expanded by the super-resolution processing is used.

  The correction processing unit 105 is a unit that performs correction by the above-described convolution calculation, and a PSF pattern stored in the PSF pattern storage unit 106 is used for correction. In the present embodiment, a plurality of PSF patterns corresponding to different parallaxes are stored in the PSF pattern storage unit 106, and PSF patterns corresponding to the parallax detected by the parallax detection processing unit 103 are read from the PSF pattern storage unit 106. And is used by the correction processing unit 105.

  Depending on the required subject size and device thickness, the image quality of the reconstructed image may be reduced due to increased distortion or a decrease in the amount of light around the individual eye image for each individual eye image constituting the compound eye image. However, since distortion and a decrease in the amount of peripheral light can be estimated at the stage of optical design, they can be corrected using estimated values. In that case, at the compound eye image stage, it is only necessary to generate a compound eye image consisting of substantially normal single eye images in which distortion of each single eye image and peripheral light amount reduction are corrected, and to reconstruct a single image from the compound eye image. Even if an ordinary monocular optical system is used to observe a wide field of view of a subject while thinning the apparatus, the above distortion and a decrease in the amount of peripheral light become problems, and it can be corrected by the above method. It can be said that when a single lens constituting the monocular optical system and one lens constituting the lens array is to be corrected for distortion of the same size or a decrease in peripheral light amount, the apparatus according to the present invention is more suitable than when using a monocular optical system. Can be made thinner.

  In the present embodiment, a mode for registering personal data for personal authentication and a mode for executing personal authentication can be selected as the operation mode.

  When the mode for registering personal data is selected, a person who wants to register personal data touches the flat plate 10 with a finger. The blood vessel pattern of the finger is imaged, the compound eye image is taken into the image input unit 100, and a single image of the blood vessel pattern whose distortion is corrected is output from the correction processing unit 105 through the processing described above. The registration processing unit 107 stores the single image as it is in the registration data storage unit 108 as personal data, or extracts feature information such as a branching point coordinate of blood vessel running from the single image, and uses this feature information as a personal data. The data is stored in the registered data storage unit 108 as data.

  When the personal authentication mode is selected, the person who wants to receive authentication brings his finger into contact with the flat plate 10. The blood vessel pattern of the finger is imaged, the compound eye image is taken into the image input unit 100, and a single image of the blood vessel pattern whose distortion is corrected is output from the correction processing unit 105 through the processing described above. The authentication processing unit 109 collates the single image with the personal data stored in the registered data storage unit 108. When the personal data is a single image of the subject itself, even if collation is performed by pattern matching calculation, feature information such as the branch point coordinates of blood vessel running from the input single image and the single image as personal data May be extracted and compared by comparing the feature information. When feature information such as branch point coordinates is registered as personal data, similar feature information is extracted from the input single image, and collation is performed by comparing the feature information. If the authentication processing unit 109 determines that the data matches with the personal data stored in the registration data storage unit 108, the authentication processing unit 109 outputs a message indicating that the authentication is successful and is stored in the registration data storage unit 108. If it is determined that the data does not match with any personal data, an authentication failure message is output. Note that when feature information such as the branch point coordinates of blood vessel travel is used in the authentication process, a feature extraction processing unit is required, but there is an advantage that the amount of data stored in the registered data storage unit 108 can be reduced.

  Now, the resolving power of the subject is the product of the Nyquist frequency of the image sensor and the optical magnification at the subject position. As in the present embodiment, when the subject distance varies due to individual differences in skin thickness or the like, the optical magnification decreases as the subject distance increases, and the resolution of the image decreases. As the resolution decreases, the number of blood vessel patterns that can be used for authentication decreases, resulting in a decrease in authentication accuracy and individual differences in the ease of authentication. In this regard, fluctuations in resolving power due to subject distance can be compensated by the above-described super-resolution processing using relative parallax between single-eye images that have become smaller (sampling is densified) as the subject distance becomes longer. It is possible to suppress the decrease in authentication accuracy and individual differences in the ease of authentication.

  The lenses constituting the lens array 3 will be described with reference to FIG. In the case of the lens array shown in FIG. 1, the lens surface has a lens surface only on one side as shown in FIG.

  According to one aspect of the present invention, as shown in FIG. 6B, the lens array 3 has a pair of a lens formed on the side surface of the subject and a lens formed on the imaging side surface. This lens is also configured to have negative power on the subject side. In a lens that is desired to be thinned, a large amount of power is required for the lens, and the amount of sag increases accordingly. Surfaces with large sag and steep surfaces are difficult to machine. In the case of the configuration of FIG. 6B, the power of the lens surface can be dispersed into two, and the curvature of the lens surface can be reduced. Therefore, the sag amount z of the lens is changed to the sag amount z of the lens of the configuration of FIG. 'Because it can be made smaller, processing becomes easy, which is advantageous in a molding process using a mold.

  Note that the preprocessing unit 101, the parallax detection processing unit 103, the single image reconstruction processing unit 104, the correction processing unit 105, the registration processing unit 107, and the authentication processing unit 109 are realized by hardware in terms of high speed. Although it is advantageous, in the case of a personal authentication device mounted on a notebook computer or an information device with a built-in microcomputer, it would be advantageous in terms of cost to implement all or part of the processing unit by software. .

It is explanatory drawing of the apparatus structure which concerns on embodiment of this invention. It is the figure observed from the flat plate top. It is a figure for demonstrating the compound eye image of a to-be-photographed object. It is a figure explaining a mode that a visual field is expanded by the air layer between a flat plate and a lens array. It is explanatory drawing of the joining process of a single eye image. It is a figure explaining the structure of the lens array for ensuring a wide visual field, without giving strong power to a lens surface.

Explanation of symbols

1 finger (living body)
2 Blood vessels (in-vivo subjects)
3 Lens array 4 Shading member 6 Image sensor 8 LED
10 flat plate 100 image input unit 101 preprocessing unit 102 parallax detection processing unit 103 single image reconstruction processing unit 105 correction processing unit 106 PSF pattern storage unit 107 registration processing unit 108 registration data storage unit 109 authentication processing unit

Claims (7)

An image input device for inputting an image of a subject inside a living body,
A lens array in which a plurality of lenses are arrayed;
A light shielding member for preventing crosstalk on the image plane of light rays passing through each lens of the lens array;
A flat plate member in contact with the living body defines the position of the body in the lens optical axis direction of the lens array,
An imaging that has an imaging surface and captures a compound eye image that is a set of reduced images of a subject inside a living body, the position of which is defined by the flat plate member, which is substantially imaged by a plurality of lenses of the lens array on the imaging surface. Means,
Processing means for reconstructing a single image from a compound eye image captured by the imaging means,
The light shielding member is disposed between the lens array and the imaging surface and has a rectangular hole corresponding to each lens of the lens array,
The size of the rectangular hole is g × h (where g and h are the length of each side of the rectangular hole cross section in a plane substantially orthogonal to the lens optical axis of the lens array), The distance from the plane to the subject is a, the distance from the substantially main plane of each lens of the lens array to the imaging surface is b, and the size of the imaging surface is x × y (where x and y are rectangles) The length of each side of the imaging surface), and the size of the field of view at the position of the subject is u × v (where u and v are the length of each side of the field of view that is a rectangle)
Relationship of g × a / b + x ≧ u and h × a / b + y ≧ v
Or
Relationship of h × a / b + y ≧ u and g × a / b + x ≧ v
An image input device characterized by satisfying The image input apparatus according to claim 1, wherein a height of the light shielding member in a lens optical axis direction is substantially equal to the distance b. The image input device according to claim 1, wherein a gas or liquid layer is interposed between the flat plate member and the lens array. The image input device according to claim 1 , wherein a vacuum layer is interposed between the flat plate member and the lens lens array. The lens array has a lens formed so as to be paired with a surface facing the subject and a surface facing the imaging surface. The lens formed on the surface facing the subject and the imaging surface 5. The image input apparatus according to claim 1, wherein the lenses formed on the opposing surfaces each have a negative power in the direction of the subject. The light source for irradiating light to the biological body which contacts the said flat plate member, The said flat plate member has the transmittance | permeability with respect to the light irradiated by the said light source. The image input device according to claim 1. The image input device according to any one of claims 1 to 6,
Authentication processing means for performing authentication processing using a subject image input by the image input device;
A personal authentication device characterized by comprising:

Images