CN114422729A - Image sensor, image acquisition system and image acquisition method - Google Patents
Image sensor, image acquisition system and image acquisition method Download PDFInfo
- Publication number
- CN114422729A CN114422729A CN202111586654.3A CN202111586654A CN114422729A CN 114422729 A CN114422729 A CN 114422729A CN 202111586654 A CN202111586654 A CN 202111586654A CN 114422729 A CN114422729 A CN 114422729A
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- pixels
- pixel
- image sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000012545 processing Methods 0.000 claims abstract description 20
- 230000003287 optical effect Effects 0.000 claims description 78
- 238000009792 diffusion process Methods 0.000 claims description 29
- 238000013461 design Methods 0.000 claims description 24
- 230000001427 coherent effect Effects 0.000 claims description 19
- 239000000463 material Substances 0.000 claims description 15
- 238000013507 mapping Methods 0.000 claims description 3
- 206010070834 Sensitisation Diseases 0.000 claims 1
- 230000008313 sensitization Effects 0.000 claims 1
- 238000003384 imaging method Methods 0.000 abstract description 5
- 230000008569 process Effects 0.000 abstract description 3
- 238000005265 energy consumption Methods 0.000 abstract 1
- 238000001514 detection method Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 238000006243 chemical reaction Methods 0.000 description 4
- 230000010354 integration Effects 0.000 description 4
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 229910052710 silicon Inorganic materials 0.000 description 3
- 239000010703 silicon Substances 0.000 description 3
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 241001270131 Agaricus moelleri Species 0.000 description 1
- 239000004793 Polystyrene Substances 0.000 description 1
- 229920007962 Styrene Methyl Methacrylate Polymers 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 239000005388 borosilicate glass Substances 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- ADFPJHOAARPYLP-UHFFFAOYSA-N methyl 2-methylprop-2-enoate;styrene Chemical compound COC(=O)C(C)=C.C=CC1=CC=CC=C1 ADFPJHOAARPYLP-UHFFFAOYSA-N 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000000149 penetrating effect Effects 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 229920003023 plastic Polymers 0.000 description 1
- 229920003229 poly(methyl methacrylate) Polymers 0.000 description 1
- 229920000515 polycarbonate Polymers 0.000 description 1
- 239000004417 polycarbonate Substances 0.000 description 1
- 239000004926 polymethyl methacrylate Substances 0.000 description 1
- 229920002223 polystyrene Polymers 0.000 description 1
- SCUZVMOVTVSBLE-UHFFFAOYSA-N prop-2-enenitrile;styrene Chemical compound C=CC#N.C=CC1=CC=CC=C1 SCUZVMOVTVSBLE-UHFFFAOYSA-N 0.000 description 1
- 239000005368 silicate glass Substances 0.000 description 1
- 229920000638 styrene acrylonitrile Polymers 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/70—SSIS architectures; Circuits associated therewith
- H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/128—Adjusting depth or disparity
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Input (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
Disclosed are an image sensor, an image acquisition system and an image acquisition method, which set pixels for designing object distance differences to obtain the depth of field of an acquired image of a corresponding pixel according to the designed object distance of a pixel for clear imaging to obtain a plurality of acquired images for depth of field differences, and then process the plurality of acquired images for depth of field differences to obtain a three-dimensional image. The image sensor, the image acquisition system and the image acquisition method of the invention obtain a plurality of acquired images with different depth of field according to the hardware pixels with different designed object distances, thus reducing the data processing amount of obtaining three-dimensional images, improving the speed of acquiring and obtaining three-dimensional images, improving the image acquisition efficiency and reducing the energy consumption of the system.
Description
Technical Field
The present invention relates to the field of image sensor technology, and in particular, to an image sensor, an image capturing system, and an image capturing method.
Background
With the gradual deepening of the information revolution, the requirements of intelligent equipment on the acquisition and interaction of photoelectric signals are more and more strong, and the intelligent equipment has great requirements on image acquisition in the fields of smart phones, smart homes, future automatic driving, information plants and the like.
Among various types of image sensors, Complementary Metal Oxide Semiconductor (CMOS) type image sensors have been developed in great length in recent years. Nowadays, it has become the mainstream of image acquisition chip to replace the Charge-coupled Device (CCD) type image sensor which has been the mainstream.
At present, optical fingerprint identification has been widely used in multiple intelligent recognition equipment, simultaneously, along with removing payment, the popularization of authentication has just provided higher requirement to optical fingerprint identification's security performance.
In the current optical fingerprint identification application, partial color pixels are usually added into gray-scale pixels, and the effect of distinguishing false fingerprints is achieved by judging the color of an image, however, with the development of material science, the color difference between the false fingerprints and the true fingerprints is reduced, the identification effect of the false fingerprints is reduced, and the safety of the optical fingerprint identification is further influenced.
Disclosure of Invention
In view of the above problems, it is an object of the present invention to provide an image sensor, an image capturing system, and an image capturing method, whereby a three-dimensional image can be obtained by an optical image sensor, and security of optical fingerprint recognition can be improved.
According to an aspect of the present invention, there is provided an image sensor including:
a plurality of pixel units arranged in an array; and
an optical assembly disposed above the plurality of pixel units, for guiding an optical signal reflected from a surface of an object to the plurality of pixel units, each pixel unit for converting the received optical signal into an electrical signal,
wherein each pixel unit comprises a plurality of sub-pixels with different design object distances to obtain an acquired image with depth difference.
Optionally, each of the pixel units further includes:
and the optical path material is arranged on at least part of the sub-pixels and is used for changing the design object distance of the corresponding sub-pixels.
Optionally, the thicknesses of the optical path materials of the plurality of sub-pixels with different designed object distances in the pixel unit are different.
Optionally, the optical component includes a lens, and the curvature radius of the lens corresponding to the sub-pixels with different designed object distances is different.
Optionally, the sub-pixel comprises:
the optical sensing unit is used for converting a received optical signal into a signal charge;
the floating diffusion region is used for converting the signal charges into signal voltages, and the signal voltages are used for representing the collected gray scale information of the sub-pixels; and
a transmitter for transmitting the signal charges to the floating diffusion region.
Optionally, each optical sensing unit of a plurality of sub-pixels in the pixel unit is connected to the same floating diffusion region through a corresponding transmitter.
Optionally, the optical sensing unit includes: at least one of a photodiode and a phototransistor.
Optionally, the photosensitive wavelength band of the optical sensing unit includes a visible light wavelength band and an infrared light wavelength band.
Optionally, the acquired image is any one of a fingerprint image, a palm print image and an iris image.
According to another aspect of the present invention, there is provided an image acquisition system comprising:
an image sensor provided according to the present invention;
and the processing module is used for obtaining a three-dimensional collected image according to the collected image with the depth of field difference obtained by the image sensor.
Optionally, in the photosensitive acquisition, the processing module is configured to perform the following operations:
resetting a plurality of optical sensing cells and a floating diffusion region in the image sensor;
sequentially starting a plurality of transmitters in each pixel unit according to a preset sequence, and controlling the starting time of each transmitter according to the acquisition time;
integrating the signal voltage converted by the floating diffusion region to obtain a plurality of coherent images;
and differentiating the plurality of coherent images to obtain a plurality of collected images which are in one-to-one correspondence with a plurality of sub-pixels in the pixel unit, and obtaining collected images with different depth of field according to the plurality of collected images and the design object distance of the corresponding sub-pixels.
Optionally, the on-times of the plurality of transmitters in the pixel unit are uniform.
Optionally, the image acquisition system is arranged in any one of a smart phone, a computer, a fingerprint acquirer and a palm print acquirer.
According to another aspect of the present invention, there is provided an image capturing method applied to an image capturing system including an image sensor, wherein the image sensor includes a plurality of pixel units arranged in an array, each pixel unit includes a plurality of sub-pixels having different design object distances, and an optical component disposed above the plurality of pixel units, the image capturing method including:
acquiring an acquired image with depth of field difference according to acquired images of a plurality of sub-pixels with different design object distances;
and mapping the collected image with the depth of field difference to a three-dimensional space to obtain a three-dimensional collected image.
Optionally, the step of obtaining the captured image with depth difference comprises:
resetting a plurality of optical sensing cells and a floating diffusion region in the image sensor;
sequentially starting a plurality of transmitters in each pixel unit according to a preset sequence, and controlling the starting time of each transmitter according to the acquisition time;
integrating the signal voltage converted by the floating diffusion region to obtain a plurality of coherent images;
and differentiating the plurality of coherent images to obtain a plurality of collected images which are in one-to-one correspondence with a plurality of sub-pixels in the pixel unit, and obtaining collected images with different depth of field according to the plurality of collected images and the design object distance of the corresponding sub-pixels.
Optionally, the method further comprises:
and acquiring an acquired image corresponding to the depth of field according to the position, the designed object distance and acquired gray scale data of each effective sub-pixel, wherein the effective sub-pixel is a sub-pixel of which the acquired gray scale data is greater than or equal to the corresponding reference gray scale data.
Optionally, the image acquisition method is used for acquisition of at least one of a fingerprint image, a palm print image and an iris image.
Optionally, the image acquisition method is configured in any one of a smart phone, a computer, a fingerprint acquirer and a palm print acquirer.
The image sensor and the image acquisition method provided by the invention design the sub-pixels with different object distances, the sub-pixels corresponding to the positions of the actual object distance and the designed object distance can clearly image, a plurality of acquired images with different depth of field can be obtained according to the positions of the clearly imaged sub-pixels and the designed object distance, and three-dimensional images can be processed and obtained according to the plurality of acquired images with different depth of field, so that convenience is provided for the three-dimensional imaging of the optical sensing unit, and the method can be used for CMOS image sensors, optical fingerprint acquisition equipment, optical cameras and other equipment. And the difference design of the object distance is realized through hardware, the collected images with different depth of field can be conveniently and directly obtained, the calculation amount of back-end data processing can be reduced, and the collection speed is improved.
The image acquisition system provided by the invention comprises the image sensor and the processing module, the processing module processes the acquired image according to the depth of field difference acquired by the image sensor to acquire a three-dimensional image, and the image acquisition system is small in data processing calculated amount, high in processing speed and high in efficiency.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent from the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:
FIGS. 1A and 1B illustrate a fingerprint acquisition system according to one embodiment;
fig. 2A illustrates a pixel circuit diagram of a general CMOS image sensor;
FIG. 2B shows a top view of the pixel circuit of the CMOS image sensor shown in FIG. 2A disposed on a silicon substrate surface;
FIGS. 3A and 3B are schematic diagrams illustrating a pixel structure of an image sensor according to an embodiment of the invention;
fig. 4 shows a schematic view of a sub-pixel optical path structure of an image sensor according to an embodiment of the present invention.
Detailed Description
Various embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Like elements in the various figures are denoted by the same or similar reference numerals. For purposes of clarity, the various features in the drawings are not necessarily drawn to scale.
The following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples.
Fig. 1A and 1B show schematic diagrams of a fingerprint acquisition system according to an embodiment.
Referring to fig. 1A and 1B, an optical fingerprint device 110 of this embodiment is used for a terminal device 100, in this embodiment, the terminal device 100 is a smart phone, in an optional embodiment, the terminal device is a tablet computer or other mobile terminal with a display screen or other terminal devices such as a fingerprint acquirer and a palm print acquirer, and the optical fingerprint device 110 is disposed below the display screen 120, so as to form an Under-screen (Under-display) optical fingerprint system.
The area that the optical sensing unit 111 array in the optical fingerprint device 110 was arranged corresponds fingerprint detection area 103, and finger 01 presses to fingerprint detection area, can realize fingerprint input.
The optical fingerprint device 110 further comprises an optical assembly 112 and a detection portion 114, wherein the detection portion comprises a reading circuit and other auxiliary circuits connected with the optical sensing unit 111 so as to obtain a corresponding electric signal according to the electric information photoelectrically converted by the optical sensing unit 111, and the photosensitive intensity can be obtained according to the electric signal; the optical assembly 112 is disposed above the optical sensing unit 111, and may specifically include a Filter layer (Filter) for filtering ambient light penetrating through the finger, a light guide layer for guiding the reflected light reflected from the surface of the finger to the optical sensing unit 111 for optical detection, and other optical elements.
In a specific implementation, the optical component 112, the optical sensing unit 111, and the detection portion 114 are packaged in the same optical fingerprint module, wherein the light guide layer may be a Collimator (Collimator) layer or a Lens (Lens) layer fabricated on a semiconductor silicon wafer, and the light guide layer has a plurality of collimating units or Lens units, and the collimating units may be small holes, so that, in the reflected light reflected from the finger, the optical fiber perpendicularly incident to the collimating units can pass through and be received by the optical sensing unit 111 below the collimating units, and the obliquely incident light is attenuated by multiple reflections inside the collimating units, so that each optical sensing unit can only receive the reflected light reflected from the fingerprint lines directly above the optical sensing unit, and the fingerprint image of the finger can be detected according to the sensing result of each optical sensing unit 111.
In an alternative embodiment, the display screen 120 is a display screen having a self-luminous display unit, such as an Organic Light Emitting Diode (OLED) display screen or a Micro-LED (Micro-LED) display screen. Taking an OLED display screen as an example, the optical fingerprint device 110 may use a display unit of the OLED display screen located in the fingerprint detection area 103 as an excitation light source for optical fingerprint detection. When the finger 01 is pressed on the fingerprint detection area 103, the display screen 120 emits a light beam 121 like the finger 01 above the fingerprint detection area 103, and the light beam 121 is emitted on the surface of the finger 01 to form reflected light. The ridge and valley of the fingerprint have different light reflection capabilities, reflected light from the ridge and valley of the fingerprint has different light intensities, the reflected light passes through the optical component 112, is received by the optical sensing unit 111 and is converted into corresponding electric signals, and the electric signals are processed to obtain fingerprint image data, so that the fingerprint acquisition function is realized.
Fig. 2A shows a pixel circuit diagram of a general CMOS image sensor, and fig. 2B shows a top view in which the pixel circuit of the CMOS image sensor shown in fig. 2A is disposed on a silicon substrate surface.
Referring to fig. 2A and 2B, a main structure of a pixel of a general CMOS image sensor includes: pd (photodiode): a photodiode (in the present embodiment, the photodiode, in an alternative embodiment, other photosensitive elements such as a phototransistor) correspondingly disposed in the photosensitive region of the CMOS pixel, for sensing light and converting the light into signal charges, wherein the amount of the converted signal charges is related to the intensity of the received light according to the sensing time; fd (floating dispersion): a floating diffusion region for converting the signal charge into a signal voltage; and (c) Cfd: the capacitance of FD; tg (transfer gate): a transmitter for transmitting the signal charge to the FD; rs (reset gate): a reset gate operable to set a voltage of the FD; sf (source follower): a source follower for converting a signal voltage of the FD node into an output signal; SL (selector transistor): a selection transistor; ADC (analog-to-digital conversion): the analog-digital converter converts the received signal voltage (SF-converted output signal, which is transferred to the ADC under SL control) of the FD into gray-scale data.
The dimensions of a transmitter TG and a floating diffusion region FD of pixels with different dimensions generally have fixed parameters, the transmitter TG corresponds to a transistor structure, and the occupied area of the transmitter TG is larger than that of the floating diffusion region FD. The main factor influencing the pixel size is the size of the photodiode PD, and the size of the photodiode PD is determined according to the requirement of image photosensitive acquisition, that is, the area occupation of each structure in the pixel is sequentially reduced according to the sequence from the photodiode PD, the transmitter TG to the floating diffusion FD. It should be noted that the main circuit structure and physical distribution of the CMOS pixels are not particularly limited, and may be flexibly selected according to different application requirements.
Fig. 3A and 3B illustrate a pixel structure schematic diagram of an image sensor according to an embodiment of the present invention, and fig. 4 illustrates a sub-pixel optical path structure schematic diagram of an image sensor according to an embodiment of the present invention.
The light source is generally visible light, the photosensitive band corresponding to the photodiode PD is a visible light band, in an alternative embodiment, other light sources are used, for example, infrared light, correspondingly, the photosensitive band of the photodiode PD includes an infrared light band, the corresponding collected image may also be a deep image (which is convenient for supporting the collection of an iris image), and the collected image that can be obtained by the image sensor of the present invention may be, for example, a fingerprint image, a palm print image, an iris image, and the like.
Referring to fig. 3A, 3B and 4, an image sensor according to an embodiment of the present invention is a CMOS image sensor, and includes a plurality of pixel units 10 arranged in an array, where one pixel unit 10 includes four sub-pixels (a first sub-pixel 1 to a fourth sub-pixel 4), and photodiodes (a first photodiode PD1 to a fourth photodiode PD4) of the four sub-pixels are connected to a same floating diffusion FD through corresponding four transmitters (a first transmitter TG1 to a fourth transmitter TG 4).
The common floating diffusion region FD can increase the usable area of the photodiode PD in the pixel unit 10, increase the size of the photodiode PD within a limited size, and improve the image photosensitive acquisition effect.
An optical component is disposed on a photosensitive path of the photodiode of each sub-pixel to guide an optical signal reflected from a surface of a target to a photosensitive region of each sub-pixel, in this embodiment, the optical component includes a lens 12 (in this embodiment, a convex lens, one lens is disposed corresponding to each sub-pixel, or one lens is shared by a plurality of sub-pixels, and a deviation generated by the lens can be corrected by a back-end data processing, which is not described in detail herein) to introduce lens imaging, and the optical component corresponding to a part of the sub-pixels further includes an optical path material 11, the optical path materials 11 having successively increased thicknesses are disposed on the photosensitive paths of the second sub-pixel 2 to the fourth sub-pixel 4, and radii of curvature of the lenses 12 are different from each other, so that an actual image distance (a design value, a corresponding design image distance) of each lens 12 is consistent with a distance from the lens to the corresponding photodiode PD by adjusting the thickness of the optical path material 11, when the actual object distance is consistent with the designed object distance (in the present invention, the designed object distance may be understood as the distance from the surface of the captured object to the corresponding photodiode PD, that is, the designed object distance of the sub-pixel corresponds to the depth of field of the captured image), the actual image is clearly imaged on the photodiode PD at the designed image distance position, and the photosensitive intensity of the corresponding photodiode PD is the largest.
The optical path material 11 is, for example, transparent glass or plastic material, and is matched with the shape of the photodiode PD, for example, a square column structure, and the specific material is, for example, silicate glass, high borosilicate glass, polymethyl methacrylate, polystyrene, polycarbonate, styrene acrylonitrile, styrene-methyl methacrylate copolymer, and the like, and the specific material is selected according to the actual optical path adjustment requirement and the processing difficulty, and is not described in detail herein.
In an alternative embodiment, the curvature radius of each lens 12 is the same, the smaller the object distance imaged by the lens is, the larger the image distance is, and the adjustment of the designed object distance can be realized by adjusting the thickness of the optical path material 11 to adjust the actual image distance to the actual distance from the lens to the photodiode PD. That is, when the curvature radii of the lenses 12 are uniform, the larger the thickness of the optical path material 11 is, the smaller the design object distance is.
In an alternative embodiment, the design object distance of the corresponding sub-pixel is also adjusted by adjusting the radius of curvature of the lens 12 and the distance from the lens 12 to the photodiode PD.
In an alternative embodiment, the adjustment of the designed object distance is achieved by designing the radii of curvature of the lenses 12 to be different, corresponding to their different object distance-image distance matching relationships, and to be different when they have the same designed image distance.
In practical application, an actual designed object distance adjusting scheme is selected according to the implementation difficulty and the actual requirement of each scheme, and details are not described herein.
When the finger 30 is pressed on the screen 20, the distances from different surfaces of the finger to the screen 20 are different, and when the actual object distance of each position of the finger is consistent with the designed object distance of the sub-pixel at the corresponding position, the corresponding position of the finger can be accurately imaged on the photodiode of the photosensitive area of the corresponding sub-pixel, and the contrast of the image acquired by the corresponding sub-pixel is high.
According to the difference between the preset object distance (designed object distance) and the actual distance (actual object distance) between the surface of the object to be acquired in the photosensitive area of each sub-pixel and the lens, the amount of signal charges obtained by conversion of the photodiode PD in the same photosensitive acquisition time is different, the correspondingly converted signal voltages are different, the gray scale data obtained according to the signal voltages are also different (the contrast of the corresponding pixel point is different in the acquired image obtained by the sub-pixel with the same designed object distance), when the actual object distance is consistent with the designed object distance (the specific range is set according to the actual situation, and is not described in detail), the acquired gray scale data is taken as reference gray scale data (the reference gray scale data corresponding to the pixels with different designed object distances is different), the sub-pixel with the acquired gray scale data larger than or equal to the reference gray scale data is taken as an effective sub-pixel, and according to the position of the effective sub-pixel and the designed object distance, the acquired image corresponding to the depth of field can be obtained, and the minimum resolution size of the three-dimensional image obtained correspondingly is the size of one sub-pixel.
When the difference between the actual object distance and the designed object distance is large, effective imaging cannot be achieved, the contrast of the corresponding acquired image is low, the data are removed, and the corresponding area of the corresponding finally obtained three-dimensional image is empty (the empty area can be subjected to fitting, filling and completing through the whole three-dimensional image).
In the present embodiment, the step of obtaining the captured images of the four sub-pixels in the pixel unit 10 (the execution of the relevant operations is controlled by the processing module of the corresponding image capturing system) includes:
resetting the leds (PD1 to PD4) and the floating diffusion FD of the four sub-pixels (reset operation such as turning on each transmitter and grounding each end of the floating diffusion FD to release the charges emptied therein to realize reset), then turning off each transmitter (TG1 to TG4), after an integration time t1, sequentially turning on a first transmitter TG1, a second transmitter TG2, a third transmitter TG3 and a fourth transmitter TG4 (signal charges corresponding to four sub-pixels are transferred to the floating diffusion FD), and respectively maintaining an acquisition time t2 (signal charges ensuring photoelectric conversion are all transferred to the floating diffusion and all converted to a signal voltage), the corresponding total integration time t1+4 t2, obtaining five coherent images (signal voltage integration provided to the floating diffusion FD is obtained, differentiation is performed according to the turn-on sequence of the sub-pixels, the charge amount obtained by photoelectric conversion of each sub-pixel can be obtained, and then the photosensitive gray scale data of each sub-pixel is obtained), the number is numbered from the zeroth coherent image to the fourth coherent image according to the integration time, the fourth coherent image subtracts the third coherent image to obtain the collected image of the fourth sub-pixel 4, the third coherent image subtracts the second coherent image to obtain the collected image of the third sub-pixel 3, the second coherent image subtracts the first coherent image to obtain the collected image of the second sub-pixel 2, and the first coherent image subtracts the zeroth coherent image to obtain the collected image of the first sub-pixel 1 (one sub-pixel is taken as a reference, and the collected image corresponds to the gray scale data).
In the present embodiment, four kinds of sub-pixels with designed object distances are provided, and the values of the designed object distances are in an arithmetic progression (for example, the designed object distances are arranged in a size of 0.5, 0.6, 0.7, 0.8 (unit: mm), the specific designed object distance of each sub-pixel can be arbitrarily selected and arranged in the order of the first sub-pixel 1 to the fourth sub-pixel 4, the designed object distances can be 0.6, 0.5, 0.7, 0.8, or 0.8, 0.7, 0.6, 0.5, only the specific designed object distance and the position of each sub-pixel need to be confirmed, so as to confirm the depth of field of each image pixel of the acquired image, that is, the basic requirement of the present application for the specific designed object distance of each sub-pixel is different, and the respective specific designed object distance and the arrangement mode are not particularly limited), the curvature radii of the corresponding lenses 12 are the same or different, and the optical path materials 11 with matched thicknesses are correspondingly arranged so that the image distances are all consistent with the distance from the lens 12 to the photodiode PD.
In an alternative embodiment, the sub-pixels with object distance difference are two, three or more, that is, two, three or more sub-pixels are included in one pixel unit (corresponding to the present embodiment, the photodiodes PD of two, three or more sub-pixels are connected to the same floating diffusion region FD through a corresponding number of transmitters TG), the corresponding three-dimensional levels are different, and the precision of the three-dimensional images obtained by the three-dimensional levels is different, and actually, the three-dimensional levels can be selected according to specific requirements, and are not particularly limited herein.
In this embodiment, a set of sub-pixels with different design object distances (a set of sub-pixels 1 to 4 with different design object distances) is integrated into a pixel unit, and a floating diffusion region FD is used to convert signal charges into signal voltages, so as to save the number of the floating diffusion regions FD and reduce the area occupation of the floating diffusion regions FD.
In an embodiment, the four images obtained by the four sub-pixels are respectively subjected to the recognition processing, and the specific content of the recognition processing for each of the four images may include:
the maximum value and the minimum value of the gray scale data of each collected image are differentiated to obtain gray scale difference, each gray scale data is grouped according to the gray scale difference and the size of the gray scale data to obtain multiple groups of gray scale data (for example, the gray scale of 0-255 is divided into four groups of 0-63, 64-127, 128-191 and 192-255) with the average gray scale data sequentially increasing or decreasing, the sub-pixel corresponding to the maximum group of gray scale data (for example, 192-255, namely, the group of image pixels with the maximum contrast) in the multiple groups of gray scale data is marked as an effective sub-pixel, the effective sub-pixel is the sub-pixel with the actual object distance of the collected image consistent with the designed object distance, the texture and the position of the collected image corresponding to the depth of field can be confirmed according to the position of the effective sub-pixel (or other three groups of gray scales are deleted, and the left pixel image is the collected image corresponding to the depth of field), and mapping the four images to a three-dimensional space according to the lines, the depth of field and the positions of the four depth of field images to obtain a three-dimensional acquisition image.
The four sub-pixels have different designed object distances, self-contrast is carried out according to the contrast of the images obtained respectively, image lines corresponding to the depth of field are obtained, and effective sub-pixels can be confirmed without comparison based on reference gray scale data.
The effective sub-pixels can be confirmed by comparing each gray scale data with the reference gray scale data, and the sub-pixels with the gray scale data being greater than or equal to the reference gray scale data are the effective sub-pixels. Or according to the contrast of each collected image, the partial image with the contrast larger than the preset reference value is the collected image corresponding to the depth of field.
It should be noted that the present invention mainly lies in the design of the design object distance of the sub-pixels, and several possible data processing methods are shown above, but the implementation of the present invention is not limited to the data processing method disclosed in the present application.
The image sensor and the image acquisition method provided by the invention are provided with the sub-pixels with different object distances, the sub-pixels corresponding to the positions of the actual object distance and the designed object distance can clearly image, a plurality of acquired images with different depth of field can be obtained according to the positions of the sub-pixels which clearly image and the designed object distance, and three-dimensional images can be processed and obtained according to the plurality of acquired images with different depth of field, so that convenience is provided for the three-dimensional imaging of the optical sensing unit, and the method can be used for devices such as a CMOS image sensor, an optical fingerprint acquisition device and an optical camera. And the difference design of the object distance is realized through hardware, the collected images with different depth of field can be conveniently and directly obtained, the calculation amount of back-end data processing can be reduced, and the collection speed is improved.
The invention also provides an image acquisition system, which comprises the image sensor and the processing module, wherein the processing module processes the acquired image according to the depth of field difference acquired by the image sensor to acquire a three-dimensional image, and the image acquisition system has the advantages of small data processing calculated amount, high processing speed and high efficiency.
While embodiments in accordance with the invention have been described above, these embodiments are not intended to be exhaustive or to limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. The invention is limited only by the claims and their full scope and equivalents.
Claims (18)
1. An image sensor, comprising:
a plurality of pixel units arranged in an array; and
an optical assembly disposed above the plurality of pixel units, for guiding an optical signal reflected from a surface of an object to the plurality of pixel units, each pixel unit for converting the received optical signal into an electrical signal,
wherein each pixel unit comprises a plurality of sub-pixels with different design object distances to obtain an acquired image with depth difference.
2. The image sensor of claim 1, wherein each of the pixel cells further comprises:
and the optical path material is arranged on at least part of the sub-pixels and is used for changing the design object distance of the corresponding sub-pixels.
3. The image sensor of claim 2, wherein the optical path material thickness of the plurality of sub-pixels in the pixel unit for which object distances are designed to be different is different.
4. The image sensor of claim 1, wherein the optical assembly includes a lens, and wherein the radius of curvature of the lens is different for sub-pixels having different design object distances.
5. The image sensor of claim 1, wherein the sub-pixels comprise:
the optical sensing unit is used for converting a received optical signal into a signal charge;
the floating diffusion region is used for converting the signal charges into signal voltages, and the signal voltages are used for representing the collected gray scale information of the sub-pixels; and
a transmitter for transmitting the signal charges to the floating diffusion region.
6. The image sensor of claim 5, wherein each photo-sensing element of a plurality of sub-pixels in the pixel cell is connected to the same floating diffusion region by a corresponding transmitter.
7. The image sensor of claim 5, wherein the optical sensing unit comprises: at least one of a photodiode and a phototransistor.
8. The image sensor of claim 5,
the photosensitive wave band of the optical sensing unit comprises a visible light wave band and an infrared light wave band.
9. The image sensor of claim 1,
the collected image is any one of a fingerprint image, a palm print image and an iris image.
10. An image acquisition system comprising:
the image sensor according to any one of claims 1 to 9;
and the processing module is used for obtaining a three-dimensional collected image according to the collected image with the depth of field difference obtained by the image sensor.
11. The image acquisition system according to claim 10,
in the sensitization acquisition, the processing module is configured to execute the following operations:
resetting a plurality of optical sensing cells and a floating diffusion region in the image sensor;
sequentially starting a plurality of transmitters in each pixel unit according to a preset sequence, and controlling the starting time of each transmitter according to the acquisition time;
integrating the signal voltage converted by the floating diffusion region to obtain a plurality of coherent images;
and differentiating the plurality of coherent images to obtain a plurality of collected images which are in one-to-one correspondence with a plurality of sub-pixels in the pixel unit, and obtaining collected images with different depth of field according to the plurality of collected images and the design object distance of the corresponding sub-pixels.
12. The image acquisition system according to claim 11, wherein the on-times of the plurality of transmitters in the pixel unit coincide.
13. The image acquisition system according to claim 10,
the image acquisition system is arranged in any one of a smart phone, a computer, a fingerprint acquirer and a palm print acquirer.
14. An image acquisition method is applied to an image acquisition system comprising an image sensor, wherein the image sensor comprises a plurality of pixel units which are arranged in an array, each pixel unit comprises a plurality of sub-pixels with different design object distances, and an optical assembly arranged above the plurality of pixel units, and the image acquisition method comprises the following steps:
acquiring an acquired image with depth of field difference according to acquired images of a plurality of sub-pixels with different design object distances;
and mapping the collected image with the depth of field difference to a three-dimensional space to obtain a three-dimensional collected image.
15. The image acquisition method according to claim 14, wherein the step of obtaining the acquired image with a difference in depth of field comprises:
resetting a plurality of optical sensing cells and a floating diffusion region in the image sensor;
sequentially starting a plurality of transmitters in each pixel unit according to a preset sequence, and controlling the starting time of each transmitter according to the acquisition time;
integrating the signal voltage converted by the floating diffusion region to obtain a plurality of coherent images;
and differentiating the plurality of coherent images to obtain a plurality of collected images which are in one-to-one correspondence with a plurality of sub-pixels in the pixel unit, and obtaining collected images with different depth of field according to the plurality of collected images and the design object distance of the corresponding sub-pixels.
16. The image acquisition method according to claim 14, further comprising:
and acquiring an acquired image corresponding to the depth of field according to the position, the designed object distance and acquired gray scale data of each effective sub-pixel, wherein the effective sub-pixel is a sub-pixel of which the acquired gray scale data is greater than or equal to the corresponding reference gray scale data.
17. The image acquisition method according to claim 14,
the image acquisition method is used for acquiring at least one of a fingerprint image, a palm print image and an iris image.
18. The image acquisition method according to claim 14,
the image acquisition method is configured in any one of a smart phone, a computer, a fingerprint acquirer and a palm print acquirer.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111586654.3A CN114422729B (en) | 2021-12-23 | 2021-12-23 | Image sensor, image acquisition system and image acquisition method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111586654.3A CN114422729B (en) | 2021-12-23 | 2021-12-23 | Image sensor, image acquisition system and image acquisition method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN114422729A true CN114422729A (en) | 2022-04-29 |
CN114422729B CN114422729B (en) | 2024-09-17 |
Family
ID=81266638
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111586654.3A Active CN114422729B (en) | 2021-12-23 | 2021-12-23 | Image sensor, image acquisition system and image acquisition method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114422729B (en) |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011061134A (en) * | 2009-09-14 | 2011-03-24 | Zycube:Kk | Semiconductor image sensor |
CN102289048A (en) * | 2010-06-15 | 2011-12-21 | 夏普株式会社 | Lens element, imaging lens, and imaging module |
CN103004180A (en) * | 2010-05-12 | 2013-03-27 | 派力肯影像公司 | Architecture of Imager Arrays and Array Cameras |
CN106470299A (en) * | 2015-08-18 | 2017-03-01 | 杭州海康机器人技术有限公司 | Camera lens, video camera, parcel detecting system and image processing method |
CN206115120U (en) * | 2016-08-31 | 2017-04-19 | 佛山科学技术学院 | Solve image device of many first depth of field with adjustable |
CN106847092A (en) * | 2017-03-10 | 2017-06-13 | 京东方科技集团股份有限公司 | A kind of display panel, display device and its method for obtaining image |
CN111669509A (en) * | 2020-07-15 | 2020-09-15 | 江苏骠马智能工业设计研究有限公司 | Automatic focusing device and automatic focusing method for changing image space optical path |
WO2020181489A1 (en) * | 2019-03-12 | 2020-09-17 | 深圳市汇顶科技股份有限公司 | Fingerprint recognition device, fingerprint recognition method and electronic device |
US20210056284A1 (en) * | 2019-08-23 | 2021-02-25 | Shenzhen GOODIX Technology Co., Ltd. | Fingerprint detection apparatus and method and electronic device |
CN213659066U (en) * | 2020-12-22 | 2021-07-09 | 华天慧创科技(西安)有限公司 | Splicing lens capable of extending imaging depth of field |
-
2021
- 2021-12-23 CN CN202111586654.3A patent/CN114422729B/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011061134A (en) * | 2009-09-14 | 2011-03-24 | Zycube:Kk | Semiconductor image sensor |
CN103004180A (en) * | 2010-05-12 | 2013-03-27 | 派力肯影像公司 | Architecture of Imager Arrays and Array Cameras |
CN102289048A (en) * | 2010-06-15 | 2011-12-21 | 夏普株式会社 | Lens element, imaging lens, and imaging module |
CN106470299A (en) * | 2015-08-18 | 2017-03-01 | 杭州海康机器人技术有限公司 | Camera lens, video camera, parcel detecting system and image processing method |
CN206115120U (en) * | 2016-08-31 | 2017-04-19 | 佛山科学技术学院 | Solve image device of many first depth of field with adjustable |
CN106847092A (en) * | 2017-03-10 | 2017-06-13 | 京东方科技集团股份有限公司 | A kind of display panel, display device and its method for obtaining image |
WO2020181489A1 (en) * | 2019-03-12 | 2020-09-17 | 深圳市汇顶科技股份有限公司 | Fingerprint recognition device, fingerprint recognition method and electronic device |
US20210056284A1 (en) * | 2019-08-23 | 2021-02-25 | Shenzhen GOODIX Technology Co., Ltd. | Fingerprint detection apparatus and method and electronic device |
CN111669509A (en) * | 2020-07-15 | 2020-09-15 | 江苏骠马智能工业设计研究有限公司 | Automatic focusing device and automatic focusing method for changing image space optical path |
CN213659066U (en) * | 2020-12-22 | 2021-07-09 | 华天慧创科技(西安)有限公司 | Splicing lens capable of extending imaging depth of field |
Non-Patent Citations (3)
Title |
---|
YANG JOSEPH: "36‐3: Microlens Collimation Film with Near‐Infrared Spectral Filter for Large‐Area Fingerprint Sensor", SID SYMPOSIUM DIGEST OF TECHNICAL PAPERS, 28 June 2021 (2021-06-28) * |
吉紫娟: "偶次非球面镜在指纹采集系统中的应用研究", 湖北第二师范学院学报, 20 February 2012 (2012-02-20) * |
梁建霞;胡超;刘晓娜;: "基于CCD的掌纹掌脉采集模块设计", 传感器与微系统, no. 07, 20 July 2016 (2016-07-20) * |
Also Published As
Publication number | Publication date |
---|---|
CN114422729B (en) | 2024-09-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107346152B (en) | Display device, display screen and terminal | |
US8411375B2 (en) | Method and apparatus providing gradient index of refraction lens for image sensors | |
CN109844766B (en) | Optical image acquisition system and electronic device | |
CN111095282A (en) | Fingerprint detection device and electronic equipment | |
CN105679782A (en) | Contact image sensor | |
CN110555367B (en) | Optical fingerprint sensor module | |
CN104318199A (en) | Compound optical sensor and manufacture method and use method thereof | |
CN111353405B (en) | Fingerprint identification device, fingerprint identification system and electronic equipment | |
KR20130011218A (en) | Method of measuring a distance and three-dimensional image sensor performing the same | |
US9525005B2 (en) | Image sensor device, CIS structure, and method for forming the same | |
CN212624070U (en) | Fingerprint Sensing Module | |
CN113627227B (en) | Optical imaging device | |
CN213278093U (en) | Semiconductor device | |
KR20220147448A (en) | Image sensor including color separating lens array and electronic apparatus including the image sensor | |
CN111213152B (en) | Optical image acquisition unit, optical image acquisition system, display screen and electronic equipment | |
CN210038815U (en) | Optical image acquisition unit, optical image acquisition system, display screen and electronic equipment | |
CN110008840B (en) | Optical fingerprint module and fingerprint identification display device | |
US20220329747A1 (en) | Image sensor including auto-focus pixels | |
CN114422729B (en) | Image sensor, image acquisition system and image acquisition method | |
CN107704827B (en) | Photoelectric sensor and photoelectric sensor array with same | |
CN211480030U (en) | Thin optical fingerprint identification device | |
CN210864757U (en) | Fingerprint detection device and electronic equipment | |
CN210072645U (en) | Microlens light path structure for OLED screen fingerprint identification and electronic equipment | |
CN111368722A (en) | Method and system for imaging fingerprint under screen | |
US12244945B2 (en) | Image sensor including phase detection pixels |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |