CN103188499A - 3D imaging module and 3D imaging method - Google Patents

Abstract

The invention relates to a 3D (Three Dimensional) imaging module which comprises a first imaging unit, a second imaging unit, a memory, a color separating unit, a processor, an image processing unit, a first driver, a second driver and an image synthesis unit, wherein the first imaging unit and the second imaging unit are used for capturing images in the same scene at different angles simultaneously; the memory is used for memorizing the images captured by the first imaging unit and the second imaging unit; the color separating unit is used for showing the images captured by the first imaging unit and the second imaging unit with three primary colors namely red, green and blue respectively; the processor is used for determining a current shoot mode; the image processing unit performs out-of-focus fuzzy modification in a software compute simulation mode; and the first driver and the second driver focus the first imaging unit and the second imaging unit. The invention further relates to a 3D imaging method.

Description

3D imaging module and 3D imaging method

technical field

The invention relates to an imaging module, in particular to a 3D imaging module with an automatic focus function and a 3D imaging method.

Background technique

With the advancement of technology, 3D (three-dimensional) imaging modules have been increasingly used in many fields, and in order to achieve better imaging effects, 3D imaging modules are also required to have an auto-focus function.

The auto-focus technology of the imaging module is divided into mechanical auto-focus technology and digital auto-focus technology. Mechanical autofocus technology uses a mechanical structure to move the lens inside the lens to focus. Since the mechanical focusing technology needs to adopt a complicated mechanical structure, the cost of the auto-focus lens increases and the volume is relatively large.

The digital auto-focus technology processes the image sensed by the image sensor through software simulation and calculation, so that the blurred image due to out-of-focus pixels on the image sensor becomes clear. For example, Extend Depth of Field (EDoF) technology uses the three primary colors of light (red, green, blue) to have the best MTF curves at different distances. Best Primary Color uses algorithms to digitally simulate the other two primary colors to achieve full-frame clear images. However, the defect of digital autofocus is that it is not capable of short-range imaging. Generally speaking, if the object distance is within 40cm, the focusing effect of digital autofocus technology is often unsatisfactory.

Contents of the invention

In view of this, it is necessary to provide a 3D imaging module and a 3D imaging method that avoid the above problems.

A 3D imaging module, which includes a first imaging unit and a second imaging unit, a memory connected to the first image sensor and the second image sensor, a color memory connected to the memory separation unit, a processor connected to the color separation unit, an image processing unit connected to the processor, a first driver and a second driver connected to the processor respectively, and a Image synthesis unit; the first imaging unit and the second imaging are used to capture images of the same scene at different angles at the same time; the memory is used to store the images captured by the first imaging unit and the second imaging unit image; the color separation unit is used to represent the images captured by the first imaging unit and the second imaging unit with three primary colors of red, green and blue respectively; Perform MTF calculations on images captured by the imaging module and the second imaging module, determine the current shooting mode according to the calculation results, and select and control the image processing unit or the first driver and the second driver according to the determined current shooting mode The driver; the image processing unit is used to perform out-of-focus and blur correction on the images captured by the first imaging unit and the second imaging unit through image processing; the first driver and the second driver are respectively is used to focus the first imaging unit and the second imaging unit; the image synthesis unit is used to process or focus the images captured by the first imaging unit and the second imaging unit The final images are synthesized to obtain a 3D image.

A 3D imaging method, comprising the steps of:

Use two imaging units to shoot the same scene at different angles at the same time;

performing color separation on the images sensed by the image sensors of the two imaging units;

Perform MTF calculation on the image area sensed by each pixel unit of the image sensors of the two imaging units to obtain the MTF value corresponding to the image area sensed by each pixel unit;

determining the object distance of the image sensed by each pixel unit according to the MTF value of the image sensed by each pixel unit;

determining the current shooting mode according to the object distance of the image sensed by each pixel unit;

If the current shooting mode is close-focus mode, perform the following steps:

According to the object distance of the image sensed by each pixel unit, respectively determine the best focus positions of the imaging lenses of the two imaging modules;

Determine the focus drive amounts of the imaging lenses of the two imaging modules respectively according to the best focus position;

Drive the imaging lenses of the two imaging modules to the best focus position respectively according to the focus drive amount;

Combining the focused images captured by the two imaging units to obtain a 3D image;

If the current shooting mode is the far focus mode, perform the following steps:

According to the MTF value corresponding to the image area sensed by each pixel unit, determine the blur amount of the image sensed by the corresponding pixel unit;

According to the blur amount of the image sensed by each pixel unit, determine the blur correction amount of the image sensed by the corresponding pixel unit;

performing blur correction on the image sensed by each pixel unit according to the blur correction amount;

The blur-corrected images captured by the two imaging units are synthesized to obtain a 3D image.

Compared with the prior art, the 3D imaging module and the 3D imaging method use software calculation and simulation to determine the object distance of the subject, and determine the current shooting mode according to the situation of the object distance, and select the software calculation method or The way of driving the imaging lens to focus can achieve a clear-focus image no matter in the near-focus or far-focus shooting mode. The images of the same scene captured by the two imaging units at the same time can be synthesized, so you can get more Clear 3D images.

Description of drawings

FIG. 1 is a schematic diagram of a 3D imaging module according to an embodiment of the present invention.

FIG. 2 is a flowchart of an autofocus method according to an embodiment of the present invention.

Description of main component symbols

3D imaging module 100 first imaging unit A second imaging unit B first imaging lens 11 first image sensor 12 second imaging lens 13 second image sensor 14 lens 111, 131 memory 20 color separation unit 30 processor 40 MTF operation module 41 Object distance calculation module 42 Object distance judgment module 43 Fuzzy calculation module 44 Fuzzy correction calculation module 45 Focus position calculation module 46 driving volume calculation module 47 image processing unit 50 first drive 60 second drive 70 image synthesis unit 80

The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings.

Please refer to FIG. 1 , which is a schematic diagram of a 3D imaging module 100 according to an embodiment of the present invention. The 3D imaging module 100 includes a first imaging unit A and a second imaging unit A arranged side by side with the first imaging unit A. Imaging unit B. The first imaging unit A and the second imaging unit B shoot the same scene from different angles at the same time.

The first imaging unit A includes a first imaging lens 11 and a first image sensor 12 aligned with the optical axis of the first imaging lens 11 . The second imaging unit B includes a second imaging lens 13 and a second image sensor 14 aligned with the optical axis of the second imaging lens 13 .

The first imaging lens 11 and the second imaging lens 13 are used to capture images of objects, and focus and project the captured images to the first image sensor 12 and the second image sensor 14 respectively. sensing area. The first imaging lens 11 and the second imaging lens 13 respectively include at least one lens 111 , 131 with positive refractive power, and the lenses 111 , 131 are aspherical lenses.

The first image sensor 12 and the second image sensor 14 are used to sense images captured by the first imaging lens 11 and the second imaging lens 13 respectively. Each of the first image sensor 12 and the second image sensor 14 includes a plurality of pixel units (not shown in the figure), and the plurality of pixel units are distributed in an effective array of the corresponding image sensor. sensing area. Wherein, each pixel unit includes three primary colors (red, green, blue) pixels. Preferably, both the first image sensor 12 and the second image sensor 14 include at least 2048×1536 pixel units. In this embodiment, the first image sensor 12 and the second image sensor 14 may be Charged-coupled Device (CCD) sensors or Complementary Metal Oxide Semiconductor (CMOS) sensors.

The 3D imaging module 100 also includes a memory 20, a color separation unit 30, a processor 40, an image processing unit 50, a first driver 60 corresponding to the first imaging lens 11 and a corresponding to The second driver 70 of the second imaging lens 13 . The memory 20 is connected to the first image sensor 12 and the second image sensor 14, the color separation unit 30 is connected to the memory, the processor 40 is connected to the color separation unit 30, the image processing The unit 50, the first driver 60 and the second driver 70 are respectively connected to the processor 40, and the first driver 60 and the second driver 70 are also connected to the first imaging lens 11 and the first imaging lens 11 respectively. The second imaging lens is connected.

The memory 20 is used for storing images sensed by the first image sensor 12 and the second image sensor 14 .

The color separation unit 30 is used for separating the images sensed by the first image sensor 12 and the second image sensor 14 into images respectively represented by three primary colors.

The processor 40 includes a modulation transfer function (Modulation Transfer Function: MTF) operation module 41, an object distance operation module 42, an object distance judgment module 43, a fuzzy amount operation module 44, and a fuzzy correction amount operation module 45 , a focus position computing module 46, and a driving amount computing module 47. The MTF computing module 41 is connected to the color separation unit 30, the object distance computing module 42 is connected to the MTF computing module 41, the object distance judging module 43 is connected to the object distance computing module 42, and the object distance computing module 42 is connected to the object distance computing module 42. The focus position calculation module 46 and the blur amount calculation module 44 are respectively connected to the object distance judgment module 43, and the driving amount calculation module 47 is connected to the focus position calculation module 46, the first driver 60 and the The second driver 70 is connected; the blur correction amount calculation module 45 is respectively connected with the blur correction amount calculation module 45 and the image processing unit 50 .

The MTF calculation module 41 is used for performing MTF calculation on the image area sensed by each pixel unit on the first image sensor 12 and the second image sensor 14 to obtain the MTF value of the corresponding area. In this embodiment, the MTF computing module 41 performs MTF value computing on the three primary color images corresponding to each pixel unit.

The object distance calculation module 42 is used for determining the object distance of the image sensed by each pixel unit according to the calculation result of the MTF calculation module.

The object distance judging module 43 is used to determine the current shooting mode according to the calculation result of the object distance calculation module 42 . Specifically, the object distance judging module 43 performs comprehensive calculation on the calculation result of the object distance calculation module, compares the comprehensive calculation result with a preset standard value, and determines the current shooting mode according to the comparison result. In this embodiment, the comprehensive operation is to sample the object distance of the image sensed by each pixel unit obtained by the object distance operation module 42, and obtain the object distance used to represent the main target object currently photographed according to the sampled data operation. The object distance characterization quantity of the distance. The preset standard value is used to distinguish the current shooting mode as the near focus mode or the far focus mode. In this embodiment, the standard value is 40cm. If the object distance representation is greater than 40cm, the current shooting mode is far Focus mode, if the object distance characteristic is less than (equal to) 40cm, the current shooting mode is close focus mode.

The blur amount calculation module 44 is used to determine the difference between the MTF value obtained by the operation of each pixel unit and the standard MTF value within the corresponding object distance according to the calculation result of the MTF calculation module 41, and determine each The amount of blur in the image sensed by the pixel unit. The standard MTF value is the MTF value of the clearest image area sensed by each pixel unit within the corresponding object distance, therefore, the MTF value of each pixel unit calculated by the MTF calculation module 41 is the same as the corresponding standard MTF The difference between the values can characterize the amount of blur of the image sensed by each pixel unit. In this embodiment, the blur amount calculation module 44 performs blur amount calculation on the three primary color images of each pixel unit. The blur calculation module 44 determines whether its function is enabled or not according to the shooting mode determined by the object distance judging module 43 . In this embodiment, when the object distance judging module 43 judges that the current shooting mode is the far focus mode, the function of the blur amount calculation module 44 is turned on; when the object distance judging module 43 judges that the current shooting mode is the near focus mode , the function of the fuzzy quantity calculation module 44 is turned off.

The blur correction amount calculation module 45 is used for determining the blur correction amount for the image sensed by each pixel unit according to the blur amount obtained by the blur amount calculation module 44 . In this embodiment, the blur correction amount calculation module 45 performs three primary color blur correction calculations on the image of each pixel unit.

The focus position calculation module 46 is used for determining the best focus positions of the first imaging lens 11 and the second imaging lens 13 according to the calculation result of the object distance calculation module 42 . The focus position computing module 46 determines whether its function is enabled according to the shooting mode determined by the object distance judging module 43 . In this embodiment, when the object distance judging module 43 judges that the current shooting mode is the near-focus mode, the function of the focus position calculation module 46 is turned on; when the object distance judging module 43 judges that the current shooting mode is the far-focus mode , the function of the focus position computing module 46 is turned off.

The driving amount calculation module 47 is used to determine the focus of the first imaging lens 11 and the second imaging lens 13 according to the best focus position of the imaging lens 10 obtained by the object distance calculation module 42 drive volume.

The image processing unit 50 is used for performing blur correction on the image sensed by each pixel unit according to the correction amount obtained by the blur correction amount calculation module 45 to obtain a clear image. In this embodiment, the image processing unit 50 corrects the three primary colors on the image of each pixel unit. The blur-corrected images sensed by the first image sensor 12 and the second image sensor 14 are stored in the memory 20 .

The first driver 60 and the second driver 70 are respectively used to drive the first imaging lens 11- and the second imaging lens 13 according to the focus driving amounts obtained by the driving amount calculation module 47. The imaging lens 11 and the second imaging lens 13 are at the best focus position. In this embodiment, the first driver 60 and the second driver 70 are piezoelectric motors. Of course, the first driver 60 and the second driver 70 may also be other types of drivers such as voice coil motors. components. After the first imaging lens 11 and the second imaging lens 13 are driven to the best focus position by the first driver 60 and the second driver 70, the captured images are stored in the memory 20 Inside.

The 3D imaging module 100 includes an image synthesis unit 80, the image synthesis unit 80 is used to read the focused image in the memory 20 through the first driver 60 and the second driver 70 or through the The image processing unit 50 processes the image after blur correction, and synthesizes the images to obtain a 3D image. Specifically, the image synthesis unit 80 reads the images captured by the first imaging unit A and the second imaging unit B at the same time at different angles of the same scene after focusing or blur correction each time, and according to The first imaging unit and the second imaging unit recover the depth and distance information of objects in the scene from different shooting angles of the same scene to obtain images with visual depth and distance.

Please refer to Fig. 2, the 3D imaging method of the embodiment of the present invention applies the 3D imaging module described in the above embodiment, and the 3D includes the following steps:

Use two imaging units to shoot the same scene at different angles at the same time;

color-separating the images sensed by the image sensors of the two imaging units respectively, and the images are represented as three primary color images of red, green and blue respectively;

Perform MTF calculation on the image area sensed by each pixel unit of the image sensors of the two imaging units to obtain the MTF value corresponding to the image area sensed by each pixel unit;

determining the object distance of the image sensed by each pixel unit according to the MTF value of the image sensed by each pixel unit;

determining the current shooting mode according to the object distance of the image sensed by each pixel unit;

If the current shooting mode is close-focus mode, perform the following steps:

According to the object distance of the image sensed by each pixel unit, respectively determine the best focus positions of the imaging lenses of the two imaging modules;

Determine the focus drive amounts of the imaging lenses of the two imaging modules respectively according to the best focus position;

Drive the two imaging modules to the best focus position of the imaging lens respectively according to the focus drive amount;

Combining the focused images captured by the two imaging units to obtain a 3D image;

If the current shooting mode is the far focus mode, perform the following steps:

According to the MTF value corresponding to the image area sensed by each pixel unit, determine the blur amount of the image sensed by the corresponding pixel unit;

According to the blur amount of the image sensed by each pixel unit, determine the blur correction amount of the image sensed by the corresponding pixel unit;

performing blur correction on the image sensed by each pixel unit according to the blur correction amount;

The blur-corrected images captured by the two imaging units are synthesized to obtain a 3D image.

The 3D imaging module and the 3D imaging method determine the object distance of the object by means of software calculation and simulation, and determine the current shooting mode according to the situation of the object distance, and select the software calculation method or the way of driving the imaging lens according to the shooting mode Focusing can achieve a clear-focus image regardless of the close-focus or far-focus shooting mode, and the images of the same scene captured by the two imaging units at the same time are synthesized, so a clearer 3D image can be obtained.

In addition, those skilled in the art can also make other changes within the spirit of the present invention. Of course, these changes made according to the spirit of the present invention should be included within the scope of protection claimed by the present invention.

Claims (10)

1. 3D imaging modules, it comprises first image-generating unit and second image-generating unit, a memory that is connected with described first image-generating unit and described second image-generating unit, a color-separated unit that is connected with described memory, a processor that is connected with described color-separated unit, a graphics processing unit that is connected with described processor, first driver and second driver of being connected with described processor respectively, and an image synthesis unit; Described first image-generating unit and described second imaging are used for catching with different angles simultaneously the image of Same Scene; Described memory is for storing the image that described first image-generating unit and described second image-generating unit catch; Described color-separated unit is used for the image that described first image-generating unit and described second image-generating unit capture is represented with Red Green Blue respectively; Described processor is used for the image that respectively described first image-forming module and described second image-forming module is captured is carried out the MTF computing, determine the current shooting pattern according to operation result, and control described graphics processing unit or described first driver and second driver according to determined current shooting model selection; Described graphics processing unit is used for by the image processing method formula image that described first image-generating unit and described second image-generating unit capture being carried out fuzzy correction out of focus; Described first driver and described second driver are respectively applied to described first image-generating unit and described second image-generating unit are focused; Described image synthesis unit is used for will described first image-generating unit and synthesizing through described image processing or to defocused image of catching of described second image-generating unit, obtains 3D rendering.

2. 3D imaging modules as claimed in claim 1, it is characterized in that: described first image-generating unit comprises first image sensor of first sampling image lens and and the optical axis alignment of described first sampling image lens, described second image-generating unit comprises second image sensor of second sampling image lens and and the optical axis alignment of described second sampling image lens, and each in described first sampling image lens and described second sampling image lens includes at least one aspherical lens with positive optic angle degree.

3. 3D imaging modules as claimed in claim 2 is characterized in that: described processor comprises a MTF computing module, object distance computing module, object distance judge module, fuzzy quantity computing module, fuzzy correction computing module, focusing position computing module and the computing module of driving amount one by one; The image-region that described MTF computing module is used for each pixel cell on described first image sensor and described second image sensor is sensed carries out the MTF computing, obtains the mtf value of corresponding region; Described object distance computing module is used for the operation result according to described MTF computing module, determines the object distance of the image that each pixel cell senses; Described object distance judge module is used for the operation result according to described object distance computing module, determines current screening-mode; Described fuzzy quantity computing module is used for the operation result according to described MTF computing module, determine this resulting mtf value of each pixel cell computing and homologue apart from the difference of internal standard mtf value, and determine the fuzzy quantity of the image that each pixel cell senses according to this difference; Described fuzzy correction computing module is used for according to the resulting fuzzy quantity of described fuzzy quantity computing module, determines the image that each pixel cell is sensed is blured the correction of revising; Described focusing position computing module is used for the operation result according to described object distance computing module, determines the best focusing position of described first sampling image lens and described second sampling image lens respectively; Described driving amount computing module is used for the best focusing position according to resulting described first sampling image lens of described object distance computing module and described second sampling image lens, determines the focusing driving amount of described first sampling image lens and described second sampling image lens.

4. 3D imaging modules as claimed in claim 3, it is characterized in that: described MTF computing module carries out the mtf value computing respectively to the three primary colors image of each pixel cell correspondence.

5. 3D imaging modules as claimed in claim 3, it is characterized in that: described object distance judge module is done comprehensive computing with the operation result of described object distance computing module, and result that will this comprehensive computing and a default standard value compare, and determines the current shooting pattern according to comparative result.

6. 3D imaging modules as claimed in claim 5, it is characterized in that: described comprehensive computing is for to sample to the object distance of described object distance computing module image that resulting each pixel cell senses, and obtains object distance sign amount for the distance that characterizes current shooting main target thing according to the data operation of sampling.

7. 3D imaging modules as claimed in claim 6, it is characterized in that: described default standard value is 40cm, if described object distance sign amount is greater than 40cm, then described object distance judge module judges that the current shooting pattern is burnt pattern far away, if described object distance sign amount is less than or equal to 40cm, then described object distance judge module judges that the current shooting pattern is nearly burnt pattern.

8. 3D imaging modules as claimed in claim 7 is characterized in that: when the current shooting pattern was burnt pattern far away, described fuzzy quantity computing module function was opened, and when the current shooting pattern was nearly burnt pattern, described focusing position computing module function was opened.

9. 3D imaging modules as claimed in claim 1, it is characterized in that: described first driver and described second driver are the piezoelectric type motor.

10. 3D formation method, it comprises the steps:

With different angles Same Scene is taken respectively simultaneously with two image-generating units;

The image that the image sensor of described two image-generating units is sensed carries out color-separated respectively;

The image-region that each pixel cells of image sensor of described two image-generating units is sensed carries out the MTF computing, obtains the mtf value of the image-region correspondence that each pixel cell senses;

The mtf value of the image that described each pixel cell of foundation senses is determined the object distance of the image that described each pixel cell senses;

The object distance of the image that described each pixel cell of foundation senses is determined the current shooting pattern;

If the current shooting pattern is nearly burnt pattern, then carry out the following step:

The object distance of the image that described each pixel cell of foundation senses, the best focusing position of the sampling image lens of definite described two image-forming modules respectively;

Determine the focusing driving amount of the sampling image lens of described two image-forming modules respectively according to described best focusing position;

Drive the sampling image lens of described two image-forming modules respectively according to described focusing driving amount to best focusing position;

The warp that described two image-generating units are caught synthesizes defocused image, obtains 3D rendering;

If the current shooting pattern is burnt pattern far away, then carry out the following step:

The mtf value of the image-region correspondence that senses according to described each pixel cell is determined the fuzzy quantity of the image that the respective pixel unit senses;

The fuzzy quantity of the image that senses according to described each pixel cell is determined the fuzzy correction of the image that the respective pixel unit senses;

According to described fuzzy correction, the image that each pixel cell is sensed blurs correction;

The fuzzy revised image of warp that described two image-generating units are caught synthesizes, and obtains 3D rendering.

Images