JP2010103949A - Apparatus, method and program for photographing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To adjust a setting variation of a zoom position of a compound eye lens more easily than before.
A distance between feature points obtained from one of viewpoint images SR and SL (distance between feature points) and a distance between corresponding points obtained from the other of images SR and SL (distance between corresponding points). And determine whether or not they substantially match. If No, the zoom position of one or both imaging systems is finely adjusted so that the distance between the two points obtained from the images of both imaging systems is substantially matched.
[Selection] Figure 9

Description

  The present invention relates to a zoom magnification adjustment technique for an apparatus that captures a multi-viewpoint image.

  In Japanese Patent Laid-Open No. 2004-260688, after an imaging system is incorporated into a compound-eye imaging system, errors occurring between the imaging systems, for example, a chart in each imaging system, a focal length, a subject distance, and a convergence angle are fixed. Shooting while changing the combination, read the output image at the time of shooting, measure the optical axis shift and magnification shift of the shooting system from the read result, and write the measured values to the storage means in a matrix. The storage means sends back the stored data corresponding to the transmitted information signal to the signal processing circuit. The signal processing circuit synthesizes the two images by correcting the registration deviation in the two image signals based on the stored data sent, and the processed composite image signal is output to the recording means and the output means to the outside. Is output. As a result, even when the focal length, the F number, the optical axis angle of the photographic lens, and the like between the plurality of photographic lenses used vary due to variations, the difference between the photographic lenses can be easily corrected.

  In Patent Document 2, of two zoom lenses arranged in a pair of left and right, one zoom lens (master side) directly controls the motor driver by operating the zoom switch to drive the zoom lens in the tele direction or the wide direction. The other zoom lens (slave side) is driven by a control signal corresponding to the zoom position error voltage (voltage indicating the error amount between the zoom lens and the zoom lens) generated by the differential amplifier and comparator. And the second variable power lens is driven in a direction to reduce the error amount. Either the left or right zoom lens is the master side, and the other is the slave side. By making the slave-side zoom lens follow the movement of the master-side zoom lens, the left and right focal lengths are matched with high accuracy. Suppose you can.

  In Patent Document 3, the zoom positions of the left and right zoom lenses are respectively detected by a position detector as the position of a zoom lens, and the zoom position data is digitized by an AD converter and then corrected by an LUT. The LUT is an input / output conversion table in which an output value corresponding to a digital value of an input signal is stored in advance, and matches the output characteristic of the position detector with a linear reference zoom function. By correcting the zoom position data of at least one of the position detectors that detect the left and right zoom positions with a look-up table and making the output characteristics of both uniform, the left and right shooting magnifications can be matched with high accuracy. Suppose you can.

In addition, patent documents 4-9 are mentioned as prior art relevant to this invention.
JP 7-87385 A JP-A-9-127400 Japanese Patent Laid-Open No. 9-187039 JP-A-8-294143 Japanese Patent Laid-Open No. 9-153148 JP 2003-284098 A Japanese Patent Laid-Open No. 5-28246 JP 7-49944 A JP-A-7-336669

  When shooting with a compound-eye camera to perform stereoscopic shooting or panoramic shooting, if the mounted lens includes a zoom lens, in addition to variations in individual lens differences, individual imaging systems when changing the zoom position There are also variations in settings. To suppress these variations, high-precision adjustment is necessary.

  In this respect, Patent Document 1 is complicated because it is necessary to repeat photographing under different conditions, and it is not possible to deal with a zoom lens in which variation varies for each setting. In Patent Document 2, it is necessary to adjust the error detection voltage with respect to the zoom position with high accuracy, which is complicated. In Patent Document 3, it is necessary to make a high-precision adjustment for each camera object for creating a lookup table, which is complicated.

  An object of the present invention is to adjust the setting variation of the zoom position of a compound eye lens more easily than in the past.

  An imaging apparatus according to the present invention includes at least a pair of features from an imaging unit capable of acquiring multi-viewpoint images taken from two or more viewpoints and any one viewpoint image taken by the imaging unit. A feature point extraction unit that extracts points; and at least one pair corresponding to at least one pair of feature points from viewpoint images other than the viewpoint image from which the feature point extraction unit has extracted feature points among the multi-viewpoint images captured by the imaging unit. A corresponding point extraction unit that extracts the corresponding points, a distance calculation that calculates a distance between feature points that is a distance between at least one pair of feature points, and a distance between corresponding points that is a distance between at least one pair of corresponding points And a control unit that controls the zoom magnification of the photographing unit so that the distance between the feature points and the distance between the corresponding points substantially coincide with each other.

  The feature point extraction unit extracts at least two pairs of feature points.

  The feature point extraction unit limits a range in which at least one pair of feature points is extracted according to the subject distance and / or zoom magnification of the imaging unit to the vicinity of the center of the multi-viewpoint image.

  The zoom magnification includes optical zoom and / or electronic zoom magnification.

  The multi-viewpoint image from which the feature point extraction unit extracts feature points includes a through image acquired at an arbitrary timing before the main photographing.

  Preferably, the line segment between the feature points for which the distance calculation unit calculates the distance between the feature points is orthogonal to the direction in which the viewpoints of the photographing unit are arranged.

  An imaging method according to the present invention is an imaging method executed by an imaging device including an imaging unit that acquires multi-view images captured from two or more viewpoints, and is an arbitrary one of multi-view images captured by the imaging unit. The step of extracting at least one pair of feature points from one viewpoint image, and at least one pair of feature points from viewpoint images other than the viewpoint image from which the feature points are extracted from the multi-viewpoint images captured by the photographing unit. Extracting at least one pair of corresponding points; calculating a distance between feature points that is a distance between at least one pair of feature points; and calculating a distance between corresponding points that is a distance between at least one pair of corresponding points; Controlling the zoom magnification of the photographing unit so that the distance between feature points and the distance between corresponding points substantially coincide.

  An imaging program for causing the imaging apparatus to execute this imaging method is also included in the present invention.

  According to the present invention, variations in zoom positions of different imaging systems can be easily adjusted, and shooting of failed images can be prevented.

<First Embodiment>
The best mode for carrying out the photographing apparatus according to the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a front perspective view showing an external configuration of a digital camera according to a first embodiment of the present invention. FIG. 2 is a rear perspective view showing the external configuration of the digital camera according to the first embodiment of the present invention.

  The digital camera 10 of the present embodiment is a digital camera (corresponding to the compound-eye digital camera of the present invention) provided with a plurality of (two are illustrated in FIG. 1) imaging means (also referred to as an imaging system). Images can be taken from a plurality of viewpoints (two left and right viewpoints are illustrated in FIG. 1).

  In the present embodiment, two imaging units are illustrated for convenience of explanation, but the present invention is not limited to this. Even three or more imaging means can be applied in the same manner. The arrangement of the imaging means (mainly the photographing lenses 14R and 14L) may not be one horizontal row along the horizontal direction, but may be two-dimensionally arranged. Stereo shooting, multi-viewpoint, or shooting in all directions may be used.

  The camera body 12 of the digital camera 10 is formed in a rectangular box shape, and a pair of photographing lenses 14R and 14L, a strobe 16 and the like are provided on the front surface thereof as shown in FIG. On the upper surface of the camera body 12, a shutter button 18, a power / mode switch 20, a mode dial 22, and the like are provided.

  On the other hand, as shown in FIG. 2, a monitor 24, a zoom button 26, a cross button 28, a MENU / OK button 30, a DISP button 32, a BACK button 34, a macro button 36, and the like are provided on the back of the camera body 12. Yes.

  Although not shown, the bottom of the camera body 12 is provided with a tripod screw hole, an openable / closable battery cover, and the like. A battery storage chamber for storing a battery, a memory is provided inside the battery cover. A memory card slot or the like for installing a card is provided.

  Each of the pair of left and right photographing lenses 14R and 14L is constituted by a retractable zoom lens, and has a macro photographing function (proximity photographing function). The photographing lenses 14R and 14L are extended from the camera body 12 when the digital camera 10 is turned on.

  In addition, since the zoom mechanism, the retracting mechanism, and the macro photographing mechanism in the photographing lens are well-known techniques, description of the specific configuration is omitted here.

  The strobe 16 is composed of a xenon tube, and emits light as necessary when shooting a dark subject or when backlit.

  The shutter button 18 is composed of a two-stroke switch that includes a so-called “half-press” and “full-press”. When the digital camera 10 shoots a still image (for example, when the still image shooting mode is selected with the mode dial 22 or when the still image shooting mode is selected from the menu), when the shutter button 18 is pressed halfway, a shooting preparation process, that is, AE ( Automatic exposure (AF), AF (auto focus), and AWB (automatic white balance) processing are performed, and when fully pressed, image capture / recording processing is performed. Also, during movie shooting (for example, when the movie shooting mode is selected with the mode dial 22 or when the movie shooting mode is selected from the menu), when the shutter button 18 is fully pressed, shooting of the movie is started, and when the shutter button 18 is fully pressed again, shooting is performed. Exit. Depending on the setting, the moving image can be shot while the shutter button 18 is fully pressed, and the shooting can be terminated when the full press is released. A shutter button dedicated to still image shooting and a shutter button dedicated to moving image shooting may be provided.

  The power / mode switch 20 functions as a power switch of the digital camera 10 and also functions as a switching unit that switches between the playback mode and the shooting mode of the digital camera 10, and includes “OFF position”, “playback position”, and “shooting position”. It is slidably provided between the two. The digital camera 10 is set to the playback mode when the power / mode switch 20 is positioned at the “playback position”, and is set to the shooting mode when it is positioned at the “shooting position”. Further, when it is positioned at the “OFF position”, the power is turned off.

  The mode dial 22 is used for setting the shooting mode. The mode dial 22 is rotatably provided on the upper surface of the camera body 12, and “2D still image position”, “2D moving image position”, “3D still image position”, “3D moving image position” are not shown by a click mechanism (not shown). It is provided so that it can be set. The digital camera 10 is set to a 2D still image shooting mode for shooting a 2D still image by setting the mode dial 22 to “2D still image position”, and the 2D / 3D mode switching flag 168 is set in the 2D mode. A flag indicating that there is something is set. Also, by setting the “2D moving image position”, the 2D moving image shooting mode for shooting a 2D moving image is set, and the 2D / 3D mode switching flag 168 is set with a flag indicating the 2D mode.

  Also, by setting the “3D still image position”, the 3D still image shooting mode for shooting a 3D still image is set, and the 2D / 3D mode switching flag 168 is set with a flag indicating the 3D mode. The Further, by setting the “3D moving image position”, the 3D moving image shooting mode for shooting a 3D moving image is set, and the 2D / 3D mode switching flag 168 is set with a flag indicating the 3D mode. The CPU 110 to be described later refers to the 2D / 3D mode switching flag 168 and grasps whether the mode is the 2D mode or the 3D mode.

  The monitor 24 is a display device such as a color liquid crystal panel in which a so-called lenticular lens having a semi-cylindrical lens group is arranged on the front surface. The monitor 24 is used as an image display unit for displaying captured images, and is used as a GUI when various settings are made. In addition, the monitor 24 sequentially displays images (through images) continuously captured by the imaging elements 134R / L during shooting, and is used as an electronic viewfinder.

  Here, a mechanism that enables stereoscopic display on the monitor 24 will be described with reference to the drawings.

  FIG. 3 is a diagram for explaining a mechanism that enables stereoscopic display on the monitor 24. A lenticular lens 24a is disposed on the front surface of the monitor 24 (in the z-axis direction where the viewer's viewpoint (left eye EL, right eye ER) is present). The lenticular lens 24a is configured by connecting a plurality of cylindrical convex lenses in the x-axis direction in FIG.

  The display area of the stereoscopic image displayed on the monitor 24 includes a strip image display area 24R for the right eye and a strip image display area 24L for the left eye. The strip image display area 24R for the right eye and the strip image display area 24L for the left eye each have an elongated strip shape in the y-axis direction in FIG. 3 of the screen, and are alternately arranged in the x-axis direction in FIG.

  Each convex lens of the lenticular lens 24a has a strip aggregate image display region 24c including a pair of right eye strip image display regions 24R and a left eye strip image display region 24L with reference to a given observation point of the viewer. It is formed at a position corresponding to.

  In FIG. 3, the strip image for the right eye displayed in the strip image display area 24R for the right eye of the monitor 24 is incident on the viewer's right eye ER due to the light refraction action of the lenticular lens 24a. The left eye strip image displayed in the left eye strip image display area 24L of the monitor 24 is incident on the left eye EL of the viewer due to the light refraction action of the lenticular lens 24a. Therefore, the viewer's right eye sees only the right-eye strip image, and the viewer's left eye sees only the left-eye strip image, and the right-eye image and the left eye are a set of these right-eye strip images. Stereoscopic viewing is possible by the left-right parallax of the left-eye image that is a set of ophthalmic strip images.

  The monitor 24 includes a display element capable of displaying both two-dimensional and three-dimensional such as liquid crystal and organic EL. Self-emission or a method of controlling the amount of light with a separate light source may be used. In addition, a system such as a polarized light system, an anaglyph, and a naked eye system are not limited. Moreover, the system which laminated | stacked the liquid crystal and organic EL in the multilayer may be sufficient.

  The zoom button 26 is used for a zoom operation of the photographing lenses 14R and 14L, and includes a zoom tele button for instructing zooming to the telephoto side and a zoom wide button for instructing zooming to the wide angle side.

  The cross button 28 is provided so that it can be pressed in four directions, up, down, left and right, and a function corresponding to the setting state of the camera is assigned to the button in each direction. For example, at the time of shooting, a function for switching on / off of the macro function is assigned to the left button, and a function for switching the strobe mode is assigned to the right button. In addition, a function for changing the brightness of the monitor 24 is assigned to the upper button, and a function for switching ON / OFF of the self-timer is assigned to the lower button. Further, during playback, a frame advance function is assigned to the left button, and a frame return function is assigned to the right button. Also, a function for changing the brightness of the monitor 24 is assigned to the upper button, and a function for deleting the image being reproduced is assigned to the lower button. In various settings, a function for moving the cursor displayed on the monitor 24 in the direction of each button is assigned.

  The MENU / OK button 30 is used to call a menu screen (MENU function), and is used to confirm selection contents, execute a process, etc. (OK function), and is assigned according to the setting state of the digital camera 10. Is switched.

  On the menu screen, for example, all adjustment items that the digital camera 10 has such as image quality adjustment such as exposure value, hue, ISO sensitivity, number of recorded pixels, self-timer setting, photometry method switching, and whether or not to use digital zoom. Settings are made. The digital camera 10 operates according to the conditions set on this menu screen.

  The DISP button 32 is used to input an instruction to switch the display contents of the monitor 24, and the BACK button 34 is used to input an instruction to cancel the input operation.

  The vertical / horizontal shooting switch button 36 is a button for instructing whether to perform vertical shooting or horizontal shooting (vertical shooting mode or horizontal shooting mode). The vertical / horizontal shooting detection circuit 166 detects whether shooting is performed in vertical shooting or horizontal shooting depending on the state of this button.

  FIG. 4 is a block diagram showing an electrical configuration of the digital camera 10 shown in FIGS. 1 and 2.

  As shown in FIG. 4, the digital camera 10 of the present embodiment is configured to acquire image signals from each of the two imaging systems, and includes a CPU 110, a 2D / 3D display switching unit 40, an entire image synthesis circuit 42, 3D image editing circuit 44, editing control input unit 46, 2D / 3D switching viewpoint number switching unit 48, thumbnail image creation circuit 50, cursor creation circuit 52, operation unit (shutter button 18, power / mode switch 20, mode dial 22, Zoom button 26, cross button 28, MENU / OK button 30, DISP button 32, BACK button 34, 2D / 3D mode switching button 36, etc.) 112, ROM 116, flash ROM 118, SDRAM 120, VRAM 122, taking lenses 14R, 14L, zoom lens Control units 124R, 124L, focus Control unit 126R, 126L, aperture control unit 128R, 128L, image sensor 134R, 134L, timing generator (TG) 136R, 136L, analog signal processing unit 138R, 138L, A / D converter 140R, 140L, image input controller 141R 141L, digital signal processing units 142R and 142L, AF detection unit 144, AE / AWB detection unit 146, 3D image generation unit 150, compression / decompression processing unit 152, media control unit 154, memory card 156, display control unit 158, A monitor 24, a power supply control unit 160, a battery 162, a strobe control unit 164, a strobe 16 and the like are provided.

  The imaging unit R on the right side in FIG. 1 mainly includes a photographic lens 14R, a zoom lens control unit 124R, a focus lens control unit 126R, an aperture control unit 128R, an image sensor 134R, a timing generator (TG) 136R, an analog signal processing unit 138R, An A / D converter 140R, an image input controller 141R, a digital signal processing unit 142R, and the like are included.

  1 mainly includes a photographing lens 14L, a zoom lens control unit 124L, a focus lens control unit 126L, an aperture control unit 128L, an image sensor 134L, a timing generator (TG) 136L, an analog signal processing unit 138L, An A / D converter 140L, an image input controller 141L, a digital signal processing unit 142L, and the like are included.

  The CPU 110 functions as a control unit that controls the overall operation of the camera, such as shooting, display, and recording, and controls each unit according to a predetermined control program based on an input from the operation unit 112. The operation unit 112 includes a shutter button 18, a power / mode switch 20, a mode dial 22, a zoom button 26, a cross button 28, a MENU / OK button 30, a DISP button 32, a BACK button 34, a macro button 36, and the like.

  The ROM 116 connected via the bus 114 stores a control program executed by the CPU 110 and various data necessary for control (AE / AF control cycle and the like described later). Various setting information relating to the operation of the digital camera 10 such as setting information is stored.

  The SDRAM 120 is used as a calculation work area for the CPU 110 and is also used as a temporary storage area for image data, and the VRAM 122 is used as a temporary storage area dedicated to image data for display.

  A pair of left and right photographing lenses 14R and 14L (also collectively referred to as photographing lens 14) are zoom lenses 130ZR and 130ZL (also collectively denoted as zoom lens 130Z), focus lenses 130FR and 130FL (collectively focus lenses). 130F), which includes the apertures 132R and 132L, and is arranged on the camera body 12 with a predetermined interval.

  The zoom lenses 130ZR and 130LR are driven by a zoom actuator (not shown) to move back and forth along the optical axis. The CPU 110 controls the position of the zoom lens by controlling the driving of the zoom actuator via the zoom lens control units 124R and 124L, and zooms the photographing lenses 14R and 14L.

  The focus lenses 130FR and 130FL are driven by a focus actuator (not shown) to move back and forth along the optical axis. The CPU 110 controls the position of the focus lens by controlling the drive of the focus actuator via the focus lens control units 126R and 126L, and performs focusing of the photographing lenses 14R and 14L.

  The diaphragms 132R and 132L are constituted by, for example, iris diaphragms, and are operated by being driven by a diaphragm actuator (not shown). The CPU 110 controls the aperture amount (aperture value) of the diaphragms 132R and 132L by controlling the driving of the diaphragm actuator via the diaphragm controllers 128R and 128L, and controls the amount of light incident on the image sensors 134R and 134L.

  The CPU 110 drives the left and right photographing lenses 14R and 14L in synchronism when driving the zoom lenses 130ZR and 130ZL, the focus lenses 130FR and 130FL, and the apertures 132R and 132L constituting the photographing lenses 14R and 14L. That is, the left and right photographing lenses 14R and 14L are always set to the same focal length (zoom magnification), and focus adjustment is performed so that the same subject is always in focus. In addition, the aperture is adjusted so that the same incident light amount (aperture value) is always obtained.

  The image sensors 134R and 134L are composed of color CCDs having a predetermined color filter array. In the CCD, a large number of photodiodes are two-dimensionally arranged on the light receiving surface. The optical image of the subject formed on the light receiving surface of the CCD by the photographing lenses 14R and 14L is converted into signal charges corresponding to the amount of incident light by the photodiode. The signal charges accumulated in the respective photodiodes are sequentially read out from the image sensors 134R and 134L as voltage signals (image signals) corresponding to the signal charges based on the drive pulses given from the TGs 136R and 136L in accordance with instructions from the CPU 110.

  The imaging elements 134R and 134L have an electronic shutter function, and the exposure time (shutter speed) is controlled by controlling the charge accumulation time in the photodiode.

  In the present embodiment, a CCD is used as the image sensor, but an image sensor having another configuration such as a CMOS sensor can also be used.

  Analog signal processing units 138R and 138L are correlated double sampling circuits (CDS) for removing reset noise (low frequency) included in the image signals output from the image sensors 134R and 134L, amplify the image signals, and are constant An AGS circuit for controlling the magnitude of the level is included, and the image signals output from the image sensors 134R and 134L are subjected to correlated double sampling processing and amplified.

  The A / D converters 140R and 140L convert the analog image signals output from the analog signal processing units 138R and 138L into digital image signals.

  The image input controllers 141R and 141L take in the image signals output from the A / D converters 140R and 140L and store them in the SDRAM 120.

  The digital signal processing units 142R and 142L take in the image signal stored in the SDRAM 120 in accordance with a command from the CPU 110, perform predetermined signal processing, and generate a YUV signal including the luminance signal Y and the color difference signals Cr and Cb.

  FIG. 5 is a block diagram showing a schematic configuration of the digital signal processing units 142R and 142L.

  As shown in FIG. 5, the digital signal processing units 142R and 142L include a white balance gain calculation circuit 142a, an offset correction circuit 142b, a gain correction circuit 142c, a gamma correction circuit 142d, an RGB interpolation calculation unit 142e, and an RGB / YC conversion circuit 142f. , A noise filter 142g, a contour correction circuit 142h, a color difference matrix circuit 142i, and a light source type determination circuit 142j.

  The white balance gain calculation circuit 142a takes in the integrated value calculated by the AE / AWB detection unit 146 and calculates a gain value for white balance adjustment.

  The offset correction circuit 142b performs an offset process on the image signals of R, G, and B colors captured via the image input controllers 141R and 141L.

  The gain correction circuit 142c takes in the image signal that has been subjected to the offset processing, and performs white balance adjustment using the gain value calculated by the white balance gain calculation circuit 142a.

  The gamma correction circuit 142d takes in an image signal that has undergone white balance adjustment, and performs gamma correction using a predetermined γ value.

  The RGB interpolation calculation unit 142e interpolates the gamma-corrected R, G, and B color signals to obtain R, G, and B3 color signals at each pixel position. That is, in the case of a single-plate image sensor, each pixel outputs only one color signal of R, G, and B. Therefore, a color that is not output is obtained from the color signals of surrounding pixels by interpolation calculation. For example, in a pixel that outputs R, the level of G and B color signals at this pixel position is determined by interpolation from the G and B signals of surrounding pixels. As described above, since the RGB interpolation calculation is specific to the single-plate image sensor, it is not necessary when a three-plate image sensor is used as the image sensor 134.

  The RGB / YC conversion circuit 142f generates a luminance signal Y and color difference signals Cr and Cb from the R, G, and B signals after the RGB interpolation calculation.

  The noise filter 142g performs noise reduction processing on the luminance signal Y and the color difference signals Cr and Cb generated by the RGB / YC conversion circuit 142f.

  The contour correction circuit 142h performs contour correction processing on the luminance signal Y after noise reduction, and outputs a luminance signal Y ′ whose contour has been corrected.

  On the other hand, the color difference matrix circuit 142i performs color correction by multiplying the color difference signals Cr and Cb after noise reduction by the color difference matrix (C-MTX). That is, the color difference matrix circuit 142i is provided with a plurality of types of color difference matrices corresponding to the light sources, and the color difference matrix to be used is switched according to the light source type obtained by the light source type determination circuit 142j. The input color difference signals Cr and Cb are multiplied, and color-tone-corrected color difference signals Cr ′ and Cb ′ are output.

  The light source type determination circuit 142j determines the light source type by taking in the integrated value calculated by the AE / AWB detection unit 146, and outputs a color difference matrix selection signal to the color difference matrix circuit 142i.

  In the digital camera of this embodiment, the digital signal processing unit is configured by a hardware circuit as described above, but the same function as that of the hardware circuit can be configured by software.

  The AF detection unit 144 captures R, G, and B color image signals captured from one image input controller 141R, and calculates a focus evaluation value necessary for AF control. The AF detection unit 144 includes a high-pass filter that allows only a high-frequency component of the G signal to pass, an absolute value processing unit, a focus area extraction unit that extracts a signal within a predetermined focus area set on the screen, and a focus area An integration unit for integrating the absolute value data is included, and the absolute value data in the focus area integrated by the integration unit is output to the CPU 110 as a focus evaluation value.

  During the AF control, the CPU 110 searches for a position where the focus evaluation value output from the AF detection unit 144 is maximized, and moves the focus lenses 130FR and 130FL to the position to perform focusing on the main subject. . That is, during the AF control, the CPU 110 first moves the focus lenses 130FR and 130FL from the close range to the infinity, sequentially acquires the focus evaluation value from the AF detection unit 144 in the moving process, and the focus evaluation value becomes maximum. Detect position. Then, the position where the detected focus evaluation value is maximum is determined as the in-focus position, and the focus lenses 130FR and 130FL are moved to that position. Thereby, the subject (main subject) located in the focus area is focused.

  The AE / AWB detection unit 146 takes in image signals of R, G, and B colors taken from one image input controller 141R, and calculates an integrated value necessary for AE control and AWB control. That is, the AE / AWB detection unit 146 divides one screen into a plurality of areas (for example, 8 × 8 = 64 areas), and calculates an integrated value of R, G, and B signals for each divided area.

  At the time of AE control, the CPU 110 acquires an integrated value of R, G, B signals for each area calculated by the AE / AWB detection unit 146, obtains the brightness (photometric value) of the subject, and obtains an appropriate exposure amount. Set the exposure to obtain That is, sensitivity, aperture value, shutter speed, and necessity of strobe light emission are set.

  In addition, during the AWB control, the CPU 110 calculates the integrated values of the R, G, and B signals for each area calculated by the AE / AWB detection unit 146, the white balance gain calculation circuit 142a and the light source type determination circuit 142j of the digital signal processing unit 142. Add to.

  The white balance gain calculation circuit 142a calculates a gain value for white balance adjustment based on the integrated value calculated by the AE / AWB detection unit 146.

  The light source type determination circuit 142j detects the light source type based on the integrated value calculated by the AE / AWB detection unit 146.

  The compression / decompression processing unit 152 performs compression processing in a predetermined format on the input image data in accordance with a command from the CPU 110 to generate compressed image data. Further, in accordance with a command from the CPU 110, the input compressed image data is subjected to a decompression process in a predetermined format to generate uncompressed image data. In the digital camera 10 according to the present embodiment, a still image is subjected to compression processing conforming to the JPEG standard, and a moving image is subjected to compression processing conforming to the MPEG2 standard.

  The media control unit 154 controls reading / writing of data with respect to the memory card 156 in accordance with a command from the CPU 110.

  The display control unit 158 controls display on the monitor 24 in accordance with a command from the CPU 110. That is, in accordance with a command from the CPU 110, the input image signal is converted into a video signal (for example, NTSC signal, PAL signal, SCAM signal) to be displayed on the monitor 24 and output to the monitor 24. The graphic information is output to the monitor 24.

  The power control unit 160 controls power supply from the battery 162 to each unit in accordance with a command from the CPU 110.

  The strobe control unit 164 controls the light emission of the strobe 16 in accordance with a command from the CPU 110.

  The height detection unit 38 is a circuit for detecting a shooting height (distance) from a reference plane (for example, the ground).

  The vertical shooting / horizontal shooting detection circuit 166 detects whether the shooting is vertical shooting or horizontal shooting depending on the state of the vertical shooting / horizontal shooting switching button 36.

  The 2D / 3D mode switching flag 168 is set with a flag indicating the 2D mode or the 3D mode.

  The distance calculation unit 7 first obtains corresponding points corresponding to each other on the images SR and SL acquired by the imaging elements 134R and 134L, respectively, using a stereo matching technique. What is necessary is just to apply the method of calculating | requiring a corresponding point. For example, a corresponding matrix is obtained by extracting a partial matrix (for example, 3 × 3 pixels) from the images SR and SL and obtaining a correlation value. Specifically, a boundary portion (edge component) where the luminance / color difference changes from one of the images SR and SL is used as a feature point, and a portion having a similar edge component in the other image is used as a corresponding point. Alternatively, each face may be detected from the images SR and SL, and the top of the detected face area and the tip of the chin may be used as feature points or corresponding points of each image, and the feature point and corresponding point extraction methods are mutually dependent. There is no need to let them. In short, the feature point is not limited to an edge as long as a corresponding point can be found, and may be an arbitrary point.

  Then, the distance calculation unit 7 calculates the distance along the vertical axis between the feature points obtained in the image SR and the distance along the vertical axis between the corresponding points obtained in the image SL. However, this distance calculation method is employed when the photographing lenses 14R and 14L are arranged side by side in the horizontal direction. When the photographic lenses 14R and 14L are arranged side by side in the vertical direction, the distance calculation unit 7 determines the distance between the feature points obtained in the image SR along the horizontal axis and the corresponding points obtained in the image SL. Find the distance along the horizontal axis. This is because, when the arrangement direction of the photographic lenses coincides with the distance calculation direction, the parallax of the photographic lenses influences, and the relationship between the distance between the feature points and the distance between the corresponding points is from each of the photographic lenses 14R and 14L to the subject. This is because the relationship with the distance is not accurately reflected.

  In short, it is preferable that the distance calculation unit 7 obtains the distance between feature points and the distance between corresponding points along a direction orthogonal to the arrangement direction of the photographing lenses 14R and 14L. Of course, when the influence of parallax is small, such as when the subject is located relatively far from the camera 10, the distance calculation unit 7 determines the distance between the feature points on the two-dimensional plane regardless of the arrangement direction of the photographing lenses 14 </ b> R and 14 </ b> L. The distance between the distance and the corresponding point may be obtained.

  For example, the feature points P1-R and P2-R in FIG. 7A are obtained from the image SR in FIG. 6A, and the correspondence in FIG. 7B is obtained from the image SL in FIG. 6B. Assume that the points P1-L and P2-L are obtained.

  In this case, as shown in FIG. 8, the vertical distance DR between the feature points P1-R and P2-R and the vertical distance DL between the corresponding points P1-L and P2-L are calculated. The

  Note that the images for obtaining the feature points and the corresponding points are not limited to those shown in the figure, and the distance between the two points obtained from both images as a result is obtained from either image regardless of whether the feature points or the corresponding points are obtained. Can be easily understood. That is, the feature point may be obtained first from the image SR or the feature point may be obtained first from the image SL.

  Hereinafter, the flow of the zoom position adjustment process that is controlled by the main CPU 110 of the camera 10 will be described with reference to the flowchart of FIG. A program for causing the main CPU 110 to execute this processing is stored in the ROM 116, and the main CPU 110 reads this program into the SDRAM 120 for interpretation and execution.

  In S1, the CPU 110 controls the position of the zoom lens by controlling the driving of the zoom actuator via the zoom lens control units 124R and 124L in accordance with the operation of the zoom button 26, and zooms the photographing lenses 14R and 14L. I do.

  In S2, a through image is output.

  In S3, the distance between feature points obtained from one of the images SR and SL (distance between feature points) is compared with the distance between corresponding points obtained from the other of the images SR and SL (distance between corresponding points). Then, it is determined whether or not both substantially match. If Yes, the process proceeds to S4. If No, the process proceeds to S5. Note that the term “substantially match” includes not only the case where both match completely, but also the case where the absolute value of the difference between the two is equal to or less than a predetermined error threshold value stored in advance in the ROM 116.

  In S4, the zoom position adjustment of the photographing lenses 14R and 14L is finished.

  In S5, control is performed to move the zoom position of the photographic lens 14 that captures the image corresponding to the smaller one of the distances compared in S3 to the wide side, or the larger one of the distances compared in S3. Control is performed to move the zoom position of the photographic lens 14 that picks up the image to be telephoto to the tele side, and the zoom position of one of the image pickup systems is finely adjusted, so that the images of both image pickup systems can be adjusted. The obtained distance between the two points is substantially matched.

  Alternatively, the control is such that the zoom position of the photographing lens 14 of the imaging system that captures the image corresponding to the smaller one of the feature point distance and the corresponding point distance compared in S3 is moved to the wide side, and S3. Control may be performed such that the zoom position of the photographing lens of the imaging system that captures the image corresponding to the larger one of the compared distances is moved to the tele side. That is, by finely adjusting both zoom positions, the distance between the two points obtained from the images of both imaging systems is substantially matched.

  Alternatively, instead of the optical zoom for changing the zoom position of the photographing lens 14 or together with the optical zoom, the distance between the two points obtained from the images of both imaging systems is substantially matched by electronic zoom using image processing. Also good.

  Note that the purpose of this process is to obtain the same magnification image from different image pickup systems by adjusting the zoom position of the taking lens 14 of both image pickup systems when taking images from now on. Even if distance error adjustment is performed, there is a merit related to the through image display, but there is not much merit in the actual imaging. However, if the error of the zoom magnification of the images of both imaging systems that have already been photographed is adjusted after photographing, it is conceivable to perform error adjustment of the distance using only the electronic zoom.

  Through the above processing, variations in zoom positions of different imaging systems can be easily adjusted, and shooting of failed images can be prevented.

<Second Embodiment>
In the first embodiment, the distance calculation unit 7 may obtain a plurality of distances between the two points obtained from the images obtained from the respective imaging systems instead of one. In this case, the distance calculation unit 7 obtains a plurality of distances between feature points that are distances between two arbitrary feature points, and a distance between corresponding points that is a distance between two corresponding points corresponding to the two feature points for which the distances are obtained. Are determined, and it is determined whether or not the corresponding distance between feature points and the distance between corresponding points substantially match. If the distance between any feature points and the distance between corresponding points substantially match, the process proceeds to S4, and if the distance between at least one feature point and the distance between corresponding points does not substantially match, the process proceeds to S5.

For example, as shown in FIG. 10, four feature points P1-1-R, P1-2-R, P2-1-R, and P2-2-R are obtained from the image SR. , P1-1-L, P1-2-L, P2-1-L, P2 corresponding to the characteristic points of P1-1-R, P1-2-R, P2-1-R, and P2-2-R, respectively. Suppose that four corresponding points of 2-L are obtained.
For example, the distance calculation unit 7 calculates the distance DR-1 between feature points between P1-1-R and P1-2-R, and between feature points between P2-1-R and P2-2-R. While calculating | requiring distance DR-2, distance DL-1 between corresponding points between corresponding points P1-1-L and P1-2-L corresponding to P1-1-R and P1-2-R, and P2 Corresponding point distance DL-2 between corresponding points P2-1-L and P2-2-L corresponding to -1-R and P2-2R is obtained. Generally, when the feature points are n = 3 or more, _n C ₂ distances can be calculated, and the distance between each of the feature points may be obtained. Only the distance between feature points close to the direction is obtained.

  Then, CPU 110 determines whether DR-1 and DL-1 substantially match and DR-2 and DL-2 substantially match. That is, the difference Δ1 = | (DR−1) − (DL−1) | and the difference Δ2 = | (DR−2) − (DL−2) | are obtained, and both Δ1 and Δ2 are equal to or less than a predetermined threshold value. If yes, the process proceeds to S4, and if No, the process proceeds to S5.

  As described above, by adjusting the error of the distance between the plurality of sets of feature points / corresponding points, the determination of the error of the distance between the corresponding points between the images becomes precise, and the zoom adjustment becomes accurate.

<Third Embodiment>
In the first or second embodiment, when the distance from the photographing lens to the subject becomes very small, there is a possibility that the distance from one photographing lens to the subject and the distance from the other photographing lens to the subject are greatly different. .

  For example, as shown in FIG. 11A, the linear distances from the two photographing lenses 14R / L to the subject are xR and xL, respectively. If the subject is on a perpendicular extending from the center O of the straight line in which the two photographing lenses 14R / L are arranged, xR = xL. Alternatively, if the subject is sufficiently far from the center O, xR≈xL. In these cases, in the images SR and SL obtained from the two photographing lenses 14R / L, the subject images will be substantially the same size, and the distance between the feature points and the distance between the corresponding points will substantially coincide.

  However, as shown in FIG. 11B, the subject does not exist on the perpendicular extending from the center of the straight line where the two photographing lenses 14R / L are arranged, and the subject is not as in macro photography. When the distance r to the center O is very small, xR >> xL (or xR << xL), and the subject in the image obtained from the photographing lens 14 closer to the subject has an enlarged state similar to that of the high magnification zoom. Become. Therefore, when the feature point and the corresponding point are obtained from the subject image, the absolute value of the difference between the distance between the feature points and the distance between the corresponding points may exceed a predetermined error threshold range, and the two may not substantially coincide with each other. . In this case, according to the above-described processing, the zoom adjustment is performed by proceeding to S5. However, this is not the adjustment of the variation in the zoom position of the different imaging systems, but the adjustment of the subject due to the difference in the distance from the photographing lens 14 to the subject. The size is adjusted, and the formation of a correct stereoscopic image (a close one is close and a far one is far away) is prevented.

  Therefore, by limiting the detection range of the feature points to a region that has no or little influence on parallax and that is near the center of the photographic lens array line, the difference in distance from the photographic lens 14 to the subject can be reduced. Prevents malfunction of zoom position adjustment caused by it.

  Specifically, the CPU 110 stores a value obtained by converting position information when the focus lens 130F is in focus into a main subject distance or a distance to a subject measured by a known method such as triangulation, which is stored in the ROM 116. When the distance is equal to or smaller than the distance threshold (for example, the distance d between the photographing lenses), the feature point detection range is limited in the same direction as the arrangement direction of the photographing lenses. Alternatively, the feature point detection range may be limited when the macro shooting mode is set from the operation unit 112.

  The size and shape of the limited feature point detection range are arbitrary. For example, as shown in FIG. 12, the horizontal detection range of feature points is not within the entire images SR and SL, but within a range of a predetermined distance ST from the center X of the image (the center of the photographing lenses 14R and 14L). To do. As a result, only feature points that are substantially equidistant from the photographing lenses 14R and 14L are obtained, and malfunction of zoom position adjustment can be prevented. Note that the distance ST may be reduced in proportion to the distance r. This is because as the distance r becomes smaller, the influence of the shift of the zoom position due to the parallax becomes larger, and accordingly, the detection range where the influence of the parallax is smaller is narrowed.

  Note that the feature point detection range may always be limited to the area near the center of the image regardless of the distance to the subject. In this case, however, the feature point detection in the periphery of the image as in the second embodiment is performed. (For example, P1-1-R and P1-2-R in FIG. 10) cannot be performed. Therefore, when the distance to the subject is closer than a predetermined value, or when an equivalent shooting situation is set (for example, when the macro shooting mode is set from the operation unit 112, or when the zoom lens 130Z is at or near the wide end) The feature point detection range is limited, and when the distance to the subject is longer than a predetermined value or a shooting situation equivalent to that is set (for example, landscape shooting mode from the operation unit 112) Is set or when the zoom lens 130Z is moved to or near the tele end), it is preferable to enlarge the detection range of the feature points (for example, the entire image).

  Also, the focus evaluation value calculation area and the multi-point distance measurement value calculation point are usually set near the center of the screen, and when such an object near the center of the screen is measured. The detection range of feature points is limited depending on whether or not an object near the center of the screen is at a short distance. When it is desired to limit the feature point detection range depending on whether or not an object that does not exist near the center of the screen (for example, FIG. 11B) is at a short distance, the focus evaluation value calculation area and the distance measurement value calculation point are set in advance. It is necessary to extend not only near the center of the screen but also to other peripheral areas.

<Other embodiments>
The invention described in the first to third embodiments can be applied not only to a 3D camera, but also to a panoramic shooting camera. Applicable to the above).

Front perspective view of digital camera Rear perspective view of digital camera The figure for demonstrating the structure in which a stereoscopic display is possible on a monitor Block diagram showing the electrical configuration of the digital camera Block diagram showing schematic configuration of digital signal processor Diagram illustrating images obtained by each imaging system Diagram illustrating feature points and corresponding points Diagram illustrating distance between feature points and distance between corresponding points Zoom position adjustment process flowchart Diagram illustrating multiple feature points and corresponding points A diagram illustrating subjects at equal distances or different distances from each imaging system A diagram exemplifying a restricted area for detecting feature points

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Compound eye digital camera, 12 ... Camera body, 14L, 14R ... Shooting lens, 16 ... Strobe, 18 ... Shutter button, 20 ... Mode switch, 22 ... Mode dial, 24 ... Monitor, 24L ... Strip image display area for left eye 24R ... Right-eye strip image display area, 24a ... Lenticular lens, 24c ... Strip collection image display area, 26 ... Zoom button, 28 ... Cross button, 30 ... button, 32 ... button, 34 ... button, 36 ... Macro Button ... 36 ... Switching button 36 ... Switching button 38 ... Detection unit 40 ... 2D / 3D display switching unit 42 ... Whole image composition circuit 44 ... 3D image editing circuit 46 ... Editing control input unit 48 ... 2D / 3D switching viewpoint number switching unit, 50... Thumbnail image generation circuit, 52... Cursor generation circuit, 112. , 124R ... zoom lens control unit, 126L, 126R ... focus lens control unit, 128L, 128R ... control unit, 130FR ... focus lens, 130ZR ... zoom lens, 134L, 134R ... image sensor, 138L, 138R ... analog signal processing unit, 140L, 140R ... converters, 141L, 141R ... image input controller, 142L, 142R ... digital signal processing unit, 142a ... white balance gain calculation circuit, 142b ... offset correction circuit, 142c ... gain correction circuit, 142d ... gamma correction circuit, 142e: Interpolation calculation unit, 142f ... Conversion circuit, 142g ... Noise filter, 142h ... Contour correction circuit, 142i ... Color difference matrix circuit, 142j ... Light source type determination circuit, 144, 146 ... Detection unit, 150 ... Image generation unit, 152 Compression / decompression processing unit, 154 ... media control unit, 156 ... memory card, 158 ... display control unit, 160 ... power supply control unit, 162 ... battery, 164 ... strobe control unit, 166 ... detection circuit, 168 ... mode switching flag, R, L ... Imaging means (imaging system), 7 ... Distance calculator

Claims (8)

An imaging unit capable of acquiring multi-viewpoint images taken from two or more viewpoints;
A feature point extraction unit that extracts at least one pair of feature points from any one viewpoint image among the multi-viewpoint images captured by the imaging unit;
Corresponding to extract at least one pair of corresponding points corresponding to the at least one pair of feature points from viewpoint images other than the viewpoint image from which the feature point extraction unit has extracted the feature points among the multi-viewpoint images captured by the photographing unit. A point extractor;
A distance calculation unit that calculates a distance between feature points that is a distance between the at least one pair of feature points and a distance between corresponding points that is a distance between the at least one pair of corresponding points;
A control unit that controls a zoom magnification of the imaging unit such that the distance between the feature points and the distance between the corresponding points substantially match;
An imaging device comprising:   The imaging device according to claim 1, wherein the feature point extraction unit extracts at least two pairs of feature points.   3. The feature point extraction unit limits a range in which the at least one pair of feature points is extracted in the vicinity of the center of the multi-viewpoint image according to a subject distance and / or zoom magnification of the imaging unit. Shooting device.   The photographing apparatus according to claim 1, wherein the zoom magnification includes an optical zoom and / or electronic zoom magnification.   The imaging device according to claim 1, wherein the multi-viewpoint image from which the feature point extraction unit extracts a feature point includes a through image acquired at an arbitrary timing before actual imaging.   The imaging device according to claim 1, wherein a line segment between feature points for which the distance calculation unit calculates the distance between the feature points is orthogonal to a direction in which the viewpoints of the imaging unit are arranged. An imaging method executed by an imaging apparatus including an imaging unit that acquires multi-viewpoint images taken from two or more viewpoints,
Extracting at least one pair of feature points from any one viewpoint image among the multi-viewpoint images captured by the imaging unit;
Extracting at least one pair of corresponding points corresponding to the at least one pair of feature points from viewpoint images other than the viewpoint image from which the feature points have been extracted from the multi-viewpoint images captured by the photographing unit;
Calculating a distance between feature points that is a distance between the at least one pair of feature points and a distance between corresponding points that is a distance between the at least one pair of corresponding points;
Controlling the zoom magnification of the photographing unit so that the distance between the feature points and the distance between the corresponding points substantially coincide with each other;
Including shooting method.   An imaging program for causing an imaging apparatus to execute the imaging method according to claim 7.