JP2006187032A - Stereoscopic photographing apparatus and photographing method of same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To facilitate stereoscopic photographing by presenting not only a stereoscopic visual field but also a processing region for performing stereoscopic matching to a photographer. <P>SOLUTION: The stereoscopic photographing apparatus comprises a stereoscopic image pickup means for picking up an image of a photographic object from a plurality of viewpoints, a display means for displaying an image picked up by the stereoscopic image pickup means as an output image on a display screen, a computation means for applying stereoscopic matching processing to a first closed region, predetermined before picking up the image of the photographic object, within the image picked up by the stereoscopic image pickup means; and a means for displaying on the display screen of the display means an eyemark showing a second closed region included in the first closed region as an eyemark for capturing the photographic object within the first closed region when picking up the image of the photographic object. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a three-dimensional information acquisition method and apparatus for acquiring three-dimensional information from a stereo image, and a stereo photographing apparatus applicable to the apparatus and a photographing method of the stereo photographing apparatus.

  Conventionally, stereo measurement is known as a method for acquiring three-dimensional information without contact. This three-dimensional information acquisition method by stereo measurement acquires images from different viewpoints, and acquires three-dimensional information from the positional relationship between the viewpoints and the difference in appearance of the images.

  For example, as shown in FIG. 13, a plurality of images 2 from a plurality of (in this case, two) viewpoints are acquired, and one of the images is used as a reference image 3 (in this case, the left image in the figure). . Then, the difference between the position A1 on the coordinates of the reference image 3 of the point A on the photographing object 4 and the position A2 on the coordinates of the other image 2 (hereinafter referred to as parallax), and a triangle from the viewpoint position and the line-of-sight direction. Based on the principle of surveying, the distance to point A is calculated. Similarly, the distance to the point B on the subject 4 can be calculated using the position B1 on the coordinates of the reference image 3 and the position B2 on the coordinates of the other image 2. As described above, usually, when acquiring three-dimensional information using a stereo image, it is recognized by corresponding point search which point on the image serving as one reference corresponds to which point on the other image.

  A technique for recognizing to which point on the other image the point on the image serving as the reference corresponds by such a corresponding point search is known (for example, see Patent Document 1).

  Further, in order to capture a stereo image from a plurality of viewpoints with a general camera, a stereo adapter in which an optical path is divided into two by a mirror in a normal imaging system is widely used.

  In addition, there is a technique in which a function for switching a stereo photographing field of view according to the attachment of such a souffle adapter is provided in the finder (see, for example, Patent Document 2).

In order to prevent an input error of 3D information, a technique for displaying an area where 3D information can be obtained has been proposed (for example, a patent). Reference 3).
JP-A-6-215111 JP-A-7-64216 JP 2000-283739 A

  However, the conventional technique disclosed in Patent Document 2 is premised on film photography, and stereo matching is performed on a digital image photographed by attaching a stereo adapter to a digital camera using a computer. No consideration is given to a shape recognition system that uses a digital camera that performs shape recognition of a subject.

  Further, in the case of the conventional technique disclosed in Patent Document 3, since there is one imaging unit, the display position of the area where the three-dimensional information can be obtained is located at the approximate center of the output screen displayed on the display unit. It was. However, when shooting with a stereo imaging means, the interrelationship between a plurality of viewpoints, that is, the area where stereo matching is performed and the area where three-dimensional information can be obtained changes depending on the distance, convergence angle and angle of view of different viewpoints. Therefore, an appropriate area for capturing a shooting target is significantly different from that for shooting from a single viewpoint, and it is difficult for a photographer to recognize an appropriate area.

  For example, as shown in FIGS. 14 (a) and 14 (b), when the convergence angles are different, it is different where the imaging object needs to be captured in the field of view from the left and right viewpoints 51 and 52, respectively. That is, a superimposition area 33 (shown by hatching in the figure) that is an imaging space that can be imaged from each of the plurality of viewpoints 51 and 52 is an area where stereo matching is performed, and an area where three-dimensional information can be obtained. Since the angle varies depending on the convergence angle, the display position of the area suitable for capturing the object to be imaged is not simply set at the approximate center of the output screen.

  Furthermore, the appropriate area for capturing the object to be imaged is also affected by the distance and the angle of view from each viewpoint.

  By the way, since many digital cameras in recent years have an electronic viewfinder mounted on the back side of the camera, it is easy to check the stereo field of view.

  However, in the shape recognition system using a digital camera as described above, the processing area when performing stereo matching using a computer is the shape of the stereo adapter attached to the digital camera, the lens distortion, the capability of the computing device, and the requirements. Because of the relationship between the quality of the three-dimensional information to be displayed, it does not necessarily match the stereo field of view displayed on the viewfinder.

  For this reason, in the shape recognition system using the conventional digital camera as described above, there is a problem that it is difficult to photograph the subject in the three-dimensional shape recognizable region.

  In addition, in the shape recognition system using the conventional digital camera as described above, the electronic viewfinder is used when the inside of the stereo adapter attached to the digital camera is displayed in black, when shooting a dark place or a black subject. There is a problem that it is difficult to confirm the stereo field of view itself.

  For the above reason, in the shape recognition system using the conventional digital camera as described above, the image region actually used is limited to the inside of the stereo visual field.

  Further, even in the case of a stereo imaging system that does not use a stereo adapter, stereo matching is not performed on the entire captured image, so that the stereo field of view and the processing area for performing stereo matching do not match.

  The present invention pays attention to such points, and when a stereo image is captured using a stereo image capturing means, for example, a portion (stereo matching region) that can be used for stereo matching processing in the captured image is manipulated. By displaying an area mark such as a frame line on the electronic finder built in the stereo image pickup means or an external display screen as a display means so that the person can determine the object to be photographed within the processing area. An object of the present invention is to provide a stereo photographing device and a photographing method of the stereo photographing device that can facilitate stereo photographing by presenting not only a stereo field of view but also a processing region for performing stereo matching to the photographer. To do.

[Definition of terms]
Note that “stereo shooting” in this specification means shooting from a plurality of viewpoints.

  Furthermore, “stereo shooting” in this specification includes a case where a single shooting device is used to obtain images from a plurality of viewpoints using a stereo converter or the like, and a case where a plurality of shooting devices are used.

  Further, “an image obtained by stereo shooting” in this specification means an image from a plurality of viewpoints. In this case, not only a plurality of images from a plurality of photographing apparatuses but also one in which images of a plurality of viewpoints are embedded in one image.

  Furthermore, when the imaging target is imaged from a plurality of viewpoints, the imaging space that can be imaged from each of the plurality of viewpoints, that is, the imaging angle of view has an overlapping region. In this specification, this region is referred to as a “superimposed region”.

  The “calibration pattern” in this specification means an object whose shape and / or surface attribute is known. This calibration pattern is photographed and used to calculate camera parameters such as lens distortion of the photographing system used for photographing from the degree of distortion of the calibration pattern on the photographed image. There are a case where a “calibration pattern” having a three-dimensional shape is used to align the conditions for calculating camera parameters, and a case where a plurality of “calibration patterns” having a two-dimensional shape are photographed. It does not depend on the method.

  One aspect of the stereo imaging device of the present invention is a stereo image capturing unit that captures images to be captured from a plurality of viewpoints, and a display unit that displays an image captured by the stereo image capturing unit as an output image on a display screen. Calculating means for performing a stereo matching process on a first closed region which is a region in an image captured by the stereo imaging unit and which has been determined in advance before imaging the imaging target; When the imaging object is imaged, a mark indicating the second closed area included in the first closed area is used as a mark for capturing the shooting object in the first closed area. Means for displaying on a display screen.

  Another aspect of the stereo imaging device of the present invention is a stereo imaging device that captures images of the imaging target from a plurality of viewpoints in order to perform stereo matching processing for obtaining three-dimensional information of the imaging target. Stereo image capturing means for capturing the image to be imaged, display means for displaying an image captured by the stereo image capturing means as an output image on a display screen, and “in the image captured by the stereo image capturing means” When the imaging target is imaged with a mark indicating the second closed area included in the “first closed area determined in advance before imaging the imaging target as the area to be subjected to the stereo matching process” in FIG. And a means for displaying on the display screen of the display means as a mark for capturing the photographing object in the first closed region. To.

  Further, according to one aspect of the imaging method of the stereo imaging device of the present invention, an image to be subjected to stereo matching processing for obtaining three-dimensional information of an imaging target is captured from a plurality of viewpoints, and the captured images are displayed on a display screen. In the imaging method of the stereo imaging device to be displayed, “a first closed area that is determined in advance before imaging the imaging target as an area to be subjected to the stereo matching process in the image captured by the stereo imaging unit”. Displaying a mark indicating the second closed region included in the “region” on the display screen as a mark for capturing the shooting target in the first closed region when imaging the shooting target. Features.

  According to the present invention, when a stereo image is captured using the stereo image capturing unit, for example, the operator can determine a portion (stereo matching region) that can be used for stereo matching processing in the captured image. By displaying area marks such as frame lines on the electronic viewfinder built in the stereo image pickup means or on the screen of an external display as a display means so that the shooting target can be put in the processing area, not only the stereo field of view By presenting the processing area for performing stereo matching to the photographer, it is possible to provide a stereo photographing apparatus and a photographing method for the stereo photographing apparatus that can facilitate stereo photographing.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[First Embodiment]
FIG. 1A is a diagram showing a configuration of the stereo photographing apparatus according to the first embodiment of the present invention and a state where the photographing object 4 is photographed.

  That is, the stereo photographing apparatus according to the first embodiment of the present invention is a digital camera 10 in which a stereo adapter 9 is mounted on the front surface thereof as stereo image photographing means for photographing images of the photographing object 4 from a plurality of viewpoints. And a computing device 6 such as an external personal computer connected to the digital camera 10.

  Here, a digital camera 10 having a stereo adapter 9 attached to the front surface thereof as a stereo image capturing means has a rear surface portion as a display means for displaying an output image captured by the digital camera 10 on a display screen. An electronic viewfinder 5 is provided.

  In other words, the first embodiment is an example in which a stereo matching processing area frame 8 as described later is displayed on the electronic viewfinder 5 of the digital camera 10 to which the stereo adapter 9 is attached.

  2 (a) and 2 (b) show a digital camera as a stereo image capturing means used in the stereo photographing apparatus according to the first embodiment of the present invention configured as shown in FIG. 1 (a). 10 and a plan view and a front view showing the configuration of the stereo adapter 9 attached to the front surface of the digital camera 10.

  That is, the stereo adapter 9 mounted on the front surface of the digital camera 10 has a pair of first reflecting mirrors 18a and 18b and a second reflecting mirror that are arranged to face each other with a predetermined inclination angle. It is possible for the digital camera 10 to photograph the photographing object 4 as images from two viewpoints via the mirrors 18c and 18d.

  FIG. 1 (b) shows a stereo adapter 9 mounted on the front surface of the stereo image pickup means as shown in FIG. 1 (a) constituting the stereo photographing apparatus according to the first embodiment of the present invention. When the photographing object 4 is photographed using the digital camera 10 and the computing device 6 as the calculation means, a closed region (hereinafter referred to as a stereo matching processing region) 7 on which stereo matching processing is performed is presented to the photographer. It is a figure which illustrates the stereo matching process area | region frame 8 which shows the closed area | region which exists inside the outer edge of the image displayed on the display screen of the electronic finder 5 as a display means as a mark for it.

  That is, in the stereo photographing apparatus according to the first embodiment of the present invention, the digital camera 10 having the stereo adapter 9 attached to the front surface thereof as the stereo image capturing means and the calculation device 6 as the calculation means are used. When shooting the shooting target 4 as shown in FIG. 1A, an output image captured by the digital camera 10 is displayed on the display screen of the electronic viewfinder 5 as display means in the digital camera 10. In order to present to the photographer a stereo matching processing region 7 as shown in FIG. 1B determined by lens distortion, rectification, the ability of the calculation device 6 itself as a calculation means, etc. A state in which the stereo matching processing region frame 8 is electrically superimposed on the output image inside the portion where the image is displayed In is to be displayed.

  As a result, the user looks at the stereo matching processing area frame (frame surrounding the second closed area) 8 displayed on the display screen of the electronic viewfinder 5 as display means in the digital camera 10, thereby capturing the object 4. It becomes easy to put the image in the stereo matching processing area (first closed area) 7. Here, in the stereo matching processing area frame (frame surrounding the second closed area) 8, it is preferable that the distortion of the output image captured by the digital camera 10 as the stereo image capturing unit is not more than a predetermined level. .

  The first closed region is a region where the stereo matching process is actually performed, and the second closed region is set with a certain margin inside the first closed region. It is a region inside the first closed region. However, in the first embodiment, the outer edge of the first closed region and the frame surrounding the second closed region are assumed to coincide.

  Then, the calculation device 6 as the calculation means is stereo for the stereo matching processing area 7 as the first closed area in the output image displayed on the display screen of the electronic viewfinder 5 as the display means. Execute the matching process.

  It should be noted that the configuration of the first embodiment of the present invention can naturally be variously modified and changed.

  For example, the stereo matching processing area frame 8 does not have to be a frame constituted by a straight line as shown in FIG. 1B, but as shown in FIGS. 3A to 3C, The stereo matching processing area frame 8 may be presented by a broken line or the like.

  That is, as shown in FIG. 3A, the stereo matching processing area frame 8 may be presented with a dotted line without completely surrounding the stereo matching processing area frame 8.

  Further, as shown in FIG. 3B, as the stereo matching processing region frame 8, a mark indicating the corner of the region by two intersecting line segments may be provided. As a modified example, as shown in FIGS. 4A and 4B, a closed curve such as a point or a small circle may be arranged at the intersection position of the line segments. Of course, as shown in (c) to (e) of FIG. 4, the marks may be arranged at appropriate intervals in portions other than the corners of the stereo matching processing region frame 8. That is, any mark may be used as long as it is a clue at the time of photographing, such as a line segment, a crossed line segment, a curve, a crossed curve, a point, or a closed curve. In short, any mark may be used as long as the user can recognize the stereo matching processing area frame 8.

  Further, as shown in FIG. 3C, the stereo matching processing area frame 8 may be presented as a stereo matching processing area frame 8 with a black frame (however, shown in the figure as a hatched frame). good.

  Alternatively, instead of displaying the stereo matching processing area frame 8, the area other than the stereo matching processing area 7 may not be displayed, or the area may be indicated by changing the color.

  Further, the stereo matching processing area 7 may be determined by the photographer from the various parameters of the stereo photographing apparatus, even if the calculation apparatus 6 as the calculation means determines at the time of photographing.

  Further, the stereo matching processing area frame 8 does not have to be a quadrangle as shown. For example, when the stereo matching processing region 7 is determined using lens distortion or the like as a threshold, it may be a semicircular frame (shown by a hatched frame in the drawing) as shown in FIG.

  Further, the stereo matching processing area 7 may be determined at the time of photographing or before.

  Furthermore, in this embodiment, there are two viewpoints, but three or more viewpoints and output images may be used.

  In addition, a stereo field of view may be presented to the user simultaneously with the stereo matching processing area frame 8.

  Furthermore, the display means is not necessarily limited to the case where the electronic viewfinder 5 built in the digital camera 10 is used, and may be a system using an external display.

  Further, the display of the stereo matching processing area frame 8 may be blinked. If such a blinking display is performed, the stereo matching processing area frame 8 is conspicuous, and the alignment is further facilitated.

  In this case, the presence / absence of the blinking display operation is used by using, for example, an input device (not shown) of the digital camera 10 such as a menu button or an input device (not shown) of the computing device 6 such as a keyboard. The user may be able to select.

  Furthermore, the presence / absence of display of the stereo matching processing area frame 8 itself may be selected using this input device. Of course, a configuration in which the mounting state of the stereo adapter 9 to the digital camera 10 is detected and the display / non-display of the stereo matching processing area frame 8 is automatically switched according to the detection result is also conceivable.

[Second Embodiment]
FIG. 6 is a perspective view showing a configuration of a stereo photographing apparatus according to the second embodiment of the present invention.

  That is, as shown in FIG. 6, the stereo imaging apparatus according to the second embodiment of the present invention has a stereo imaging system 1 including two imaging cameras instead of using the above-described stereo adapter as a stereo image imaging unit. An external calculation device 6 as calculation means connected to the stereo imaging system 1 and an external display 5A as display means connected to the calculation device 6 as calculation means.

  And when the stereo imaging device by the 2nd Embodiment of this invention image | photographs the imaging | photography object 4 with the stereo imaging system 1, the output image imaged with the stereo imaging system 1 is displayed on the display 5A as a display means. Is displayed. In this case, in the stereo imaging device according to the second embodiment of the present invention, the lens distortion and rectification of the stereo imaging system 1 and the capability of the calculation device 6 itself as the calculation unit are displayed on the display 5A as the display unit. A state in which the stereo matching processing region frame 8 is electrically superimposed on the output image in order to present the photographer with a closed region (hereinafter referred to as a stereo matching processing region) 7 to be subjected to stereo matching processing determined by Is displayed.

  As a result, the user looks at the stereo matching processing area frame (frame surrounding the second closed area) 8 displayed on the display 5A as the display means, thereby making the photographing object 4 the stereo matching processing area (first area). It is easy to put it in the closed region 7).

  Here, in the stereo matching processing area frame (frame surrounding the second closed area) 8, it is desirable that the distortion of the output image captured by the stereo imaging system 1 is not more than a predetermined level.

  Then, the calculation device 6 as the calculation means performs stereo matching with respect to the stereo matching processing area 7 as the first closed area in the output image displayed on the display screen of the display 5A as the display means. Execute the process.

  It should be noted that the configuration of the second embodiment of the present invention can naturally be variously modified and changed.

  For example, the stereo matching processing area frame 8 does not need to be a frame constituted by a straight line. As shown in FIGS. 3A to 3C, the stereo matching processing area frame 8 is described with a dotted line, a broken line, or the like. You may make it present similarly.

  Further, instead of displaying the stereo matching processing area frame 8, the area other than the stereo matching processing area 7 may not be displayed, or the area may be indicated by changing the color.

  Further, the stereo matching processing area 7 may be determined from various parameters by the calculation device 6 as the calculation means at the time of shooting, or may be determined by the photographer.

  Furthermore, the stereo matching processing area frame 8 does not have to be a quadrangle as shown in the figure. For example, when the stereo matching processing region 7 is determined using the lens distortion or the like of the stereo imaging system 1 as a threshold, it is a semicircular frame with a black frame as shown in FIG. 5 (however, it is indicated by a hatched frame in the drawing). There may be.

  Further, the stereo matching processing area 7 may be determined at the time of photographing or before that.

  Further, the stereo imaging system 1 may be composed of three or more cameras.

  Furthermore, a form in which a plurality of displays 5A as display devices are prepared and an image from each viewpoint is displayed on each display may be employed.

  In addition, a stereo field of view may be presented to the user simultaneously with the stereo matching processing area frame 8.

  Further, the display of the stereo matching processing area frame 8 may be blinked. Further, in this case, the presence or absence of the blinking display operation may be selected by the user using an input device of the calculation device 6.

[Third Embodiment]
The configuration of the stereo photographing apparatus according to the third embodiment is basically as shown in FIG. 1A of the first embodiment or FIG. 6 of the second embodiment. It is the same as that of the stereo imaging device comprised.

  Then, the stereo photographing apparatus according to the third embodiment of the present invention has a lens of the stereo photographing apparatus on the electronic finder 5 of the first embodiment or the display 5A of the second embodiment described above as the display means. In order to present to the photographer a closed region (hereinafter referred to as a stereo matching processing region) 7 subjected to stereo matching processing determined in advance by distortion, rectification, the ability of the calculation device 6 itself as a calculation means, and the like. Instead of displaying the matching processing area frame 8 in a state where it is electrically superimposed on the output image as in the first or second embodiment described above, it is previously displayed on the electronic viewfinder 5 or display 5A as a display means. This is an example of printing directly on the inside of the image displayed on the screen.

  That is, in the third embodiment, when the photographing object 4 is photographed using the stereo photographing apparatus configured as shown in FIG. 1A or FIG. Alternatively, on the display 5A, the stereo as shown in FIG. 1B or FIG. 6, which is determined in advance by the lens distortion and rectification of the stereo photographing device, the ability of the calculation device 6 itself as the calculation means, and the like. In order to present the matching processing area 7 to the photographer, a stereo matching processing area frame 8 is fixed in advance by direct printing.

  Thereby, the user sees the stereo matching processing area frame (frame surrounding the second closed area) 8 fixed in advance by direct printing on the display screen of the electronic finder 5 or display 5A as the display means. It becomes easy to put the image of the subject 4 into the stereo matching processing area (first closed area) 7.

  Here, in the stereo matching processing area frame (frame surrounding the second closed area) 8, it is desirable that the distortion of the output image captured by the stereo imaging device is not more than a predetermined level.

  Then, the calculation device 6 as the calculation means performs the stereo matching processing area 7 as the first closed area in the output image displayed on the display screen of the electronic finder 5 or the display 5A as the display means. The stereo matching process is executed.

  As described above, the stereo photographing apparatus according to the third embodiment of the present invention includes the second closed region included in the first closed region on the display screen of the electronic finder 5 or the display 5A as the display unit. By displaying a frame (stereo matching process area frame 8) fixed by direct printing on the display screen as a mark to be shown, the photographing object 4 is captured in the frame 8 and the stereo matching process is facilitated. .

  It should be noted that each configuration of the third embodiment of the present invention can be variously modified and changed.

  For example, the stereo matching processing area frame 8 does not have to be a straight line frame, as shown in FIGS. 3A, 3B, and 3C described above. Alternatively, the stereo matching processing region frame 8 may be directly fixed and presented by a dotted line, a broken line, a black frame (however, indicated by a hatched frame in the drawing), or the like.

  In addition, instead of printing the stereo matching processing area frame 8, the area other than the stereo matching processing area 7 may not be displayed, or the area may be indicated by changing the color.

  Other means for presenting the above-mentioned mark so as to be visible are as follows.

  For example, instead of fixing the stereo matching processing area frame 8 on the electronic viewfinder 5 or the display 5A as a display means by direct printing, a seal as a stereo matching processing area frame display body on which the stereo matching processing area frame 8 is printed is provided. You may make it adhere by sticking on the electronic finder 5 or display 5A as a display means.

  Further, the plate as the stereo matching processing region frame display body in which the portion of the stereo matching processing region 7 is visible and the other portions are invisible is fixed on the electronic finder 5 or the display 5A as the display means. Also good.

  Further, there is a method of presenting a mark by optical image superposition in an imaging device having an optical viewfinder other than or in place of the display means such as the electronic viewfinder 5 or the display 5A.

  An example of this optical image superimposition will be described with reference to FIGS.

  FIG. 7A is a diagram illustrating a configuration in the case of a twin-lens imaging device. In this case, the external light taken in from the diffusion plate 20 passes through the glass plate 27 which is a member similar to the stereo matching processing region frame 8 to generate a stereo matching processing region frame image. The image from the objective lens 21 is superposed by the half mirror 17 via the mirror 18 and the optical path adjustment lens 37. Then, the image enters the user's eye 19 through the eyepiece 22.

  FIG. 7B is a diagram illustrating a configuration in the case of a single-lens reflex photographing apparatus. In this case, the image from the objective lens 21 is reflected by the movable mirror 24 disposed on the front surface of the image sensor 25, passes through the glass plate 27, and the stereo matching processing region frame image is superimposed. Then, the image passes through the prism 23 and the eyepiece 22 and enters the user's eye 19.

  In the case of such optical image superimposition, for example, instead of taking in external light from the diffuser plate 20, the stereo matching processing region frame 8 is presented by a self-luminous method using LEDs or the like as a light source, or transmitted instead of the glass plate 27. A type LCD or the like may be used. The stereo matching processing area frame image may be an image of a plurality of LEDs themselves.

  Also, the stereo matching processing area frame image is displayed / hidden by turning on / off the LED or the transmissive LCD or by blocking the optical path that guides the stereo matching processing area frame image to the photographer's eyes. Or flash.

  Alternatively, the stereo matching processing area frame 8 may be drawn or printed on the front or back surface of the finder so that the stereo matching processing area frame 8 is visible.

  Further, as shown in FIG. 8A, the stereo matching processing region frame 8 presented as a mark presenting member 30, such as a film or panel that transmits visible light, and the stereo matching processing region frame 8 are visible. It may be attached to the viewfinder. Alternatively, it may be provided in the form of a thin seal that transmits visible light and attached to the viewfinder.

  Alternatively, as shown in FIG. 8 (b), a perforated member 36 having a hole that transmits visible light may be attached to the viewfinder in accordance with the shape of the stereo matching processing region frame 8. In this case, it suffices to shoot so that the photographic subject 4 fits in the hole corresponding to the stereo matching processing region frame 8, and at least if the outer frame of the field of view is recognizable, the focus of the eye is completely on the photographic subject 4. Position alignment is possible even in a state that does not fit. Therefore, there is an advantage that the restriction of the attachment position is loose compared with the case where a member with a linear mark as shown in FIG. 4 is attached.

  A mark indicating the closed region may be presented on the optical viewfinder using the above-described means.

  In addition, when a transmissive LCD is used instead of a glass plate in the optical image superposition, or when the electronic finder 5 or the display 5A is used as a display unit as in the above embodiments, the user's If desired, the display or non-display of the mark may be selected using an input device of the photographing apparatus.

  That is, as described above, the stereo matching processing region frame 8 is not always displayed by always superimposing it, but at the user's request, that is, at the time of shooting, the user uses the operation input device, The display / non-display of the stereo matching processing area frame 8 as the mark may be selected. In this manner, by enabling the display / non-display of the stereo matching processing area frame 8, for example, by selecting not to display the stereo matching processing area frame 8 at the time of shooting not intended for calibration, the stereo matching processing area frame 8 can be selected. It is possible to prevent the matching processing area frame 8 from becoming an obstacle.

  Furthermore, the display of the stereo matching processing area frame 8 may be blinked. In this case, the presence or absence of the blinking display operation may be selected by the user using the input device, similar to the presence or absence of the display of the stereo matching processing area frame 8.

  In the present embodiment, there are two viewpoints, but three or more viewpoints and output images may be used.

  Further, a stereo field of view may be presented to the user simultaneously with the stereo matching processing area frame 8.

  Further, the stereo matching processing area frame 8 does not have to be a quadrangle as shown in the figure, and is, for example, a semicircular frame with a black frame as shown in FIG. 5 (however, shown in the figure as a hatched frame). There may be.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described.

  In the present embodiment, a predetermined calibration pattern is photographed to calculate image distortion, a stereo matching processing region (first closed region) 7 is determined based on the image distortion, and a stereo matching processing region frame (second closed region) is determined. (Frame surrounding the area) 8 is presented.

  Here, the calibration pattern 11 may be a three-dimensional pattern as shown in FIG. 9A or a two-dimensional pattern as shown in FIG. 9B. Alternatively, one is known as long as it is known and information for calculating image distortion can be obtained.

  FIG. 10 is a diagram showing a configuration of a stereo photographing apparatus according to the fourth embodiment of the present invention and a state where the photographing object 4 is photographed.

  The configuration of the stereo photographing apparatus according to the fourth embodiment is basically the same as that of the stereo photographing apparatus configured as shown in FIG. 1A of the first embodiment described above. is there.

  Then, in the stereo photographing apparatus according to the fourth embodiment of the present invention, using a digital camera 10 having an electronic viewfinder 5 and a stereo adapter 9, a corner cube type calibration whose shape and surface attributes are known is used. A calibration pattern image 13 is acquired. The calculation device 6 recognizes the pattern (surface attribute) on the calibration pattern 11 shown on the calibration pattern image 13. Then, how the calibration pattern image 13 is distorted is calculated based on the recognized pattern, the shape, and the degree of deviation of the pattern of the calibration pattern 11 whose surface attribute is known. Here, the closed region, that is, the stereo matching processing region (first closed region) 7 in which the shift is less than one pixel with respect to each pixel of the calibration pattern image 13 is stored in the storage device 26 in the calculation device 6.

  Thus, when the photographing object 4 is photographed using the digital camera 10, the calculation device 6 is an electronic display device for presenting the stereo matching processing area 7 stored in the storage device 26 to the photographer. A stereo matching processing area frame (frame surrounding the second closed area) 8 is displayed on the finder 5.

  The user can easily put the photographing object 4 in the stereo matching processing area 7 by looking at the stereo matching processing area frame 8 displayed on the electronic viewfinder 5.

  It should be noted that various changes and modifications can naturally be made to each configuration of the embodiment of the present invention.

  For example, a frame indicating a region where the normal distortion is equal to or lower than a predetermined level is as shown in FIG. Further, the stereo matching processing area frame 8 does not have to completely coincide with the boundary line of the stereo matching processing area 7, and may be displayed by being approximated by a straight line or a curve that is easy for the user to recognize. Furthermore, the stereo matching processing area frame 8 may indicate an area with a dotted line, a broken line, or the like.

  Further, instead of displaying the stereo matching processing area frame 8, the area other than the stereo matching processing area 7 may not be displayed, or the area may be indicated by changing the color.

  In addition, the method for calculating the image distortion does not have to be the photographing of the corner cube type calibration pattern 11. For example, a plurality of plate-like calibration patterns 11 with known surface attributes as shown in FIG. 9B may be taken at different distances, and image distortion may be calculated from the image information. Further, even when an object with an unknown shape and surface attribute is photographed without photographing the calibration pattern 11, a search is made as to which location on the image 2 each pixel on the reference image 3 corresponds to. Image distortion can also be calculated (self-calibration).

  Further, the distortion level boundary that determines the boundary of the stereo matching processing region 7 does not need to be shifted by one pixel, and can be arbitrarily set (the calculated shift pixel number does not have to be an integer). Furthermore, it may be changed in conjunction with the size of the search window for performing the corresponding point search.

  Further, the place where the computing device 6 and the storage device 26 exist may be inside the digital camera 10 instead of outside the digital camera 10. Further, the storage device 26 may exist outside the computing device 6 instead of inside.

  Further, the stereo matching processing area 7 may be stored in the storage device 26 in advance. In this case, the stereo matching processing area 7 calculated from the calibration pattern image 13 photographed by another digital camera 10 having substantially the same image distortion may be stored.

  In the present embodiment, there are two viewpoints, but three or more viewpoints and output images may be used.

  In addition, a stereo field of view may be presented to the user simultaneously with the stereo matching processing area frame 8.

  Further, as a display means, a system using an external display instead of the electronic viewfinder 5 provided in the digital camera 10 may be used.

  Furthermore, the system need not be a system using the stereo adapter 9 and may be used in a stereo image measurement system using a plurality of imaging systems as shown in FIG. 6 of the second embodiment.

  In addition, after determining the stereo matching processing area frame 8 as described above, the light projecting range of the light projecting apparatus such as the strobe device or the light pattern projection apparatus is matched with the determined stereo matching processing area frame 8 of the light projecting apparatus. Adjustment may be performed by performing zooming or aperture. In this case, since no light is projected outside the stereo matching processing area frame 8, it is possible to expect downsizing and power saving of the light projecting device.

  If the digital camera 10 is equipped with functions such as AF and AE, the AF and AE may be operated based on the image in the determined stereo matching processing area frame 8.

  Further, the measurement range information defined by the stereo matching processing area frame 8 may be used for AF by sending it to the digital camera 10 as the imaging means.

[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described.

  The configuration of the stereo imaging device according to the fifth embodiment is basically the same as that of the stereo imaging device configured as shown in FIG. 1A of the first embodiment described above. is there.

  In the fifth embodiment, when photographing the photographing object 4 using the digital camera 10 having the electronic viewfinder 5 and the stereo adapter 9 attached, the computing device 6 stereos on the electronic viewfinder 5. An image in which a portion other than the matching processing area 7 is not displayed is displayed.

  The user can easily put the photographing object 4 in the stereo matching processing area 7 by looking at the displayed image.

  It should be noted that various changes and modifications can naturally be made to each configuration of the embodiment of the present invention.

  For example, not only the portion other than the stereo matching processing area 7 is displayed, but a stereo matching processing area frame 8 indicating the boundary line may be displayed. Alternatively, the area may be indicated by changing the color of the part other than the stereo matching processing area 7.

  Further, the stereo matching processing area frame 8 does not have to be a rectangle. For example, the shape shown in FIG. 5 may be used.

  In the present embodiment, there are two viewpoints, but three or more viewpoints and output images may be used.

  Further, as a display means, a system using an external display instead of the electronic viewfinder 5 provided in the digital camera 10 may be used.

  Further, the system need not be a system using the stereo adapter 9, and may be used in a stereo image measurement system using a plurality of imaging systems as shown in FIG. 6 of the second embodiment.

[Sixth Embodiment]
Next, a sixth embodiment of the present invention will be described.

  The configuration of the stereo imaging apparatus according to the sixth embodiment is basically the same as that of the stereo imaging apparatus configured as shown in FIG. 1A of the first embodiment described above. is there.

  In the present embodiment, when the photographing object 4 is photographed using the digital camera 10 equipped with the electronic viewfinder 5 and the stereo adapter 9 attached, the photographer uses an input device provided in the digital camera 10 or the computing device 6. Use to enter the measurement distance range. The input measurement distance range value is stored in the storage device 26 included in the calculation device 6. The calculation device 6 determines the stereo matching processing region 7 based on the measurement distance range that is input and stored.

  Here, in the case of the present embodiment, as shown in FIG. 11, among the left and right images divided by the stereo adapter 9, the left image is the reference image 3 and the right image is the corresponding image 12.

  In order to calculate the distance to the search point M on the reference image 3, the calculation device 6 searches for the corresponding point N in the corresponding image 12 corresponding to the search point M.

  At this time, as shown in FIG. 11, the shooting target 4 in the reference image 3 viewed from the left viewpoint 51 is shown from the right, and the shooting target 4 in the corresponding image 12 viewed from the right viewpoint 52 is shown from the left. To do.

  In this case, the corresponding point search is performed on the coordinates on the left side of the corresponding image 12 with respect to the abscissa of the search point M in the reference image 3.

  At this time, if the measurement distance range is not set, search all the left sides of the abscissa of the search point M in the corresponding image 12, that is, the search range 40 when the measurement distance range is not set. become.

  Here, the principle of triangulation is based on the difference between the abscissas of the search point M and the corresponding point N in the captured stereo image (hereinafter referred to as parallax) and various parameters (baseline length, convergence angle, etc.). The principle is that the distance between the photographing object 4 (a point corresponding to the search point M) and the photographing apparatus can be calculated, and the distance from the photographing object is far when the parallax is small, and the distance is short when the parallax is large. There is a correspondence between (photographing) distance and parallax. That is, when the measurement distance range is set, the value that the parallax can take is also limited, and as a result, the search range of the corresponding point N with respect to the search point M can be narrowed.

  In the case of the present embodiment, when the measurement distance range is set (in this case, the farthest distance value), the minimum possible parallax 42, that is, the farthest distance value with a limited distance to the imaging target 4 The parallax is determined.

  Thereby, when the measurement distance range is set, the search may be performed from the point shifted to the left by the minimum parallax 42 that can be taken from the abscissa of the search point M in the corresponding image 12. It will be. That is, in the corresponding image 12, all the points to the left of the point shifted to the left by the minimum parallax 42 that can be taken from the abscissa of the search point M, that is, the range in which the corresponding point N of the search point M is searched based on the measurement distance range. 41 will be searched.

  Therefore, when measuring the upper right point of the reference image 3, the region to be searched may be searched from the point shifted to the left by the minimum parallax 42 that can be taken from the right most on the corresponding image 12. Become. Accordingly, as shown in FIG. 12A, the range of the minimum parallax 42 from the rightmost on the corresponding image 12 is not used for stereo matching. In addition, regarding the range shifted to the right by the minimum parallax 42 from the leftmost on the reference image 3, the search on the corresponding image 12 cannot be performed (the search range goes out of the left of the corresponding image 12). Not used for stereo matching.

  By setting the measurement distance in this manner, an area that is not used for stereo matching, that is, a search unnecessary area is calculated.

  The calculation device 6 sets the stereo matching processing area 7 in a form excluding the search unnecessary area from the reference image 3 or the corresponding image 12, and presents the stereo matching processing area frame 8 to the photographer through the electronic viewfinder 5.

  It should be noted that various changes and modifications can naturally be made to each configuration of the embodiment of the present invention.

  For example, the measurement distance range is not input by the input device, but may be determined from the approximate distance obtained using another sensor.

  Also, the reference image need not be left.

  In the present embodiment, the left and right stereo systems are used. However, the stereo matching process area 7 and the stereo matching process area frame 8 can be set corresponding to each of three or more or top and bottom stereo systems. .

  Further, the system need not be a system using the stereo adapter 9, and may be used in a stereo image measurement system using a plurality of imaging systems as shown in FIG. 6 of the second embodiment.

  Further, the stereo matching processing area 7 and the search unnecessary area are excluded from a desired area determined by other means such as the maximum stereo matching processing area assumed in the reference image 3 and the corresponding image 12, for example. A stereo matching processing area frame 8 may be set.

  Further, not only the farthest distance value but also the nearest distance value or both may be set as the measurement distance range. When both the farthest distance value and the nearest distance value are set, the stereo matching processing area 7 is set as shown in FIG. 12B using the maximum parallax 43 that can be determined from the nearest distance. The stereo matching processing area frame 8 may be presented. In this case, the photographer can take an image suitable for performing the stereo matching process and performing the three-dimensional reconstruction by using the stereo matching process area frame 8 as a mark.

  The determination of the stereo matching area based on the measurement distance range does not have to be performed when the measurement distance range is input, and a correspondence table between the measurement distance range and the stereo matching area may be stored in the storage device 26 in advance.

[Seventh Embodiment]
Next, a seventh embodiment of the present invention will be described.

  The configuration of the stereo imaging apparatus according to the seventh embodiment is basically the same as that of the stereo imaging apparatus configured as shown in FIG. 1A of the first embodiment described above. However, in the present embodiment, the first reflection mirrors 18a and 18b in the stereo adapter 9 are movable mirrors, and the convergence angle θ is changed according to the input measurement distance range.

  That is, in the present embodiment, as in the sixth embodiment, when the photographing object 4 is photographed using the digital camera 10 that includes the electronic viewfinder 5 and the stereo adapter 9, the photographer inputs The measurement distance range is input using the device, and the value is stored in the storage device 26. Then, the calculation device 6 changes the convergence angle θ of the first reflecting mirrors 18a and 18b in the stereo adapter 9 according to the measurement distance range that is input and stored. Further, the stereo matching processing area frame 8 is displayed on the electronic finder 5 in order to present the range of the overlapping area 33 determined by the changed convergence angle θ and the zoom status of the digital camera 10 to the user. Thereby, the user can easily recognize the stereo matching processing region 7 that changes in accordance with the change in the convergence angle θ.

  It should be noted that various changes and modifications can naturally be made to each configuration of the embodiment of the present invention.

  For example, the measurement distance range is not input using an input device, but may be determined from the approximate distance obtained using another sensor.

  As described above, the stereo matching processing area 7 is set using one or more of the information regarding the shooting angle of view and the convergence angle of multiple viewpoints and the measurement distance range, and the stereo matching processing area frame 8 is presented. good.

  Further, instead of using the stereo adapter 9, a plurality of cameras with different viewpoints as shown in FIG. 6 of the second embodiment may be used. In this case, however, a mechanism for automatically changing the orientations of the two cameras according to the input measurement distance range, or means for visually or audibly instructing the orientation of the cameras to be changed manually. It is necessary to provide.

  By the way, in each of the first to seventh embodiments, the outer edge of the stereo matching process area 7 where the stereo matching process is performed and the stereo matching process area frame 8 displayed by the user are matched.

  However, in actual use, an allowance is provided between the stereo matching processing region 7 and the stereo matching processing region frame 8 for displaying it in consideration of camera shake, user positioning error, and the like. It is desirable.

  That is, a certain amount of margin is provided inside the stereo matching processing area 7 (first closed area) where the stereo matching process is actually performed, and further, an area (second closed area) is set inside thereof, and this inner area is set. It is preferable to arrange a stereo matching processing region frame 8 to be displayed along the outer edge of the (second closed region).

  As a result, the user intends to shoot an image in which the shooting target is put in the displayed stereo matching processing area frame 8, so that the entire shooting target is in the stereo matching processing area 7 even if there is a slight camera shake. The possibility of protruding outside is reduced, and stereo shooting of a desired object can be reliably performed.

  In stereo shooting, a region where three-dimensional information can be obtained is included in a superimposed region where imaging spaces that can be captured from each of a plurality of viewpoints overlap. Accordingly, through the first to seventh embodiments described above, a region corresponding to the superimposed region in the output image may be presented to the photographer instead of the region where the stereo matching process is performed.

[Appendix]
The invention having the following configuration can be extracted from the specific embodiment described above.

(1) Stereo image capturing means for capturing images to be photographed from a plurality of viewpoints;
Display means for displaying an output image captured by the stereo image capturing means on a display screen;
A mark displayed on the display screen in order to indicate a closed region existing inside the outer edge of the image displayed on the display screen by the display means;
A stereo photographing apparatus comprising:

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments relating to the stereo imaging device described in (1).
(Function and effect)
According to the stereo photographing apparatus described in (1), an image obtained by photographing the photographing object 4 by the stereo image photographing unit is displayed on the display screen of the electronic finder 5 or the display 5A as the display unit. At this time, in order to show the closed region existing inside the outer edge of the image displayed on the display screen of the electronic finder 5 or display 5A as the display means, a frame (stereo) as a mark displayed on the display screen is displayed. By displaying the matching processing area frame 8), it is easy to capture the photographing object 4 in the frame.

  (2) The mark is characterized in that it indicates at least the position of the corner of the closed region by arranging a line segment, a crossed line segment or curve, a crossed curve, a point or a closed curve (1) Stereo imaging device described in).

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments relating to the stereo imaging device described in (2).
(Function and effect)
According to the stereo photographing apparatus described in (2), in order to show the closed region existing inside the outer edge of the image displayed on the display screen of the electronic finder 5 or the display 5A as the display means, the display screen By arranging a line segment, a crossed line segment, a curve, a crossed curve, a point or a closed curve as a mark displayed above, a frame (stereo matching processing region frame 8) indicating at least the position of the corner of the closed region is provided. By displaying, it is easy to capture the photographing object 4 in the frame.

  (3) The stereo photographing apparatus according to (1), wherein the mark surrounds an outer edge of the closed region.

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments relating to the stereo imaging device described in (3).
(Function and effect)
According to the stereo photographing apparatus described in (3), in order to show the closed region existing inside the outer edge of the image displayed on the display screen of the electronic viewfinder 5 or the display 5A as the display means, the display screen By displaying a frame (stereo matching processing region frame 8) surrounding the outside green of the closed region as a mark displayed above, it is easy to capture the photographing object 4 in the frame.

  (4) The stereo photographing apparatus according to (1), wherein the mark is electrically superimposed on the output image captured by the stereo image capturing unit.

(Corresponding embodiment)
The first, second, fourth, sixth, and seventh embodiments correspond to the embodiments relating to the stereo imaging device described in (4).
(Function and effect)
According to the stereo photographing apparatus described in (4), in order to show the closed region existing inside the outer edge of the image displayed on the display screen of the electronic viewfinder 5 or the display 5A as the display means, the display screen By capturing a frame (stereo matching processing region frame 8) that is electrically superimposed on the output image captured by the stereo image capturing means as a mark displayed above, the imaging object 4 is captured in the frame. To make it easier.

  (5) The stereo photographing apparatus according to (1), wherein the mark is fixed on the display means.

(Corresponding embodiment)
The embodiment relating to the stereo photographing apparatus described in (5) corresponds to the third embodiment.
(Function and effect)
According to the stereo photographing apparatus described in (5), in order to show the closed region existing inside the outer edge of the image displayed on the display screen of the electronic finder 5 or the display 5A as the display means, the display screen By displaying a frame (stereo matching processing region frame 8) fixed on the display means as a mark displayed above, it is easy to capture the photographing object 4 in the frame.

  (6) The stereo photographing apparatus according to (1), wherein the closed region has a distortion of the output image captured by the stereo image capturing unit equal to or lower than a predetermined level.

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments related to the stereo photographing device described in (6).
(Function and effect)
According to the stereo photographing apparatus described in (6), the stereo image capturing unit captures a closed area existing inside the outer edge of the image displayed on the display screen of the electronic finder 5 or the display 5A as the display unit. By displaying a frame (stereo matching processing region frame 8) as a mark displayed on the display screen in order to indicate a closed region where the distortion of the output image is less than or equal to a predetermined level, shooting is performed within the frame. It makes it easy to capture the object 4.

(7) stereo image capturing means for capturing images to be photographed from a plurality of viewpoints;
Display means for displaying an output image captured by the stereo image capturing means on a display screen;
Calculation means for performing a stereo matching process on the first closed region in the output image displayed on the display screen by the display means;
A second closed region included in the first closed region, the mark displayed on the display screen;
A stereo photographing apparatus comprising:

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments relating to the stereo photographing device described in (7).
(Function and effect)
The stereo photographing apparatus described in (7) displays an image obtained by photographing the photographing object 4 by the stereo image photographing unit on the display screen of the electronic finder 5 or the display 5A as the display unit. The calculation device 6 as the calculation means is for the first closed region (stereo matching processing region 7) in the output image displayed on the display screen of the electronic finder 5 or the display 5A as the display means. Perform stereo matching processing. According to the stereo photographing apparatus described in (7), the frame as a mark indicating the second area included in the first closed area on the display screen of the electronic finder 5 or the display 5A as the display means. By displaying the (stereo matching process area frame 8), it is possible to capture the photographing object 4 in the frame and to easily perform the stereo matching process.

  (8) The mark is characterized in that it indicates at least the position of the corner of the second closed region by arranging a line segment, a crossed line segment, a curve, a crossed curve, a point, or a closed curve. The stereo photographing apparatus according to (7).

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments related to the stereo photographing apparatus described in (8).
(Function and effect)
According to the stereo photographing apparatus described in (8), on the display screen of the electronic finder 5 or the display 5A as the display means, a line segment or mark as a mark indicating the second area included in the first closed area is used. By arranging a crossed line segment or curve, a crossed curve, a point or a closed curve, and displaying a frame (stereo matching processing region frame 8) indicating at least the corner position of the second closed region, In addition to capturing the object 4 to be photographed, it is easy to perform stereo matching processing.

  (9) The stereo photographing apparatus according to (7), wherein the mark surrounds an outer edge of the second closed region.

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments related to the stereo photographing device described in (9).
(Function and effect)
According to the stereo photographing apparatus described in (9), the second view as a mark indicating the second area included in the first closed area on the display screen of the electronic viewfinder 5 or the display 5A as the display means. By displaying a frame surrounding the outer edge of the closed region (stereo matching processing region frame 8), it is possible to capture the photographing object 4 in the frame and to easily perform the stereo matching processing.

  (10) The stereo photographing apparatus according to (7), wherein the mark is electrically superimposed on the output image captured by the stereo image capturing unit.

(Corresponding embodiment)
The first, second, fourth, sixth, and seventh embodiments correspond to the stereo imaging device described in (10).
(Function and effect)
According to the stereo photographing apparatus described in (10), the stereo image is used as a mark indicating the second area included in the first closed area on the display screen of the electronic finder 5 or the display 5A as the display means. By displaying a frame (stereo matching processing region frame 8) that is electrically superimposed on the output image captured by the imaging means, it is possible to easily capture the photographing object 4 in the frame and perform stereo matching processing. To do.

  (11) The stereo photographing apparatus according to (7), wherein the mark is fixed on the display screen.

(Corresponding embodiment)
The third embodiment corresponds to the embodiment relating to the stereo photographing apparatus described in (11).
(Function and effect)
According to the stereo photographing apparatus described in (11), the display screen is used as a mark indicating the second area included in the first closed area on the display screen of the electronic finder 5 or the display 5A as display means. By displaying the frame (stereo matching processing area frame 8) fixed on the top, it is possible to capture the photographing object 4 in the frame and to easily perform the stereo matching processing.

  (12) The stereo photographing apparatus according to (7), wherein the second closed region has a distortion of the output image captured by the stereo image capturing unit equal to or lower than a predetermined level.

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments related to the stereo photographing device described in (12).
(Function and effect)
According to the stereo photographing apparatus described in (12), the stereo image capturing means as the second area included in the first closed area on the display screen of the electronic viewfinder 5 or the display 5A as the display means. By displaying a frame (stereo matching processing region frame 8) as a mark indicating a region where the distortion of the captured output image is equal to or lower than a predetermined level, the photographing target 4 is captured in the frame and stereo matching processing is performed. To make it easier.

  (13) The stereo photographing apparatus according to (7), wherein the second closed region is specified by the calculation unit based on characteristics of the stereo image capturing unit.

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments relating to the stereo imaging device described in (13).
(Function and effect)
According to the stereo photographing apparatus described in (13), the stereo image capturing means is used as the second area included in the first closed area on the display screen of the electronic viewfinder 5 or the display 5A as the display means. By displaying the area specified by the calculation means based on the characteristics, the photographing object 4 is captured in the frame (stereo matching process area frame 8), and the stereo matching process is facilitated.

(14) A stereo imaging device that captures images of the imaging target from a plurality of viewpoints in order to perform stereo matching processing for obtaining three-dimensional information of the imaging target.
Stereo image capturing means for capturing the image to be captured;
Display means for displaying an image captured by the stereo image capturing means as an output image on a display screen;
Means for presenting a mark indicating a closed region in the output image, which is a region to be subjected to stereo matching processing existing inside the outer edge of the output image, on the display screen.
A stereo photographing apparatus comprising:

(Corresponding embodiment)
The first, second, third, sixth, and seventh embodiments correspond to the stereo imaging device described in (14).
(Function and effect)
According to the stereo photographing apparatus described in (14), an image obtained by photographing the photographing target 4 by the stereo image photographing unit is displayed as an output image on the display screen of the electronic finder 5 or the display 5A as the display unit. At this time, in order to show a closed region (stereo matching processing region 7) that is a closed region in the output image and that is subjected to stereo matching processing existing inside the outer edge of the output image, the display screen displays By presenting the frame (stereo matching processing area frame 8) as a mark so as to be visible, it is possible to capture the photographing object 4 in the frame and to easily perform the stereo matching process.

(15) A stereo imaging device that captures images of the imaging target from a plurality of viewpoints in order to perform stereo matching processing for obtaining three-dimensional information of the imaging target.
Stereo image capturing means for capturing the image to be captured;
Display means for displaying an image captured by the stereo image capturing means as an output image on a display screen;
Means for visibly presenting on the display screen a mark indicating a closed region in the output image, the closed region existing on the inner side of the region on which the stereo matching process exists on the inner side of the outer edge of the output image; ,
A stereo photographing apparatus comprising:

(Corresponding embodiment)
The first, second, third, sixth, and seventh embodiments correspond to the stereo imaging device described in (15).
(Function and effect)
According to the stereo photographing apparatus described in (15), an image obtained by photographing the photographing object 4 by the stereo image photographing means is displayed as an output image on the display screen of the electronic finder 5 or the display 5A as the display means. At this time, in order to indicate a closed region in the output image, and a closed region existing inside the region (stereo matching processing region 7) on which stereo matching processing exists inside the outer edge of the output image, By presenting a frame (stereo matching process area frame 8) as a mark on the display screen so as to be visible, it is possible to capture the photographing object 4 in the frame and to easily perform the stereo matching process.

(16) A stereo imaging device that captures images of the imaging target from a plurality of viewpoints in order to perform stereo matching processing for obtaining three-dimensional information of the imaging target.
Stereo image capturing means for capturing the image to be captured;
Display means for displaying an image captured by the stereo image capturing means as an output image on a display screen;
Means for visibly presenting on the display screen a mark indicating a closed region corresponding to a superimposed region in which the imaging spaces that can be captured from each of the plurality of viewpoints overlap each other, in the output image.
A stereo photographing apparatus comprising:

(Corresponding embodiment)
The first, second, third, sixth, and seventh embodiments correspond to the stereo imaging device described in (16).
(Function and effect)
According to the stereo photographing apparatus described in (16), an image obtained by photographing the photographing object 4 by the stereo image photographing unit is displayed as an output image on the display screen of the electronic finder 5 or the display 5A as the display unit. At this time, in order to indicate a closed region in the output image corresponding to a superimposed region in which imaging spaces that can be captured from each of the plurality of viewpoints overlap, a frame (stereo) as a mark is displayed on the display screen. Presenting the matching processing area frame 8) so as to be visible makes it easy to capture the photographing object 4 in the frame.

(17) A stereo imaging device that captures images of the imaging target from a plurality of viewpoints in order to perform stereo matching processing for obtaining three-dimensional information of the imaging target.
Stereo image capturing means for capturing the image to be captured;
Display means for displaying an image captured by the stereo image capturing means as an output image on a display screen;
A mark indicating a closed region in the output image, which is inside the region corresponding to the overlapping region where the imaging spaces that can be captured from each of the plurality of viewpoints overlap, is visible on the display screen. Means to present;
A stereo photographing apparatus comprising:

(Corresponding embodiment)
The first, second, third, sixth, and seventh embodiments correspond to the stereo imaging device described in (17).
(Function and effect)
According to the stereo photographing apparatus described in (17), an image obtained by photographing the photographing object 4 by the stereo image photographing means is displayed as an output image on the display screen of the electronic finder 5 or the display 5A as the display means. At this time, in order to show the closed region in the output image, the closed region existing inside the region corresponding to the overlapping region where the imaging spaces that can be captured from each of the plurality of viewpoints overlap each other on the display screen. By presenting a frame (stereo matching processing region frame 8) as a mark so as to be visible, it is easy to capture the photographing object 4 in the frame.

  (18) The mark is displayed or presented by fixing a member that transmits visible light and has a mark on a part thereof or a member made of the mark on the display screen (1). Stereo imaging apparatus according to any one of (7) and (14) to (17).

(Corresponding embodiment)
The embodiment relating to the stereo photographing apparatus described in (18) corresponds to the third embodiment.
(Function and effect)
According to the stereo photographing apparatus described in (18), the electronic finder 5 or the display 5A as a display means includes a member that transmits visible light and has a mark displayed on a part thereof or the mark. By presenting the frame (stereo matching processing region frame 8) as the mark by fixing the member, it is easy to capture the photographing object 4 in the frame.

  (19) The mark is displayed or presented by electrically superimposing the mark on the image displayed on the display screen. (1), (7), (14) to (17) The stereo imaging device according to any one of the above.

(Corresponding embodiment)
The first, second, sixth, and seventh embodiments correspond to the stereo imaging device described in (19).
(Function and effect)
According to the stereo photographing apparatus described in (19), the mark is electrically superimposed on the image displayed on the display screen of the electronic finder 5 or the display 5A as the display means, whereby the frame ( Presenting the stereo matching processing area frame 8) makes it easy to capture the photographing object 4 in the frame.

(20) further comprising an input means for operating the stereo photographing device;
The stereo photographing apparatus according to (19), wherein the display can be switched between display or presentation and non-display or non-presentation based on the result set by the input means.

(Corresponding embodiment)
The first, second, sixth, and seventh embodiments correspond to the stereo imaging device described in (20).
(Function and effect)
According to the stereo photographing apparatus described in (20), display or presentation of the mark and non-display or non-presentation may be performed according to a user's request, for example, a menu button of a digital camera constituting a stereo image capturing unit, It can be set by operating an input means such as a keyboard of a calculation device as a calculation means.

  (21) The stereo photographing apparatus according to (19), wherein the mark blinks.

(Corresponding embodiment)
The first, second, sixth, and seventh embodiments correspond to the stereo imaging device described in (21).
(Function and effect)
According to the stereo photographing apparatus described in (21), visibility can be improved by blinking the mark.

(22) In a finder of a stereo photographing apparatus including a finder optical system that forms an optical path for guiding an image to be photographed to the eyes of an operator,
In order to perform stereo matching processing for obtaining three-dimensional information of the imaging target, when capturing the imaging target image from a plurality of viewpoints
A means for presenting a mark indicating a closed region in the image, which is a region to be subjected to stereo matching processing existing on the inner side of the outer edge of the image, through the finder optical system. Finder.

(Corresponding embodiment)
The embodiment related to the viewfinder described in (22) corresponds to the third embodiment.
(Function and effect)
According to the finder described in (22), in the optical path of the finder optical system, a closed region in the image to be photographed, and a region to be subjected to stereo matching processing existing inside the outer edge of the image (stereo matching) By providing means for presenting a frame (stereo matching processing region frame 8) as a mark indicating the closed region which is the processing region 7) so as to be visible through the finder optical system, In addition to capturing the object 4 to be photographed, it is easy to perform stereo matching processing.

(23) In a finder of a stereo photographing apparatus including a finder optical system that forms an optical path that guides an image to be photographed to an operator's eyes,
In order to perform stereo matching processing for obtaining three-dimensional information of the imaging target, when capturing the imaging target image from a plurality of viewpoints,
Means for presenting a mark indicating a closed region in the image, which is a region inside the outer edge of the image and is present inside a region to which stereo matching processing is performed, is visible through the finder optical system. Finder characterized by that.

(Corresponding embodiment)
The embodiment related to the finder described in (23) corresponds to the third embodiment.
(Function and effect)
According to the finder described in (23), in the optical path of the finder optical system, a closed region in the image to be photographed, which is a closed region in the image, and is present inside the outer edge of the image. Means for presenting a frame (stereo matching processing region frame 8) as a mark indicating a closed region existing inside the region to be subjected to stereo matching processing (stereo matching processing region 7) so as to be visible through the finder optical system. Thus, by presenting the frame, it is easy to capture the photographing object 4 in the frame and perform the stereo matching process.

(24) A member that is attached to a viewfinder of an imaging device when capturing images of the imaging target from a plurality of viewpoints in order to perform stereo matching processing for obtaining three-dimensional information of the imaging target.
A mark presenting member which is a member which transmits visible light and has a mark presented on a part of the visible light or a member made of the mark.

(Corresponding embodiment)
The embodiment relating to the mark presenting member described in (24) corresponds to the third embodiment.
(Function and effect)
According to the mark presenting member described in (24), the mark presenting member is a member that transmits visible light and has a mark (stereo matching processing region frame 8) partially displayed thereon or a member made of the above mark. By attaching the member 30 to the viewfinder of the photographing apparatus, it is easy to capture the photographing object 4 in the frame.

(25) In a photographing method of a stereo photographing device that captures images to be photographed from a plurality of viewpoints and displays the captured images on a display screen.
A photographing method for a stereo photographing device, characterized in that a mark is displayed on the display screen to indicate a closed region existing inside an outer edge of the displayed image.

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments relating to the photographing method of the stereo photographing device described in (25).
(Function and effect)
According to the photographing method of the stereo photographing device described in (25), when the image of the photographing target 4 is picked up from a plurality of viewpoints, the above display is performed on the display screen for displaying the picked up image. By displaying a frame (stereo matching processing region frame 8) as a mark for indicating the closed region existing inside the outer edge of the image, it is easy to capture the photographing object 4 in the frame.

  (26) The distortion described in (6) or (12) is characterized in that the distortion is obtained based on an image obtained by imaging an object whose shape and / or surface attribute is known using the stereo image imaging means. Stereo shooting device.

(Corresponding embodiment)
The embodiment relating to the stereo photographing apparatus described in (26) corresponds to the fourth embodiment.
(Function and effect)
According to the stereo imaging device described in (26), the stereo image is acquired based on an image obtained by capturing the calibration pattern 11 having a known shape and / or surface attribute using the stereo image capturing unit. By obtaining the distortion of the output image captured by the image capturing means, the solid variation of the stereo image capturing means can be calibrated, and a frame (stereo matching processing region frame 8) as a correct mark can be displayed.

(27) A stereo imaging device that captures images of the imaging target from a plurality of viewpoints in order to perform stereo matching processing for obtaining three-dimensional information of the imaging target.
Stereo image capturing means for capturing the image to be captured;
Display means for displaying an image captured by the stereo image capturing means as an output image on a display screen;
Comprising
A stereo photographing apparatus characterized in that only the closed region in the output image, which is a region to be subjected to stereo matching processing existing inside the outer edge of the output image, is displayed by the display means.

(Corresponding embodiment)
The fifth to seventh embodiments correspond to the embodiment relating to the stereo photographing apparatus described in (27).
(Function and effect)
According to the stereo photographing apparatus described in (27), an image obtained by photographing the photographing object 4 by the stereo image photographing means is displayed as an output image on the display screen of the electronic finder 5 or the display 5A as the display means. At this time, only the closed region (stereo matching processing region 7) that is a closed region in the output image and that is subjected to stereo matching processing existing inside the outer edge of the output image is displayed by the display means. This makes it easy to capture the photographing object 4 in the stereo matching processing area 7 and perform the stereo matching process.

(28) A stereo imaging device that captures images of the imaging target from a plurality of viewpoints in order to perform stereo matching processing for obtaining three-dimensional information of the imaging target.
Stereo image capturing means for capturing the image to be captured;
Display means for displaying an image captured by the stereo image capturing means as an output image on a display screen;
Comprising
Stereo imaging, characterized in that only the closed region in the output image, which is present inside the region to which stereo matching processing exists inside the outer edge of the output image, is displayed by the display means. apparatus.

(Corresponding embodiment)
The fifth to seventh embodiments correspond to the embodiment relating to the stereo photographing apparatus described in (28).
(Function and effect)
According to the stereo photographing apparatus described in (28), an image obtained by photographing the photographing object 4 by the stereo image photographing means is displayed as an output image on the display screen of the electronic finder 5 or the display 5A as the display means. At this time, only the closed region in the output image that is inside the closed region (stereo matching processing region 7), which is the region on the inner side of the outer edge of the output image to which the stereo matching process is applied, is selected. By displaying by the display means, the photographing object 4 is surely captured in the stereo matching processing area 7 and the stereo matching processing is facilitated.

  (29) (1), (14) to (17), (27), wherein the size of the closed region is determined based on a distance from the photographing target to the stereo image capturing unit. And the stereo photographing device according to any one of (28).

(Corresponding embodiment)
The first to seventh embodiments correspond to the embodiment relating to the stereo photographing apparatus described in (29).
(Function and effect)
According to the stereo imaging device described in (29), the change in the shooting distance is obtained by changing the size of the closed region based on the change in the shooting distance from the shooting object 4 to the stereo image pickup means. Even if there is, it becomes easy to capture the photographing object 4 in an appropriate area.

  (30) The size of the first closed region or the second closed region is determined based on a distance from the photographing target to the stereo image capturing unit. Stereo shooting device.

(Corresponding embodiment)
The first, second, third, fourth, sixth, and seventh embodiments correspond to the embodiments relating to the stereo photographing device described in (30).
(Function and effect)
According to the stereo imaging device described in (30), the size of the first closed region or the second closed region is based on the change in the shooting distance from the shooting target 4 to the stereo image pickup means. By changing the, it becomes easy to capture the photographing object 4 in an appropriate area even if the photographing distance changes.

(A) is a figure which shows the state which image | photographs imaging | photography object while showing the structure of the stereo imaging device by the 1st Embodiment of this invention, (b) is a digital camera as shown to (a), and calculation. It is a figure which illustrates the stereo matching process area | region frame displayed electronically on an electronic finder in order to show a stereo matching process area | region to a photographer, when image | photographing imaging | photography objects using an apparatus. It is the top view and front view which show the structure of the digital camera as a stereo image imaging means used for the stereo imaging device by 1st Embodiment, and the stereo adapter with which the front part of this digital camera is mounted | worn. (A) to (c) are each presented as a dotted line, a broken line, a black frame (however, in the drawing, indicated by a hatched frame) or the like as a stereo matching processing region frame in the stereo imaging apparatus according to the first embodiment. It is a figure which shows the modified example. It is a figure which shows the modification of the stereo matching process area | region frame shown in the stereo imaging device by 1st Embodiment. In the stereo imaging device according to the first embodiment, when a stereo matching processing area is determined using lens distortion or the like as a threshold value, a semicircular frame with a black frame as a stereo matching processing area frame (however, in the drawing, it is indicated by a hatched frame) It is a figure which shows the modification shown by this. It is a perspective view which shows the structure of the stereo imaging device by the 2nd Embodiment of this invention. It is a figure which shows the structural example of the finder of the stereo imaging device by the 3rd Embodiment of this invention in the case of a twin lens type and the case of a single lens reflex type. (A) is a figure which shows landmark presentation members, such as a film and a panel which permeate | transmit visible light with which the stereo matching process area frame was shown as a modification of 3rd Embodiment, (b) is also a stereo matching process It is a figure which shows the perforation member in which the hole which permeate | transmits visible light in the shape matched with the shape of the area | region frame was opened. It is a figure which shows the example of the calibration pattern utilized for the stereo imaging device by the 4th Embodiment of this invention. It is a figure which shows the imaging | photography condition of a calibration pattern while showing the structure of the stereo imaging device by 4th Embodiment. Relationship between the search range when the measurement distance range is not set in the stereo image capturing apparatus according to the sixth embodiment of the present invention, the range in which the search for the corresponding point N of the search point M is performed based on the minimum parallax and the measurement distance range It is a figure which shows the reference | standard image and corresponding image for demonstrating. (A) is a figure which shows the reference | standard image and corresponding image for demonstrating the search unnecessary area | region which is not used for stereo matching, (b) is the case where both the farthest distance value and the nearest distance value are set as a measurement distance range It is a figure which shows the reference | standard image and corresponding image for demonstrating the stereo matching process area | region of this, and a stereo matching process area | region frame. It is a figure which shows the example which acquires the some image 2 from a some viewpoint, and uses one image as the reference | standard image in the three-dimensional information acquisition method by the stereo measurement of a prior art. It is a figure for demonstrating the superimposition area | region which is an imaging space which can image from each of several viewpoints in case a convergence angle differs.

Explanation of symbols

    DESCRIPTION OF SYMBOLS 1 ... Stereo imaging system (stereo image imaging means), 3 ... Reference | standard image, 4 ... Shooting object, 5 ... Electronic finder (display means), 5A ... Display (display means), 6 ... Calculation apparatus (calculation means), 7 ... Stereo matching processing area, 8 ... Stereo matching processing area frame, 9 ... Stereo adapter (stereo image imaging means), 10 ... Digital camera (stereo image imaging means), 11 ... Calibration pattern (both shape and / or surface attributes are Known object), 12 ... corresponding image, 13 ... calibration pattern image (image obtained by capturing an object with known shape and / or surface attribute), 17 ... half mirror, 18 ... mirror, 19 ... user's eye 20 ... diffusion plate, 22 ... eyepiece, 23 ... prism, 24 ... movable mirror, 25 ... Image sensor, 26 ... Storage device, 27 ... Glass plate, 30 ... Mark presentation member, 33 ... Overlapping area, 36 ... Hole-opening member, 40 ... Search range when no measurement distance range is set, 41 ... Measurement A range in which the corresponding point N of the search point M is searched based on the distance range, 42 ... minimum parallax, 43 ... maximum parallax, 51 ... left viewpoint, 52 ... right viewpoint.

Claims (15)

Stereo image capturing means for capturing images to be photographed from a plurality of viewpoints;
Display means for displaying an image picked up by the stereo image pickup means as an output image on a display screen;
Calculation means for performing a stereo matching process on a first closed region that is a region in an image captured by the stereo imaging unit and that is a region determined in advance before imaging the imaging target;
When the imaging object is imaged, a mark indicating the second closed area included in the first closed area is displayed on the display unit as a mark for capturing the imaging object in the first closed area. Means for displaying on the screen;
A stereo photographing apparatus comprising:   The stereo photographing apparatus according to claim 1, wherein the first closed region is inside an outer edge of the output image.   The stereo imaging device according to claim 1, wherein the second closed region is located inside an outer edge of the first closed region.   The mark is at least a position of a corner of the second closed region by arranging a line segment, a crossed line segment, a curve, a crossed curve, a point, or a closed curve. The stereo photographing apparatus according to 1.   The stereo photographing apparatus according to claim 1, wherein the mark surrounds an outer edge of the second closed region.   The stereo imaging apparatus according to claim 1, wherein the mark is electrically superimposed on the output image captured by the stereo image capturing unit.   The stereo photographing apparatus according to claim 1, wherein the mark is fixed on the display screen.   The stereo imaging apparatus according to claim 1, wherein the second closed region is specified by the calculation unit based on characteristics of the stereo image capturing unit.   The stereo photographing apparatus according to claim 1, wherein the mark blinks.   The stereo imaging device according to claim 1, wherein the first closed region is determined based on distortion of the output image captured by the stereo image capturing unit.   The stereo imaging device according to claim 10, wherein the distortion of the first closed region determined based on the distortion of the output image is equal to or lower than a predetermined level.   The stereo imaging apparatus according to claim 10, wherein the distortion is obtained based on an image obtained by capturing an object having a known shape and / or surface attribute using the stereo image capturing unit.   The stereo imaging device according to claim 1, wherein the first closed region is determined based on a measurement distance range measured using an image captured by the stereo image capturing unit. In order to perform a stereo matching process for obtaining three-dimensional information of a shooting target, the stereo shooting device captures images of the shooting target from a plurality of viewpoints,
Stereo image imaging means for imaging the image to be imaged;
Display means for displaying an image picked up by the stereo image pickup means as an output image on a display screen;
A second closed region included in “a first closed region that is determined in advance before imaging the imaging target as the region to be subjected to the stereo matching process in the image captured by the stereo imaging unit” Means for displaying on the display screen of the display means as a mark for capturing the photographing object in the first closed region when imaging the photographing object;
A stereo photographing apparatus comprising: In an imaging method of a stereo imaging device that captures an image to be subjected to stereo matching processing for obtaining 3D information of an imaging target from a plurality of viewpoints and displays the captured image on a display screen.
A second closed region included in “a first closed region that is determined in advance before imaging the imaging target as the region to be subjected to the stereo matching process in the image captured by the stereo imaging unit” A photographing method for a stereo photographing device, wherein when the photographing object is imaged, the photographing object is displayed on the display screen as a mark for capturing the photographing object in the first closed region.

Images