JP5161025B2 - Imaging apparatus and method for three-dimensional shape measurement, and program - Google Patents

Description

  The present invention relates to a three-dimensional shape measurement photographing apparatus and method for acquiring a measurement image for measuring a three-dimensional shape of a subject, and a program for causing a computer to execute the three-dimensional shape measurement photographing method.

  A distance measurement method generally referred to as a stereo method or a stereo three-dimensional image measurement method, etc., shoots an object using at least two cameras (one reference camera and other reference cameras) provided at different positions. Then, the pixels are associated between the plurality of measurement images (the standard image by the standard camera and the reference image by the reference camera) acquired in this way, and the pixels on the standard image and the pixels on the reference image associated with each other By applying the principle of triangulation to the difference in position (parallax), the distance from the reference camera or reference camera to the point on the subject corresponding to the pixel is measured.

  Therefore, if the distances to all the pixels corresponding to the entire surface of the subject are measured, a three-dimensional shape such as the shape and depth of the subject can be measured. It is also possible to measure a three-dimensional shape of a subject by acquiring a plurality of images by shooting at different positions using only one camera.

  Here, in the stereo method, as shown in FIG. 12, the points in the real space mapped to a certain point Pa on the reference image G1 are on the line of sight from the point C, such as points P1, P2, and P3. Since there are a plurality of pixels, a pixel Pa based on the presence of a point Pa ′ on the reference image G2 corresponding to the point Pa on a straight line (epipolar line) that is a map of the points P1, P2, P3, etc. in the real space. Association is performed. In FIG. 12, point C is the viewpoint of the base camera, and point C ′ is the viewpoint of the reference camera.

  However, in the stereo method, when the subject does not have local features such as shading, shape, and color like a human face, there is a problem in that corresponding points cannot be obtained by performing pixel association. is there. In order to solve such a problem, a method has been proposed in which a subject is irradiated with pattern light such as a random dot pattern, a lattice pattern, or a barcode pattern, and corresponding points are obtained based on the position of the pattern light ( (See Patent Documents 1 and 2).

There has also been proposed a method for measuring the contrast of a subject and determining whether or not pattern light is irradiated according to the measurement result (see Patent Document 3).
JP 2004-212385 A Japanese Patent Laid-Open No. 2000-283755 Japanese Patent Laid-Open No. 2002-300605

  However, since the method described in Patent Document 3 sets the presence or absence of pattern light irradiation for the entire range of the angle of view, it has a portion with high contrast in the range of the angle of view and local features. The pattern light is also irradiated to the portion. When pattern light is irradiated onto a portion with high contrast or a local feature in this way, the high-frequency component of the acquired image increases. For this reason, moire occurs in the image corresponding to the portion, and therefore, the corresponding point cannot be searched well, and as a result, the distance to the subject cannot be measured accurately. There is.

  The present invention has been made in view of the above circumstances, and an object thereof is to accurately measure the three-dimensional shape of a subject using an image acquired by irradiating the subject with pattern light.

A three-dimensional shape measurement photographing apparatus according to the present invention, by photographing a subject, a plurality of photographing means for acquiring a measurement image group composed of a plurality of measurement images for measuring the three-dimensional shape of the subject,
Irradiating means for irradiating pattern light within the range of the angle of view by the photographing means;
The plurality of photographing units are controlled so as to acquire a pre-image of the subject or to acquire a first measurement image group including a plurality of first measurement images of the subject not irradiated with the pattern light. Photographing control means to
And a determination unit configured to set a plurality of regions on the pre-image or at least one of the first measurement images, and to determine the presence / absence of the pattern light irradiation for each region. .

  A “pre-image” is an image that is not used for measuring a three-dimensional shape. For example, an image having a smaller number of pixels can be used as compared with a measurement image.

  The three-dimensional shape measurement imaging apparatus according to the present invention may further include a storage unit that stores the determination result.

The three-dimensional shape measurement imaging apparatus according to the present invention further includes an irradiation control unit that controls the irradiation unit to irradiate the subject with pattern light.
The imaging control unit may be a unit that controls the imaging control unit to acquire a second measurement image group including a plurality of second measurement images of the subject irradiated with the pattern light.

The three-dimensional shape measurement imaging apparatus according to the present invention further includes an irradiation control unit that controls the irradiation unit to irradiate the subject with pattern light.
The imaging control unit is configured to include a second measurement image including a plurality of second measurement images of the subject irradiated with the pattern light when there is an area determined to be irradiated with the pattern light among the plurality of areas. It is good also as a means to control the said imaging | photography control means so that the image group for a measurement may be acquired.

  In addition, the three-dimensional shape measurement imaging apparatus according to the present invention may further include measurement means for measuring the three-dimensional shape of the subject based on the first and second measurement image groups.

  In the three-dimensional shape measurement imaging apparatus according to the present invention, a non-irradiation region where the pattern light is determined to be non-irradiation is extracted from the first measurement image, and the pattern is extracted from the second measurement image. A composite image group consisting of a plurality of composite images from the non-irradiation area and the irradiation area extracted from the first and second measurement image groups corresponding to each other, extracted from the irradiation area determined to emit light. It is good also as a thing further provided with the synthetic | combination means which produces | generates.

  In addition, the three-dimensional shape measurement imaging apparatus according to the present invention may further include a measurement unit that measures the three-dimensional shape of the subject based on the composite image group.

A three-dimensional shape measurement photographing method according to the present invention includes a plurality of photographing means for photographing a subject to obtain a measurement image group including a plurality of measurement images for measuring the three-dimensional shape of the subject;
An imaging method for 3D shape measurement in an imaging device for 3D shape measurement comprising irradiation means for irradiating pattern light within a range of an angle of view by the imaging means,
Obtaining a first measurement image group consisting of a plurality of first measurement images of the subject that is not irradiated with the pre-image of the subject or the pattern light;
A plurality of regions are set on the pre-image or at least one of the first measurement images, and the presence / absence of the pattern light irradiation is determined for each region.

  In addition, you may provide as a program for making a computer perform the imaging | photography method for three-dimensional shape measurement by this invention.

  According to the present invention, a first measurement image group including a plurality of first measurement images of a subject that has not been irradiated with a pre-image or pattern light of the subject is acquired, and the pre-image or at least one first measurement is acquired. A plurality of areas are set on the image for use, and the presence or absence of pattern light irradiation is determined for each set area. Therefore, the subject is photographed by irradiating the pattern light, and a second measurement image group including a plurality of second measurement images of the subject irradiated with the pattern light is obtained, and based on the determination result. Search for corresponding points using the second measurement image group for parts with high contrast in the field of view range or parts having local features, and for the other parts using the first measurement image group By doing so, it is possible to accurately obtain corresponding points using a high-quality image that does not include moire or the like, and as a result, it is possible to accurately measure the three-dimensional shape of the subject.

  Further, by storing the determination result, when measuring the three-dimensional shape of the subject using the first and second measurement image groups, the irradiation region and the pattern light irradiated with the pattern light according to the determination result Since the non-irradiated region that is not irradiated with can be acquired from the first and second image groups for measurement, it is possible to easily measure a three-dimensional shape.

  In addition, a non-irradiation area that is determined to be non-irradiated with pattern light is extracted from the first image group for measurement, and an irradiation area that is determined to be irradiated with pattern light is extracted from the second image group for measurement. By generating a composite image group composed of a plurality of composite images from the non-irradiation area and the irradiation area extracted from the first and second measurement image groups, the contrast included in the range of angle of view in the composite image is A high part or a part having a local feature may be not irradiated with pattern light. Therefore, even if the range of the angle of view has a high contrast or includes a portion having a local feature, it is possible to obtain a high-quality composite image that does not include moire, etc., and therefore, corresponding points can be obtained accurately. As a result, the three-dimensional shape of the subject can be accurately measured.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an external perspective view of a three-dimensional shape measuring apparatus to which a three-dimensional shape measuring photographing apparatus according to the first embodiment of the present invention is applied. As shown in FIG. 1, the three-dimensional shape measuring apparatus 1 according to the present embodiment includes a first camera 2, a second camera 3, a projector 4, a calculation unit 5, a monitor 6, and an input unit 7.

  The first and second cameras 2 and 3 are arranged with a predetermined interval and a predetermined convergence angle, and in accordance with instructions from the calculation unit 5, by photographing the subject from different positions, Images used for measuring the three-dimensional shape of the subject are acquired. The acquired image is input to the calculation unit 5 and used for measuring the three-dimensional shape of the subject. The first and second cameras 2 and 3 correspond to photographing means.

  The projector 4 is for irradiating a subject with pattern light having a predetermined pattern such as a lattice pattern in accordance with an instruction from the calculation unit 5. The projector 4 emits light such as an LED and the like from the light source 11 and the light source 11. A filter unit 12 is provided that converts emitted light into pattern light. In addition, the projector 4 and the irradiation control part 21 mentioned later correspond to an irradiation means.

  The filter unit 12 is configured by attaching a pattern filter 12B and a transparent filter 12C on which a pattern is printed so that the transmitted light has a predetermined pattern at opposing positions on the disk 12A across the center. Has been. In the present embodiment, a grid pattern is used as the pattern. The disk 12A can be rotated around a rotation axis parallel to the optical path X0 of the light emitted from the light source 11 through the center thereof by the motor 12D. Thereby, according to the instruction | indication from the calculating part 5, it is possible to switch and arrange | position the pattern filter 12B and the transparent filter 12C on the optical path X0.

  Here, by providing the filter 12 with the transparent filter 12C, the light can be emitted from the projector 4 to the subject when the amount of light of the subject is insufficient.

  The computing unit 5 performs computations necessary for measuring the three-dimensional shape of the subject and controls driving of the first camera 2, the second camera 3, and the projector 4.

  FIG. 2 is a schematic block diagram showing the configuration of the calculation unit 5. As shown in FIG. 2, the calculation unit 5 includes an irradiation control unit 21 that controls the driving of the projector 4, a shooting control unit 22 that controls the operation of the first and second cameras 2 and 3, and an irradiation range of light on the subject. A range setting unit 23 for setting the object, a determination unit 24 for determining the presence or absence of pattern light irradiation on the subject, and a measurement unit for measuring the three-dimensional shape of the subject using the images acquired by the first and second cameras 2 and 3 25, the file generation unit 26 that generates an image file of an image acquired by the first and second cameras 2 and 3, the recording unit 27 that records the image file on a medium (not shown), and the overall operation of the calculation unit 5. An overall control unit 28 for controlling, the irradiation control unit 21, the imaging control unit 22, the range setting unit 23, the determination unit 24, the measurement unit 25, the file generation unit 26, the recording unit 27, and the overall control unit 28. Input section It is connected by a bus 29 with.

  The overall control unit 28 includes a CPU 28A, a RAM 28B as a work area, and a ROM 28C in which various information including an operation / control program is stored.

  The irradiation control unit 21 controls on / off of the light source 11 according to an instruction from the overall control unit 28. Further, in response to an instruction from the overall control unit 28, an instruction to rotate the motor 12D is issued in order to position either the pattern filter 12B or the transparent filter 12C of the filter unit 12 on the optical path X0.

  When an instruction for 3D shape measurement is given from the input unit 7, the imaging control unit 22 instructs the first and second cameras 2 and 3 to capture an image of the subject and acquire an image for measuring the 3D shape. Against. In addition, an instruction to acquire a pre-image is given to the first camera 2 before acquiring an image for three-dimensional shape measurement. Further, the first and second cameras 2 and 3 are instructed to perform imaging without pattern light irradiation and imaging with pattern light irradiation.

  The range setting unit 23 sets a range in which the light emitted from the projector 4 is irradiated on the pre-image. FIG. 3 is a diagram for explaining the setting of the light irradiation range. As illustrated in FIG. 3, the range setting unit 23 sets an irradiation range R0 including 4 × 4 = 16 areas on the pre-image. Here, the position of each region in the irradiation range R0 is specified by the coordinate value when the xy coordinate system is set as shown in FIG.

  Note that the position and the number of divisions of the irradiation range R0 are not limited to this, and for example, the irradiation range R0 is narrowed as shown in FIGS. 4A and 4B, or as shown in FIG. Thus, the number of divisions may be 2 × 2. The irradiation range R0 may be set in advance by an instruction from the input unit 7 by the user, and is set by displaying a pre-image on the monitor 6 and viewing the displayed pre-image by the user. May be.

  The determination unit 24 calculates an evaluation value for determining the presence or absence of pattern light irradiation in each region of the irradiation range R0 set in the pre-image. Specifically, the determination unit 24 detects edges included in the image in the region by calculation using a differential filter or the like, and uses the number of detected edges as an evaluation value. Note that the contrast of the image in the region may be detected, and the detected contrast may be used as the evaluation value.

  Here, the evaluation value is low when the image in the region has few edges. The evaluation value also decreases when the contrast of the image in the region is low. Thus, when the evaluation value of the image in the region is low, it is not possible to accurately obtain corresponding points for measuring the three-dimensional shape as will be described later. For this reason, the determination unit 24 compares the evaluation value with the threshold value Th1, and when the evaluation value exceeds the threshold value Th1, the determination unit 24 determines that the region is a non-irradiation region where pattern light is not irradiated. On the other hand, when the evaluation value is equal to or less than the threshold value Th1, the region is determined as an irradiation region to which pattern light is irradiated.

  For example, in the case where the irradiation range R0 is set as shown in FIG. 3, the determination unit 24 determines that the pattern is irradiated to the regions at the positions (1, 1), (1, 2), and (4, 4). In this case, as shown in FIG. 5, the areas at the positions (1, 1), (1, 2) and (4, 4) are the irradiation areas, and the areas at the other positions are the non-irradiation areas. In FIG. 5, “◯” is given to the irradiated region and “X” is given to the non-irradiated region.

  The measurement unit 25 is irradiated with the non-irradiation reference image G1 and the non-irradiation reference image G2 acquired by the first and second cameras 2 and 3 and the pattern light by photographing a subject that is not irradiated with the pattern light. The three-dimensional shape of the subject is measured based on the irradiation reference image G1 ′ and the irradiation reference image G2 ′ acquired by the first and second cameras 2 and 3 by photographing the subject. The non-irradiation standard image G1 and the non-irradiation reference image G2 correspond to the first measurement image, and the irradiation standard image G1 ′ and the irradiation reference image G2 ′ correspond to the second measurement image.

  Note that, when measuring a three-dimensional shape, a corresponding point is obtained, but when a pixel for which the corresponding point is obtained is in a non-irradiation region, a matching window is set on the non-irradiation reference image G1 and the non-irradiation reference image G2. Then, using the information on the epipolar line, the pixels on the non-irradiation reference image G1 and the non-irradiation reference image G2 are associated to obtain corresponding points. On the other hand, if the pixel for which the corresponding point is to be found is in the irradiation area, a matching window is set on the irradiation reference image G1 ′ and the irradiation reference image G2 ′, and information on the epipolar line is used to set the irradiation reference image G1 ′ and the irradiation reference image. Corresponding points are obtained by associating pixels on the reference image G2 ′.

  Here, the measuring unit 25 represents an internal parameter representing the positional relationship between the optical paths of the lenses of the first and second cameras 2 and 3 and the imaging surface, and the positional relationship between the first and second cameras 2 and 3. External parameters are stored in the ROM 28C, and corresponding points are obtained by associating pixels with reference to internal parameters and external parameters. The internal parameters and external parameters are obtained in advance by camera calibration and stored in the ROM 28C. Then, when the pixel association is completed, the measurement unit 25 calculates the three-dimensional shape of the subject based on the corresponding points, and outputs the calculation result to the monitor 6 as a distance image.

  The file generation unit 26 performs compression processing on the non-irradiation standard image G1 and the non-irradiation reference image G2 and the image data representing the irradiation standard image G1 ′ and the irradiation reference image G2 ′ in a compression format such as JPEG, for example. Generate an image file. The image file includes image data representing the non-irradiation standard image G1 and the non-irradiation reference image G2, and the irradiation standard image G1 ′ and the irradiation reference image G2 ′. In addition, a tag in which incidental information such as shooting date and time is stored is assigned to the image file based on the Exif format or the like. In the present embodiment, the tag describes information for specifying the position of the non-irradiation region and the irradiation region within the irradiation range R0.

  The recording unit 27 records the image file generated by the file generation unit 26 on a medium.

  Next, processing performed in the first embodiment will be described. FIG. 6 is a flowchart showing processing performed in the first embodiment. First, the overall control unit 28 monitors whether or not an instruction for photographing for measuring a three-dimensional shape is given (step ST1), and when the step ST1 is affirmed, the first camera 2 performs a pre-image. (Step ST2), the range setting unit 23 sets the irradiation range R0 in the pre-image (step ST3).

  Then, the overall control unit 28 sets the determination target region among the regions within the irradiation range R0 as the first region (i = 1, step ST4), and the determination unit 24 calculates the evaluation value of the target region. Then, it is determined whether or not the evaluation value is equal to or less than the threshold value Th1 (step ST6). If step ST6 is affirmed, the determination unit 24 sets the region as an irradiation region (step ST7). If step ST6 is negative, the determination unit 24 sets the area as a non-irradiation area (step ST8).

  Then, it is determined whether or not the overall control unit 28 has made the determination for all regions (step ST9). If step ST9 is negative, the determination target is changed to the next region (i = i + 1, step ST10). ), The process returns to step ST5.

  If step ST9 is affirmed, the imaging control unit 22 performs imaging with the first and second cameras 2 and 3 (non-irradiation imaging, step ST11). Thereby, the non-irradiation standard image G1 and the non-irradiation reference image G2 are acquired.

  Next, the irradiation control unit 21 sets pattern light emission for the projector 4 (step ST12). That is, the irradiation control unit 21 turns on the light source 11 and instructs the motor 12D to rotate the filter unit 12 so that the pattern filter 12B is positioned on the optical path X0 of the light emitted from the light source 11. As a result, the light emitted from the light source 11 is converted into pattern light by the pattern filter 12B and irradiated to the subject. The pattern light reflected from the subject is incident on the first and second cameras 2 and 3.

  And the imaging | photography control part 22 image | photographs with the 1st and 2nd cameras 2 and 3 (irradiation imaging | photography, step ST13). Thereby, the irradiation standard image G1 ′ and the irradiation reference image G2 ′ are acquired. Then, the file generation unit 26 generates an image file from the image data representing the non-irradiation standard image G1 and the non-irradiation reference image G2, and the irradiation standard image G1 ′ and the irradiation reference image G2 ′ (step ST14). Note that information specifying the positions of the non-irradiation region and the irradiation region in the irradiation range R0 is described in the tag of the image file. Then, the recording unit 27 records the image file on the medium (step ST15), and the process ends.

  Thus, in the first embodiment, the irradiation range R0 including a plurality of regions is set on the pre-image, and the presence / absence of pattern light irradiation is determined for each region within the irradiation range R0. Along with the reference image G1 and the non-irradiation reference image G2, the subject is photographed by irradiating pattern light to obtain an irradiation reference image G1 ′ and an irradiation reference image G2 ′. The irradiation standard image G1 ′ and the irradiation reference image G2 ′ are used for a portion having a high contrast in the corner range and a portion having a local feature, and the non-irradiation reference image G1 and the non-irradiation reference image G2 are used for other portions. By searching for corresponding points, it is possible to accurately obtain corresponding points using a high-quality image that does not include moire, etc. As a result, the three-dimensional shape of the subject can be determined. It is possible to measure a good time.

  Next, a second embodiment of the present invention will be described. Note that the configuration of the three-dimensional shape measurement apparatus according to the second embodiment is the same as the configuration of the three-dimensional shape measurement apparatus according to the first embodiment, and a detailed description thereof will be omitted here. In the second embodiment, non-irradiation imaging and irradiation imaging are performed first without acquiring a pre-image, and the determination unit 24 uses either one of the non-irradiation reference image G1 and the non-irradiation reference image G2. The difference from the first embodiment is that the determination is made. In the second embodiment, the determination is performed using the non-irradiation reference image G1.

  Next, processing performed in the second embodiment will be described. FIG. 7 is a flowchart showing processing performed in the second embodiment. First, the overall control unit 28 monitors whether or not an instruction for imaging for measuring a three-dimensional shape is given (step ST21). When step ST21 is affirmed, the imaging control unit 22 performs first and second imaging operations. Photographing is performed by the second cameras 2 and 3 (non-irradiation photographing, step ST22). Thereby, the non-irradiation standard image G1 and the non-irradiation reference image G2 are acquired.

  Next, the irradiation control unit 21 sets the pattern light emission for the projector 4 (step ST23), and the shooting control unit 22 performs shooting with the first and second cameras 2 and 3 (irradiation shooting, step ST24). Thereby, the irradiation standard image G1 ′ and the irradiation reference image G2 ′ are acquired.

  Next, the range setting unit 23 sets the irradiation range R0 in the non-irradiation reference image G1 (step ST25). Then, the overall control unit 28 sets the region to be determined as the first region among the regions within the irradiation range R0 (i = 1, step ST26), and the determination unit 24 calculates the evaluation value of the target region. Then, it is determined whether or not the evaluation value is equal to or less than the threshold value Th1 (step ST28). If step ST28 is affirmed, the determination unit 24 sets the region as an irradiation region (step ST29). If step ST28 is negative, the determination unit 24 sets the area as a non-irradiation area (step ST30).

  Then, it is determined whether or not the overall control unit 28 has made the determination for all regions (step ST31). If step ST31 is negative, the determination target is changed to the next region (i = i + 1, step ST32). ), The process returns to step ST27.

  If step ST31 is positive, the file generation unit 26 generates an image file from image data representing the non-irradiation standard image G1, the non-irradiation reference image G2, and the irradiation standard image G1 ′ and the irradiation reference image G2 ′ (step ST33). ). Note that information specifying the positions of the non-irradiation region and the irradiation region in the irradiation range R0 is described in the tag of the image file. Then, the recording unit 27 records the image file on the medium (step ST34), and the process ends.

  In the second embodiment, the determination is performed using the non-irradiation reference image G1, but the determination may be performed using the non-irradiation reference image G2.

  Next, a third embodiment of the present invention will be described. Note that the configuration of the three-dimensional shape measurement apparatus according to the third embodiment is the same as the configuration of the three-dimensional shape measurement apparatus according to the first embodiment, and a detailed description thereof will be omitted here. In the third embodiment, the determination unit 24 obtains the non-irradiation standard image G1 and the non-irradiation reference image G2 by performing the non-irradiation photographing first without acquiring the pre-image, and the non-irradiation standard image G1 and This is different from the first embodiment in that determination is performed using any one of the non-irradiation reference images G2 and irradiation imaging is performed only when necessary. In the third embodiment, the determination is made using the non-irradiation reference image G1.

  Next, processing performed in the third embodiment will be described. FIG. 8 is a flowchart showing processing performed in the third embodiment. First, the overall control unit 28 monitors whether or not a shooting instruction for three-dimensional shape measurement is given (step ST41). If step ST41 is affirmed, the shooting control unit 22 performs first and second shootings. Photographing is performed by the second cameras 2 and 3 (non-irradiation photographing, step ST42). Thereby, the non-irradiation standard image G1 and the non-irradiation reference image G2 are acquired.

  Next, the range setting unit 23 sets the irradiation range R0 in the non-irradiation reference image G1 (step ST43). Then, the overall control unit 28 sets the determination target region among the regions within the irradiation range R0 as the first region (i = 1, step ST44), and the determination unit 24 calculates the evaluation value of the target region. Then, it is determined whether or not the evaluation value is equal to or less than the threshold value Th1 (step ST46). If step ST46 is affirmed, the determination unit 24 sets the region as an irradiation region (step ST47). If step ST46 is negative, the determination unit 24 sets the area as a non-irradiation area (step ST48).

  Then, it is determined whether or not the overall control unit 28 has made the determination for all regions (step ST49). If step ST49 is negative, the determination target is changed to the next region (i = i + 1, step ST50). ), The process returns to step ST45.

  If step ST49 is affirmed, it is determined whether pattern light needs to be irradiated (step ST51). Here, when all the areas are set as non-irradiation areas, it is not necessary to irradiate the pattern light, so step ST51 is denied. However, when there is even one area set as the irradiation area, step ST51 is performed. Is affirmed. If step ST51 is affirmed, the irradiation control unit 21 sets pattern light emission for the projector 4 (step ST52), and the imaging control unit 22 performs imaging with the first and second cameras 2 and 3 (irradiation imaging). Step ST53). Thereby, the irradiation standard image G1 ′ and the irradiation reference image G2 ′ are acquired. Then, the file generation unit 26 generates an image file from the image data representing the non-irradiation standard image G1 and the non-irradiation reference image G2, and the irradiation standard image G1 ′ and the irradiation reference image G2 ′ (step ST54). Note that information specifying the positions of the non-irradiation region and the irradiation region in the irradiation range R0 is described in the tag of the image file. Then, the recording unit 27 records the image file on the medium (step ST55), and the process ends.

  On the other hand, if step ST51 is negative, the process proceeds to step ST54. In this case, the file generation unit 26 generates an image file from the image data representing the non-irradiation reference image G1 and the non-irradiation reference image G2, and the position of only the non-irradiation region in the irradiation range R0 is set as a tag of the image file. Information to be identified is described.

  In the first embodiment, irradiation shooting is performed. However, it is determined whether pattern light irradiation is necessary based on a determination result using a pre-image, and pattern light irradiation is necessary. Irradiation photography may be performed only when it is determined that.

  Next, a fourth embodiment of the present invention will be described. In the fourth embodiment, since only the configuration of the calculation unit 5 is different from the first to third embodiments, only the configuration of the calculation unit 5 will be described here. FIG. 9 is a diagram showing a configuration of a calculation unit in the three-dimensional shape measurement apparatus according to the fourth embodiment of the present invention. As shown in FIG. 9, the calculation unit 5A of the three-dimensional shape measurement apparatus according to the second embodiment performs a composite image C1 of the non-irradiation standard image G1 and the irradiation standard image G1 ′, as well as the non-irradiation reference image G2 and the irradiation reference image G2. It differs from the first to third embodiments in that it includes a combining unit 30 that generates a composite image C2 ′.

  Next, processing performed in the fourth embodiment will be described. In the fourth embodiment, the process after step ST13 in the first embodiment is the second embodiment, and the process after step ST31 in the second embodiment is affirmed. Since the form and the processing after step ST53 in the third embodiment are different from those of the third embodiment, only the processing after these steps will be described here.

  Subsequent to step ST13 in the first embodiment, step ST31 in the second embodiment (when affirmed), and step ST53 in the third embodiment, the combining unit 30 performs the non-irradiation reference image G1 and the irradiation reference image G1. 'And a composite image C2 of the non-irradiation reference image G2 and the irradiation reference image G2' are generated (step ST71).

  FIG. 11 is a diagram for explaining generation of a composite image. Note that the process of generating the composite image C1 from the non-irradiation standard image G1 and the irradiation standard image G1 ′ and the process of generating the composite image C2 from the non-irradiation reference image G2 and the irradiation reference image G2 ′ are the same. Only the generation of the composite image C1 from the non-irradiation reference image G1 and the irradiation reference image G1 ′ will be described.

  First, the synthesis unit 30 extracts a region set as a non-irradiation region among a plurality of regions corresponding to the irradiation range R0 in the non-irradiation reference image G1. For example, when the irradiation region and the non-irradiation region are set as shown in FIG. 5, the thick lines other than the regions at the positions (1, 1), (1, 2) and (4, 4) in the non-irradiation reference image G1. Extract the area surrounded by. The synthesizing unit 30 extracts a region set as the irradiation region among a plurality of regions corresponding to the irradiation range R0 in the irradiation reference image G1 ′. For example, when the irradiation region and the non-irradiation region are set as shown in FIG. 5, the positions of (1, 1), (1, 2), and (4, 4) surrounded by a thick line in the irradiation reference image G1 ′ Extract the region. Then, the combining unit 30 combines the extracted regions to generate a combined image C1. Since the non-irradiation reference image G1 and the irradiation reference image G1 ′ are taken at different points in time, the deviation of the angle of view at the time of shooting between the non-irradiation reference image G1 and the irradiation reference image G1 ′ is corrected. It is preferable to perform extraction and synthesis of the non-irradiated region and the irradiated region later.

  A composite image C2 is similarly generated from the non-irradiation reference image G2 and the irradiation reference image G2 ′. Here, since the irradiation range R0 is set to the pre-image or the non-irradiation reference image G1, the irradiation set on the pre-image or the non-irradiation reference image G1 on the non-irradiation reference image G2 or the irradiation reference image G1 ′. An area corresponding to the range R0 may be set, and a non-irradiation area and an area corresponding to the irradiation area in the set area may be extracted to generate the composite image C2.

  Next, the file generation unit 26 generates an image file from the image data representing the composite images C1 and C2 (step ST72). Then, the recording unit 27 records the image file on the medium (step ST73), and the process ends.

  As described above, in the fourth embodiment, the non-irradiation region is extracted from the non-irradiation standard image G1 and the non-irradiation reference image G2, and the irradiation region is extracted from the irradiation standard image G1 ′ and the irradiation reference image G2 ′. The synthesized images C1 and C2 are generated by combining the regions. For this reason, in the synthesized images C1 and C2, it is possible that the portion with high contrast included in the range of the angle of view or the portion having local features is not irradiated with the pattern light. Therefore, even if the range of the angle of view has a high contrast or includes a portion having a local feature, it is possible to acquire high-quality composite images C1 and C2 that do not include moire or the like, so that the corresponding points can be accurately obtained. As a result, the three-dimensional shape of the subject can be accurately measured.

  In the fourth embodiment, since the images used for generating the image file are only the composite images C1 and C2, the capacity of the image file is reduced as compared with the first to third embodiments. be able to.

  Although the apparatus 1 according to the embodiment of the present invention has been described above, the computer corresponds to the imaging control unit 22, the range setting unit 23, the determination unit 24, the measurement unit 25, the file generation unit 26, and the synthesis unit 30 described above. A program that functions as means and performs processing as shown in FIGS. 6, 7, 8, and 10 is also one embodiment of the present invention. A computer-readable recording medium in which such a program is recorded is also one embodiment of the present invention.

1 is an external perspective view of a three-dimensional shape measuring apparatus according to a first embodiment of the present invention. Schematic block diagram showing the configuration of the arithmetic unit in the first embodiment The figure for demonstrating the setting of the irradiation range of light The figure for demonstrating the other example of the setting of the irradiation range of light The figure which shows the judgment result of the non-irradiation field and the irradiation field The flowchart which shows the process performed in 1st Embodiment. The flowchart which shows the process performed in 2nd Embodiment. The flowchart which shows the process performed in 3rd Embodiment The schematic block diagram which shows the structure of the calculating part in 4th Embodiment The flowchart which shows the process performed in 4th Embodiment Diagram for explaining generation of composite image The figure for demonstrating matching of a pixel

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 3D shape measuring device 2, 3 Camera 4 Projector 5 Calculation part 6 Monitor 7 Input part 11 Light source 12 Filter part 21 Irradiation control part 22 Shooting control part 23 Range setting part 24 Determination part 25 Measurement part 26 File generation part 27 Recording part 28 Overall Control Unit 30 Composition Unit

Claims (9)

A plurality of photographing means for obtaining a measurement image group composed of a plurality of measurement images for measuring a three-dimensional shape of the subject by photographing the subject;
Irradiating means for irradiating pattern light within the range of the angle of view by the photographing means;
Irradiation control means for controlling the irradiation means to irradiate the subject with pattern light;
Photographing control means for controlling the plurality of photographing means so as to acquire a first measurement image group including a plurality of first measurement images of the subject that are not irradiated with the pre-image of the subject and the pattern light; ,
A determination unit that sets a plurality of areas on the pre-image and determines whether each of the areas is irradiated with the pattern light or a non-irradiated area of the pattern light ;
The imaging control unit includes a second measurement image including a plurality of second measurement images of the subject irradiated with the pattern light when there is a region determined to be irradiated with the pattern light among the plurality of regions. A means for controlling the plurality of photographing means so as to acquire a measurement image group of
In the case where the second measurement image group is acquired, a non-irradiation region corresponding to a region where the pattern light is determined to be non-irradiation is extracted from the first measurement image group, and the second measurement is performed. An irradiation area corresponding to an area determined to be irradiated with the pattern light is extracted from the image group for image, and the non-irradiation area and the irradiation area extracted from the first and second measurement image groups corresponding to each other are extracted. Combining means for generating a composite image group composed of a plurality of composite images;
A three-dimensional shape measurement imaging apparatus comprising: a measuring unit that measures a three-dimensional shape of the subject based on the composite image group . A plurality of photographing means for obtaining a measurement image group composed of a plurality of measurement images for measuring a three-dimensional shape of the subject by photographing the subject;
Irradiating means for irradiating pattern light within the range of the angle of view by the photographing means;
Irradiation control means for controlling the irradiation means to irradiate the subject with pattern light;
A first measurement image group consisting of a plurality of first measurement images of the subject not irradiated with the pattern light, and a second measurement image consisting of a plurality of second measurement images of the subject irradiated with the pattern light. Photographing control means for controlling the plurality of photographing means so as to obtain two measurement image groups;
A determination unit that sets a plurality of regions on at least one of the first measurement images and determines whether each region is irradiated with the pattern light or whether the pattern light is a non-irradiated region. When,
A non-irradiation area corresponding to an area determined to be non-irradiated with the pattern light is extracted from the first image group for measurement, and an area determined to be irradiated with the pattern light from the second image group for measurement is extracted. Combining means for extracting a corresponding irradiation region and generating a composite image group composed of a plurality of composite images from the non-irradiation region and the irradiation region extracted from the first and second measurement image groups corresponding to each other. When,
A three-dimensional shape measurement imaging apparatus comprising: a measuring unit that measures a three-dimensional shape of the subject based on the composite image group . A plurality of photographing means for obtaining a measurement image group composed of a plurality of measurement images for measuring a three-dimensional shape of the subject by photographing the subject;
Irradiating means for irradiating pattern light within the range of the angle of view by the photographing means;
Irradiation control means for controlling the irradiation means to irradiate the subject with pattern light;
Photographing control means for controlling the plurality of photographing means so as to obtain a first measurement image group composed of a plurality of first measurement images of the subject not irradiated with the pattern light;
A determination unit that sets a plurality of regions on at least one of the first measurement images and determines whether each region is irradiated with the pattern light or whether the pattern light is a non-irradiated region. And
The imaging control unit includes a second measurement image including a plurality of second measurement images of the subject irradiated with the pattern light when there is a region determined to be irradiated with the pattern light among the plurality of regions. A means for controlling the plurality of photographing means so as to acquire a measurement image group of
In the case where the second measurement image group is acquired, a non-irradiation region corresponding to a region where the pattern light is determined to be non-irradiation is extracted from the first measurement image group, and the second measurement is performed. An irradiation area corresponding to an area determined to be irradiated with the pattern light is extracted from the image group for image, and the non-irradiation area and the irradiation area extracted from the first and second measurement image groups corresponding to each other are extracted. Combining means for generating a composite image group composed of a plurality of composite images;
A three-dimensional shape measurement imaging apparatus comprising: a measuring unit that measures a three-dimensional shape of the subject based on the composite image group . A plurality of photographing means for obtaining a measurement image group composed of a plurality of measurement images for measuring a three-dimensional shape of the subject by photographing the subject;
An imaging method for 3D shape measurement in an imaging device for 3D shape measurement comprising irradiation means for irradiating pattern light within a range of an angle of view by the imaging means,
Controlling the irradiating means to irradiate the subject with pattern light;
Controlling the plurality of photographing means so as to acquire a first measurement image group including a plurality of first measurement images of the subject that is not irradiated with the pattern pre-image and the pattern light;
A plurality of areas are set on the pre-image, and it is determined whether each area is irradiated with the pattern light or whether the pattern light is a non-irradiated area ,
When there is an area determined to irradiate the pattern light among the plurality of areas, a second measurement image group including a plurality of second measurement images of the subject irradiated with the pattern light Controlling the plurality of imaging means to obtain,
In the case where the second measurement image group is acquired, a non-irradiation region corresponding to a region where the pattern light is determined to be non-irradiation is extracted from the first measurement image group, and the second measurement is performed. An irradiation area corresponding to an area determined to be irradiated with the pattern light is extracted from the image group for image, and the non-irradiation area and the irradiation area extracted from the first and second measurement image groups corresponding to each other are extracted. , Generate a composite image group composed of a plurality of composite images,
A three-dimensional shape measurement photographing method , comprising: measuring a three-dimensional shape of the subject based on the composite image group . A plurality of photographing means for obtaining a measurement image group composed of a plurality of measurement images for measuring a three-dimensional shape of the subject by photographing the subject;
An imaging method for 3D shape measurement in an imaging device for 3D shape measurement comprising irradiation means for irradiating pattern light within a range of an angle of view by the imaging means,
Controlling the irradiating means to irradiate the subject with pattern light;
A first measurement image group consisting of a plurality of first measurement images of the subject not irradiated with the pattern light, and a second measurement image consisting of a plurality of second measurement images of the subject irradiated with the pattern light. Controlling the plurality of photographing means so as to obtain two measurement image groups,
A plurality of areas are set on at least one of the first measurement images , and it is determined whether each of the areas is irradiated with the pattern light or whether the pattern light is a non-irradiated area .
A non-irradiation area corresponding to an area determined to be non-irradiated with the pattern light is extracted from the first image group for measurement, and an area determined to be irradiated with the pattern light from the second image group for measurement is extracted. Extracting a corresponding irradiation region, generating a composite image group composed of a plurality of composite images from the non-irradiation region and the irradiation region extracted from the first and second measurement image groups corresponding to each other;
A three-dimensional shape measurement photographing method , comprising: measuring a three-dimensional shape of the subject based on the composite image group . A plurality of photographing means for obtaining a measurement image group composed of a plurality of measurement images for measuring a three-dimensional shape of the subject by photographing the subject;
An imaging method for 3D shape measurement in an imaging device for 3D shape measurement comprising irradiation means for irradiating pattern light within a range of an angle of view by the imaging means,
Controlling the irradiating means to irradiate the subject with pattern light;
Controlling the plurality of imaging means so as to obtain a first measurement image group composed of a plurality of first measurement images of the subject not irradiated with the pattern light;
A plurality of areas are set on at least one of the first measurement images, and it is determined whether each of the areas is irradiated with the pattern light or whether the pattern light is a non-irradiated area .
When there is an area determined to irradiate the pattern light among the plurality of areas, a second measurement image group including a plurality of second measurement images of the subject irradiated with the pattern light Controlling the plurality of imaging means to obtain,
In the case where the second measurement image group is acquired, a non-irradiation region corresponding to a region where the pattern light is determined to be non-irradiation is extracted from the first measurement image group, and the second measurement is performed. An irradiation area corresponding to an area determined to be irradiated with the pattern light is extracted from the image group for image, and the non-irradiation area and the irradiation area extracted from the first and second measurement image groups corresponding to each other are extracted. , Generate a composite image group composed of a plurality of composite images,
A three-dimensional shape measurement photographing method , comprising: measuring a three-dimensional shape of the subject based on the composite image group . A plurality of photographing means for obtaining a measurement image group composed of a plurality of measurement images for measuring a three-dimensional shape of the subject by photographing the subject;
A program for causing a computer to execute a three-dimensional shape measurement photographing method in a three-dimensional shape measurement photographing device including irradiation means for irradiating pattern light within a range of an angle of view by the photographing means,
A procedure for controlling the irradiation means to irradiate the subject with pattern light;
A procedure for controlling the plurality of photographing means so as to obtain a first measurement image group consisting of a plurality of first measurement images of the subject that is not irradiated with the pattern pre-image and the pattern light;
A procedure for setting a plurality of areas on the pre-image and determining whether each of the areas is irradiated with the pattern light or whether the pattern light is a non-irradiated area ;
When there is an area determined to irradiate the pattern light among the plurality of areas, a second measurement image group including a plurality of second measurement images of the subject irradiated with the pattern light A procedure for controlling the plurality of photographing means so as to obtain;
In the case where the second measurement image group is acquired, a non-irradiation region corresponding to a region where the pattern light is determined to be non-irradiation is extracted from the first measurement image group, and the second measurement is performed. An irradiation area corresponding to an area determined to be irradiated with the pattern light is extracted from the image group for image, and the non-irradiation area and the irradiation area extracted from the first and second measurement image groups corresponding to each other are extracted. A procedure for generating a composite image group composed of a plurality of composite images;
A program for causing a computer to execute a procedure for measuring a three-dimensional shape of the subject based on the composite image group . A plurality of photographing means for obtaining a measurement image group composed of a plurality of measurement images for measuring a three-dimensional shape of the subject by photographing the subject;
A program for causing a computer to execute a three-dimensional shape measurement photographing method in a three-dimensional shape measurement photographing device including irradiation means for irradiating pattern light within a range of an angle of view by the photographing means,
A procedure for controlling the irradiation means to irradiate the subject with pattern light;
A first measurement image group consisting of a plurality of first measurement images of the subject not irradiated with the pattern light, and a second measurement image consisting of a plurality of second measurement images of the subject irradiated with the pattern light. A procedure for controlling the plurality of photographing means so as to acquire two measurement image groups;
A procedure of setting a plurality of areas on at least one of the first measurement images and determining whether each area is to be irradiated with the pattern light or whether the pattern light is a non-irradiated area ; ,
A non-irradiation area corresponding to an area determined to be non-irradiated with the pattern light is extracted from the first image group for measurement, and an area determined to be irradiated with the pattern light from the second image group for measurement is extracted. A procedure for extracting a corresponding irradiation region and generating a composite image group including a plurality of composite images from the non-irradiation region and the irradiation region extracted from the first and second measurement image groups corresponding to each other; ,
A program for causing a computer to execute a procedure for measuring a three-dimensional shape of the subject based on the composite image group . A plurality of photographing means for obtaining a measurement image group composed of a plurality of measurement images for measuring a three-dimensional shape of the subject by photographing the subject;
A program for causing a computer to execute a three-dimensional shape measurement photographing method in a three-dimensional shape measurement photographing device including irradiation means for irradiating pattern light within a range of an angle of view by the photographing means,
A procedure for controlling the irradiation means to irradiate the subject with pattern light;
A procedure for controlling the plurality of imaging means so as to obtain a first measurement image group composed of a plurality of first measurement images of the subject not irradiated with the pattern light;
A procedure of setting a plurality of areas on at least one of the first measurement images and determining whether each area is to be irradiated with the pattern light or whether the pattern light is a non-irradiated area ; ,
When there is an area determined to irradiate the pattern light among the plurality of areas, a second measurement image group including a plurality of second measurement images of the subject irradiated with the pattern light A procedure for controlling the plurality of photographing means so as to obtain;
In the case where the second measurement image group is acquired, a non-irradiation region corresponding to a region where the pattern light is determined to be non-irradiation is extracted from the first measurement image group, and the second measurement is performed. An irradiation area corresponding to an area determined to be irradiated with the pattern light is extracted from the image group for image, and the non-irradiation area and the irradiation area extracted from the first and second measurement image groups corresponding to each other are extracted. A procedure for generating a composite image group composed of a plurality of composite images;
A program for causing a computer to execute a procedure for measuring a three-dimensional shape of the subject based on the composite image group .

Images