JP5336325B2 - Image processing method - Google Patents

Description

  The present invention relates to an image processing method for generating three-dimensional shape data from a two-dimensional image of an object by a stereo correlation method. The present invention can be used for article inspection, robot vision, personal authentication, surveillance camera, satellite image analysis, medical image analysis, microscope image analysis, and the like.

  When measuring the three-dimensional distance data of a three-dimensional object in a non-contact manner, for example, as disclosed in JP-A-2-29878 (Patent Document 1), an optical axis and a coordinate system ( It has been proposed to use two cameras whose imaging surfaces are parallel. This is to calculate the distance to the object by capturing the same object with the two cameras, associating the pixels constituting the same area of each captured image, and extracting the parallax from the result. It is what I did.

  On the other hand, a stereo correlation method is often used as a method for generating three-dimensional shape data of an object. In the stereo correlation method, an object is photographed by an imaging device having a plurality of cameras, and a plurality of two-dimensional images having different viewpoints are obtained. Corresponding points are searched for the plurality of two-dimensional images using a correlation method or a gradient method. The search results are combined and three-dimensional reconstruction is performed to generate three-dimensional shape data.

  More specifically, the stereo correlation method is a simple three-dimensional measurement method for measuring the shape of an object from a luminance image. The basic principle is to find the same point on the target object from two cameras, obtain the direction vector from each camera from the two-dimensional coordinates in the image, and calculate the three-dimensional coordinates by the principle of triangulation. The conversion formula for converting the position of the camera used for the calculation and the two-dimensional position in the image into a direction vector and the parameters of the formula are measured by performing calibration in advance.

  In the stereo correlation method, one point is selected in each pixel of the first camera image, and a window of a nearby minute region (hereinafter referred to as a nearby region) is provided around the selected point. In many cases, the neighboring region is a rectangle centered on the point of interest. Next, in the second camera, similarly, a window having the same shape is provided around a point that is a candidate for a corresponding point. The similarity of the luminance distribution is calculated between these windows. There are various methods for evaluating similarity. As an example, in a method called SAD (Sum of Absolute Difference), the sum of the absolute values of luminance differences between pixels having the same relative position is calculated over the entire window. In this case, the smaller the calculation result, the higher the similarity. When the corresponding points are determined, the direction vectors from the two cameras to the points on the respective images are determined, and the three-dimensional coordinate values of the points on the target object can be calculated.

  Here, the object is three-dimensional and its side image is rarely observed at the same point on the target object in different two-dimensional images captured by different cameras, and generally appears at a different position or one of the images. It only exists in In other words, there is a region that is visible from one camera and is not imaged because the imaged region is not visible from the other camera with a different line of sight. Hereinafter, this area is referred to as an occlusion area. As a result, the three-dimensional coordinate value calculated from the points that are incorrectly associated does not have a correct value.

  Therefore, it is difficult to search for corresponding points, and the generated three-dimensional shape data becomes inaccurate. Therefore, it is preferable to select the corresponding point from the plane area in front of the object facing the camera. Therefore, in order to correctly perform a stereo correlation method on an object, it is necessary to detect an occlusion area including a side area appearing in the object image toward the camera, and to determine a front plane area if possible. There is. And the method (patent document 2) etc. which detect the area | region of the front of the existing target object include the edge of a target object.

JP-A-2-29878 Japanese Patent Laid-Open No. 7-299782

  However, the conventional method described above cannot determine the occlusion area with higher accuracy. When the object is randomly arranged in an unspecified shape or is randomly stacked, the plane area in front of the camera cannot be specified only by edge detection. For example, the side surface of the lower object may be visible continuously from the plane of the upper object stacked in the field of view from the camera, or the plane of the left object may be continuous from the side of the right object. Sometimes it can be seen. For this reason, there is a portion where the corresponding point search is not accurately performed, and it is difficult to generate accurate three-dimensional shape data.

  An object of the present invention is to determine an occlusion region component observed in an image that becomes a problem when measuring a three-dimensional shape of an object using a stereo correlation method.

  In order to solve the above-described problems, the present invention provides a first imaging step of illuminating an object with an irradiation unit that emits light and obtaining first two-dimensional image data captured by at least one imaging unit, and a first imaging step A second imaging step of obtaining second 2D image data obtained by illuminating the same object from different illumination positions with an illuminating unit by the same imaging unit, and first 2D image data and second 2D image data 3D shape data from a plurality of two-dimensional image data obtained by imaging a target object from different viewpoints by a plurality of imaging units including the imaging unit A generation step of generating three-dimensional shape data from an image other than the fluctuation region in the two-dimensional image data by the imaging means that has determined the fluctuation region in the extraction step when generating. Those were.

  Further, in the image processing method according to claim 1 of the present invention, the extraction step generates difference data of luminance values based on the amount of reflected light from the first two-dimensional image data and the second two-dimensional image data, and an average of the difference data The image data indicating the luminance equal to or higher than the threshold value set with reference to the value is determined as the fluctuation region of the image data of the object.

  Further, in the image processing method according to claim 2 of the present invention, the threshold value is a value obtained by separating a standard deviation value from an average value of difference data.

In the image processing method according to claim 1 of the present invention, the first imaging step and the second imaging step are configured such that the angle of the illumination position by the irradiation means from the object is different.

In order to solve the above problems, the present invention provides a first extraction step for determining an occlusion region component of the first imaging means, a second extraction step for determining an occlusion region component of the second imaging means, and the first imaging means. When the three-dimensional shape data is generated from the two-dimensional image data obtained by imaging the object by the second imaging means, image processing is performed by excluding the occlusion region component determined in the first extraction step and the second extraction step. Generating the three-dimensional shape data , wherein the first and second extraction steps are different from the first imaging step and the first imaging step in which the first imaging means is different in angle from the object of the illumination position by the irradiation means. And an occlusion region is determined from the first and second two-dimensional image data obtained in the first imaging step and the second imaging step .

Further, in the image processing method according to claim 5 of the present invention, the first and second extraction steps generate difference data of luminance values based on the amount of reflected light from the first and second two-dimensional image data. The image data indicating the luminance equal to or higher than the threshold value set based on the average value is determined as the fluctuation region of the image data of the target object.

In the image processing method according to claim 6 of the present invention, the threshold value is a value obtained by separating a standard deviation value from an average value of difference data.

  According to the image processing method of the first aspect of the present invention, the fluctuation region in the image data is easily determined. As a result, accurate three-dimensional shape data can be generated.

  In addition, according to the image processing method of the present invention, it is possible to specify the fluctuation region in the two-dimensional image data with high accuracy.

According to the image processing method according to claim 3 of the present invention, an effect that may be specific with high accuracy variation region by the two-dimensional image data obtained by the first imaging step and a second imaging step Play.

  In addition, according to the image processing method of the present invention, it is possible to specify the fluctuation region of the two-dimensional image data with higher accuracy in the first imaging step and the second imaging step.

According to the image processing method of the fifth aspect of the present invention, accurate three-dimensional shape data can be generated by performing image processing while excluding the occlusion region component of the two-dimensional image data. In addition, there is an effect that the occlusion region component of the two-dimensional image data can be specified with higher accuracy in the first imaging step and the second imaging step.

Further, according to the image processing method of the sixth aspect of the present invention, there is an effect that the occlusion region component in the two-dimensional image data can be specified with high accuracy.

Further, according to the image processing method of the seventh aspect of the present invention, the occlusion region component based on the two-dimensional image data obtained in the first extraction step and the second extraction step can be specified with higher accuracy.

The block diagram which shows the functional structure of the three-dimensional shape production | generation apparatus 1 of the 1st Embodiment of this invention. The figure which shows the example of arrangement | positioning of an imaging device and an illuminating device The figure which shows the process of the three-dimensional shape production | generation apparatus 1 Explanatory drawing explaining other embodiment of the determination means of a fluctuation area The figure which shows the state which has arrange | positioned the three-dimensional shape production | generation apparatus 1 of 1st Embodiment which is the 2nd Embodiment of this invention in the arm part of an industrial robot.

  Embodiments of the present invention will be described below with reference to FIGS.

  FIG. 1 is a block diagram illustrating a functional configuration of a three-dimensional shape generation apparatus 1 according to the first embodiment of the present invention, and FIG. 2 is a diagram illustrating an example of an arrangement of an imaging device and an illumination device.

  In the present invention, for example, a three-dimensional object Q placed on the floor surface is simultaneously imaged by two cameras, a pair of stereo vision pairs is prepared, and a distance is calculated using the stereo vision pair. First, a fluctuation region where an occlusion region exists is extracted from each stereo vision image, and then three-dimensional shape image data is generated in a stereo vision pair.

  In these drawings, the three-dimensional shape generation device 1 includes an imaging device M1, illumination devices F1 and F2, and a processing device 10. The imaging device M1 is integrally provided with two cameras C1 and C2. These cameras C1 and C2 are known digital cameras having a CCD, a CMOS, or the like that captures (captures) a two-dimensional image of the object Q that is imaged via an optical system. The two cameras C1 and C2 are arranged so that their viewpoints are different from each other with respect to the object Q placed on the floor, and acquire two-dimensional image data FD having parallax with each other.

The arrangement of the two cameras C1 and C2 will be described in detail with reference to FIG. The cameras C1 and C2 are arranged at a predetermined distance (between eyes) and are inclined toward the target portion Q. Therefore, the cameras C1 and C2 are inclined in different directions. According to this arrangement aspect Q1 and the plane Q2 of the object Q as viewed projected in heart or al of the camera C1 becomes the imaging area, sides Q3 becomes occlusion area which are transparent to the camera C1. On the other hand, as viewed projected in heart or al of the camera C2 side Q3 and the plane Q2 of the object Q becomes the imaging area, sides Q1 becomes occlusion area which are transparent to the camera C2. That is, the plane Q2 enters both the imaging areas of the cameras C1 and C2. Here, when the distance between the cameras C1 and C2 is widened, the inclination angles of the cameras C1 and C2 approach the parallel to the floor surface, so the captured image of the plane Q2 between the camera C1 and the camera C2 becomes small. Since the orchestration area becomes wide, the setting of corresponding points is limited. Therefore, it while viewing the planar Q 2 is imaged to a suitable size is set.

  Based on the two two-dimensional image data FD1 and FD2 having parallax acquired by the cameras C1 and C2, the processing device 10 generates three-dimensional shape image data DT by the stereo correlation method. Three or more cameras C are used, three-dimensional shape image data DT is generated based on two two-dimensional image data FD obtained from them, and the other one-dimensional image data FD is used as the reliability of each point. It can also be used for data acquisition. In this way, it is easy to evaluate missing portions of data.

The illuminating devices F1 and F2 irradiate the object Q with illumination light and correspond to the irradiating means of the present invention. In this embodiment, since the object Q is imaged indoors, light is emitted during imaging by the imaging device M1 in order to obtain an imaging environment in which appropriate exposure in imaging is obtained. Incidentally, the amount of light by the lighting device F1, F2 intended to vary imaging environment, when the imaging of the object Q is carried out in outdoors, though depending on the weather obtain proper exposure in an environment light intensity due to natural light is large irradiation The light quantity by the bright devices F1 and F2 is at least good.

  Next, each illuminating device F1, F2 irradiates illumination light with respect to the target object Q at mutually different angles. The illuminating devices F1 and F2 are arranged on the left and right sides with the imaging device M1 interposed therebetween, and are inclined in opposite directions so as to face the object Q arranged to face the imaging device M1. These illuminating devices F1 and F2 can perform an operation in which illumination is performed by the illuminating device F1 for the first time and by the illuminating device F2 for the second time, and an operation for simultaneously lighting both. The operations of the imaging device M1 and the illumination devices F1 and F2 are controlled by the processing device 10.

  In the two-dimensional image data FD obtained by the imaging device M1, when there is an occlusion region, the position of the occlusion region to which the illumination light is irradiated differs depending on which illumination device F1, F2 is on. As shown in FIG. 2, when illuminated by the illumination device F1, the side surface Q1 and the plane Q2 of the object Q are illuminated, but the side surface Q3 is not illuminated and becomes a shadow. On the other hand, when illuminated by the illumination device F2, the side surface Q3 and the plane Q2 of the object Q are illuminated, but the side surface Q1 is not illuminated and becomes a shadow. Therefore, the illumination state of the occlusion area varies depending on the lighting state of the illumination devices F1 and F2, but the variation of the illumination state is small in the common imaging area.

  The processing device 10 includes a control unit 11, a storage unit 12, a fluctuation region determination unit 13, a three-dimensional shape image generation unit 15, and the like. The control unit 11 is provided with an imaging control unit 21 and an illumination control unit 22. The imaging control unit 21 controls the imaging device M1. For example, the shutter of the imaging device M1 is controlled to control the exposure timing, the exposure time, and the like. The illumination control unit 22 controls each illumination device F1, F2. For example, when photographing by the imaging device M1, the lighting devices F1 and F2 are switched so as to be sequentially turned on.

  In addition, the control unit 11 controls the entire processing apparatus 10. The storage unit 12 stores a plurality of sets of two-dimensional image data FD output from the imaging apparatus M1, other data, and programs. Further, the stored two-dimensional image data FD is output to the fluctuation region determination unit 13 in accordance with a command from the control unit 11.

  The fluctuation region determination unit 13 detects an occlusion component region from the two-dimensional image data FD1 and FD2. There are the following methods. A pixel or region is extracted as an occlusion component region by comparison with a predetermined threshold value based on the luminance value for each pixel in the image data. Specifically, the occlusion area included in the object Q imaged by one camera C1 of the imaging device M1 is different from the occlusion area imaged by the camera C2. In addition, in the imaging areas of the respective cameras C1 and C2, the light irradiation areas are different depending on the illumination devices F1 and F2. In this case, the plane Q2 area in front of the object Q facing the imaging device M1 is illuminated almost evenly, but the occlusion areas which are the side surfaces Q1 and Q3 of the substantially vertical plane are imaged by the camera C1. The side surface Q1 is illuminated by the illumination device F1, but is not illuminated by the illumination device F2. On the other hand, the side surface Q3 imaged by the camera C2 is in a positional relationship that is illuminated by the illumination device F2, but not illuminated by the illumination device F1. In other words, the brightness of the image using only the illumination device F1 and the image using only the illumination device F2 vary greatly on the side surfaces Q1 and Q3 when taking an image with one camera C1. The same applies to the camera C2. Therefore, by extracting a region having a large fluctuation, it is determined as an orcinal region component of the object Q.

  More specifically, two images with different illumination devices F1 and F2 used in one camera C1 are taken. In a normal camera, the amount of light incident on the image sensor is integrated, and the value is converted into an integer value by A / D conversion and held in the form of an image format. There is a maximum luminance value that can be expressed by pixels. For example, in the case of an image format that expresses luminance in 8 bits, the maximum value is 255 and the minimum value is 0. Therefore, when using such a camera, by obtaining a difference image of luminance values between the two images, it can be determined that the brightness of the two images differs as the difference value increases, that is, an occlusion region. Although details will be described later, in the present embodiment, in the difference image, the presence / absence of an occlusion region component is set using a value obtained by subtracting the standard deviation of all luminance values from the average value of all luminance values as a threshold value.

  Such a fluctuation region determination unit 13 can automatically detect occlusion region components in the two-dimensional image data FD1 and FD2. Note that the position of the occlusion region component in the two-dimensional image data FD1 is different from the position of the occlusion region component in the two-dimensional image data FD2. This is because the positions of the illumination devices F1, F2 and the cameras C1, C2 are different.

  The three-dimensional shape image generation unit 15 generates three-dimensional shape image data DT based on the two-dimensional image data FD1 and FD2. Also, a plurality of three-dimensional shape image data DT is integrated into one three-dimensional shape image data DT. Such a function of the three-dimensional shape image generation unit 13 is conventionally known. Corresponding points of the two-dimensional image data FD1 and FD2, which are the basis for generating the three-dimensional shape image data DT, are searched from images other than the occlusion region components based on the information detected by the fluctuation region determination unit 13.

  Such a processing device 10 is realized by the CPU executing a program using a CPU, RAM, ROM, magnetic disk device, magneto-optical disk device, medium drive device, input device, display device, and an appropriate interface. can do. It is also possible to configure using a personal computer or the like. Such a program can be supplied by an appropriate recording medium, and can also be downloaded from another server via a network.

  Next, an operation flow of the three-dimensional shape generation apparatus 1 will be described with reference to a time chart. FIG. 3 is a time chart showing processing steps of the three-dimensional shape generation apparatus 1. The horizontal axis shows time (t), and the vertical axis shows ON and OFF of the corresponding configuration. When the three-dimensional shape generation apparatus 1 wants to create the three-dimensional shape image data DT of the object Q, the operator turns on an operation switch (not shown) of the three-dimensional shape generation apparatus 1. Following the turn-on, the three-dimensional shape generation apparatus 1 automatically performs the following operation by the control unit 11.

  In FIG. 3, both of the illumination devices F1 and F2 are turned off, and the first imaging by the imaging device M1 is performed. At this time, since there is no illumination light quantity by the illuminating devices F1 and F2, the object Q is imaged with only extraneous light indoors. Here, the imaging time by the imaging device M1 may be the same as the subsequent processing steps, and does not need to be changed.

  Two pieces of two-dimensional image data FD11 and FD21 obtained via the cameras C1 and C2 are stored in a predetermined memory area of the storage unit 12, and extraneous light is used in the two-dimensional image data used in the variable area extraction process described later. This is used when a process for eliminating the influence of is performed (step S1). That is, since there is no illumination light amount by the illumination devices F1 and F2, the luminance values of the two-dimensional image data FD11 and FD21 are only due to the external light. Therefore, when there is an illumination light amount by the illumination devices F1 and F2, the luminance of the external light By subtracting the value, if the extraneous light is uneven, the influence of the unevenness can be reduced. However, this pre-imaging process may be omitted if the external light is in a uniform environment.

  Next, the illumination device F1 is turned on and the illumination device F2 is turned off, and the second imaging by the imaging device M1 is performed. At this time, the normal exposure conditions for photographing the object Q indoors with the imaging device M1 are set for the initial exposure time T1 and the initial illumination light quantity by the illumination device F1. That is, generally, a camera has a known automatic exposure function so as to obtain an appropriate exposure for photographing an object. In the present invention, the initial exposure time T1 and the initial irradiation light quantity are set by setting appropriate exposure to the photographing apparatus M1 by automatic exposure. Two two-dimensional image data FD12 and FD22 obtained via the cameras C1 and C2 are stored in a predetermined memory area of the storage unit 12. This corresponds to the first imaging step of the present invention (step S2).

Next, the illumination device F1 is turned off and the illumination device F2 is turned on to perform the third imaging by the imaging device M1. At this time, as for the initial exposure time T1 and the initial illumination light amount by the illuminating device F1, normal imaging conditions when the object Q is imaged indoors by the imaging device M1 are set as in step S2. Then, the two two-dimensional image data FD13 and FD23 obtained through the cameras C1 and C2 are stored in a predetermined memory area of the storage unit 12. This corresponds to the second imaging step of the present invention (step S3).

  Using the four two-dimensional image data FD12, FD13, FD22, and FD23 obtained through the cameras C1 and C2, each image is analyzed by the fluctuation region determination unit 13.

  Next, determination of the fluctuation region will be described. The determination process is performed using two pieces of image data captured by the same camera from four pieces of two-dimensional image data and having different illumination devices. Here, a determination is first made using the two-dimensional image data FD12 and FD13 captured by the camera C1.

  Difference data of luminance values represented by 256 gradations in all pixels of the two-dimensional image data FD12 and FD13 is obtained. By calculating the difference in gradation value for each pixel, the difference value from “0 (zero)” becomes larger in the region where the state of the region irradiated with irradiation light is greatly different between the two-dimensional image data FD12 and FD13. Leave. That is, the luminance value of the side surface Q1 of the object Q in the imaging region of the camera C1 is close to “0” in the two-dimensional image data FD12 irradiated by the illumination device F1. On the other hand, the brightness value of the same side surface Q1 is close to “255” in the two-dimensional image data FD13 irradiated by the illumination device F2. Therefore, the difference value indicates a negative value on the negative side of “0”. In the opposite case, the difference value is a positive value on the positive side of “0”. That is, the area where the irradiation light is not different between the two-dimensional image data FD12 and FD13, that is, the difference value approaches “0 (zero)” because the luminance value of the pixel is equal in the plane Q2.

  Here, the occlusion region component is determined from the difference data with a predetermined threshold. In the difference data, the closer the value is to “0”, it can be said that in the two-dimensional image data FD12 and FD13, the illumination state of the object Q by the illumination devices F1 and F2 is not changed. In other words, it corresponds to the plane Q2 region that is the front surface facing the camera C1. Therefore, conversely, the larger the negative or positive value, the more likely it is an occlusion region including a substantially vertical surface such as a side surface in which the amount of irradiation light varies as the irradiation state changes. Therefore, the occlusion region component can be satisfactorily determined by setting the predetermined threshold value as a boundary value corresponding to the fluctuation of the irradiation light amount indicated by the occlusion region.

More specifically, as the predetermined threshold, first, an average value and a standard deviation of all image data of difference data are obtained. Then, by subtracting or adding the standard deviation from the average value, a value obtained by separating the standard deviation value from the average value is set as a threshold value. By obtaining the standard deviation of all the difference data, variation in the amount of irradiation light in the two-dimensional image data FD12 and FD13 is obtained. Therefore, indicating the variation in the amount of irradiation light exceeding that value is the two-dimensional image data FD12. It can be said that the light irradiation fluctuates in the FD 13. The occlusion area of the object Q is different in the state of light irradiation by the illuminating devices F1 and F2, and has a large luminance value. Therefore, when the value of the standard deviation values in the difference data from the mean value of the differential data is ultra strong point distant value, it is determined that the occlusion region component. The process S3 for the camera C1 corresponds to the first extraction process of the present invention.

  Next, the determination of the occlusion region component is repeated using the two-dimensional image data FD22 and FD23. The process S3 for the camera C2 corresponds to the second extraction process of the present invention. Then, the result analyzed by the fluctuation region determination unit 13 for each image is given to the control unit 11. In the control unit 11, a flag indicating an occlusion region component is individually stored in the two-dimensional image region by the cameras C1 and C2, and the three-dimensional shape image is used when searching for corresponding points in both image data. What is necessary is just to make it the group which produces | generates data DT. In addition, these 1st extraction processes and 2nd extraction processes are also corresponded to the extraction process of 1st invention of this invention, respectively (process S3).

  Next, the illumination device F1 is turned on and the illumination device F2 is turned on to perform the fourth imaging by the imaging device M1. At this time, as for the initial exposure time T1 and the initial illumination light amount by the illuminating devices F1 and F2, normal imaging conditions when the object Q is imaged indoors by the imaging device M1 are set as in step S2. Then, the two two-dimensional image data FD14 and FD24 obtained through the cameras C1 and C2 are stored in a predetermined memory area of the storage unit 12 (step S4).

  The three-dimensional shape image generation unit 15 generates three-dimensional shape image data DT by image processing based on the two-dimensional image data FD14 and FD24. Also, a plurality of three-dimensional shape image data DT is integrated into one three-dimensional shape image data DT. Such a function of the three-dimensional shape image generation unit 13 is conventionally known. Corresponding points of the two-dimensional image data FD14 and FD24, which are the basis for generating the three-dimensional shape image data DT, are searched from images other than the occlusion region components based on the information detected by the fluctuation region determination unit 13. This corresponds to the generation process of the present invention.

  As described above, according to the first embodiment, the occlusion region component indicating the occlusion region is determined in the two-dimensional image data, thereby generating the three-dimensional shape image data from the image data other than the region component. By setting and searching for points, accurate three-dimensional shape data can be generated.

  Next, a second embodiment of the present invention will be described with reference to FIG. In the above-described embodiment, the occlusion region component region is extracted from the difference data of the two two-dimensional image data FD as a pixel or region whose luminance value is equal to or greater than a predetermined threshold value as an occlusion region component that is a variable region. However, there are the following other methods. FIG. 4 is a diagram showing a method for determining a fluctuation region according to the second embodiment of the present invention.

  A value from 0 to 255, which is a pixel value of the difference data, is correlated with a value indicating the possibility of being an occlusion region component. Specifically, if the difference data is 201 to 255, the probability is 80%, 151 to 200 is 70%, 101 to 150 is 50%, the remaining 0 to 100 is 0% And the table 30 to be set is generated. With this relationship table 30, as shown in FIG. 4, a data matrix table 32 in which the difference data matrix table 31 is replaced with a numerical value indicating the possibility for each pixel is generated in the variable region determination unit 13. In this way, when it is desired to set a flag so as to omit an area regarded as an occlusion area in the subsequent processing, the possibility threshold is set to 50%. Then, all the pixels showing a value of 50% or more may be determined as occlusion area components.

  Further, when it is desired to set a flag so that an area with a higher possibility of an occlusion area is omitted in the subsequent processing, the possibility threshold is set to 80%. Then, all the pixels showing a value of 80% or more may be determined as occlusion region components. The threshold value of this possibility may be set arbitrarily, and the threshold value may be set to 80% when the shape of the object Q is low. Further, the threshold value of the possibility of the table 30 may be set more finely.

  Next, a third embodiment according to the present invention will be described with reference to FIG. FIG. 5 is a diagram illustrating a state in which the three-dimensional shape generation apparatus 1 according to the first embodiment is arranged on an arm portion of an industrial robot. The industrial robot 200 includes a rotatable robot body 201 and an arm 202 attached to the robot body 201.

  The arm 202 is configured by a retractable multi-indirect arm, and is configured by a hand 203 that performs a clamping operation capable of sandwiching the object Q attached to the tip thereof, and the hand 203 includes the three-dimensional shape generation device 1. The illumination devices F1, F2 and the imaging device M1 are mounted as a unit.

  According to this industrial robot 200, the two-dimensional image is obtained after the three-dimensional shape generation device 1 determines the same fluctuation region as in the first embodiment before starting the operation of grasping the object Q with the hand 203. Generate data. Based on the two-dimensional image data, three-dimensional shape image data is formed, and the three-dimensional coordinate values of the points on the image of the object Q can be calculated. As a result, the operating distance of the arm 202 to the object Q is calculated, so that the hand 203 can accurately grasp the object Q.

  When an industrial product is to be handled by an automatic machine such as a robot, it is necessary to automatically measure and recognize the industrial product. In the conventional three-dimensional measurement and recognition technology of an object using a two-dimensional or three-dimensional image, if there is an occlusion area in the image, the result of imaging by the camera is significantly different, and it is difficult to search for corresponding points. For this reason, production line automation has been limited.

  If it becomes possible to measure the position and orientation of a three-dimensional target object according to the present invention, it becomes possible to automate the selection of a target in a production line. In addition to reducing production costs, this has the effect of freeing workers from unfavorable conditions by enabling mechanization of work even in areas where human work was necessary even if working conditions were poor because it was difficult to automate with conventional methods. .

  In the embodiment described above, various known devices such as a xenon tube, a flash device using other light emitting elements, an illumination device using a lamp, and an illumination device equipped with a mask shutter are used as the illumination device F. Can do. Although one photographing apparatus M1 is used, a plurality of sets of photographing apparatuses M1 may be used.

  In the embodiment described above, a digital camera is used as the photographing apparatus M1, but it is also possible to photograph using a silver halide camera and digitize the obtained two-dimensional image data using a scanner. In addition, the configuration, number, processing contents, processing order, etc. of the whole or each part of the three-dimensional shape generation apparatus 1 can be appropriately changed in accordance with the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 3D shape production | generation apparatus M1 Imaging device F1, F2 Illumination device 10 Processing apparatus C1, C2 Camera Q Target object 11 Control part 12 Storage part 13 Fluctuation area determination part 15 3D shape image generation part 21 Imaging control part 22 Illumination control part FD, FD1, FD11, FD12, FD13, FD14, FD2, FD21, FD22, FD23, FD24 2D image data DT 3D shape data

Claims (7)

A first imaging step of illuminating an object with irradiation means for irradiating light and obtaining first two-dimensional image data imaged by at least one imaging means;
A second imaging step of obtaining the second two-dimensional image data obtained by illuminating the same object with the irradiation unit from different illumination positions by the same imaging unit as the first imaging step;
An extraction step of determining a fluctuation region of a luminance value due to the amount of reflected light from the first two-dimensional image data and the second two-dimensional image data;
When the three-dimensional shape data is generated from a plurality of two-dimensional image data obtained by imaging a target object from different viewpoints by a plurality of imaging means including the imaging means, by the imaging means that has determined the fluctuation region in the extraction step A generating step of generating three-dimensional shape data from an image other than the fluctuation region in the two-dimensional image data;
An image processing method comprising the steps of:   2. The image processing method according to claim 1, wherein the extraction step generates difference data of luminance values based on a reflected light amount from the first two-dimensional image data and the second two-dimensional image data, and uses an average value of the difference data as a reference. An image processing method characterized in that image data showing luminance equal to or higher than a set threshold value is determined as a fluctuation region of image data of an object.   3. The image processing method according to claim 2, wherein the threshold value is a value obtained by separating a standard deviation value from an average value of difference data. The image processing method according to claim 1, wherein the first imaging step and the second imaging step are different from each other in an angle of an illumination position by the irradiation unit from an object. A first extraction step of determining an occlusion region component of the first imaging means;
A second extraction step of determining an occlusion region component of the second imaging means;
When the three-dimensional shape data is generated from the two-dimensional image data obtained by imaging the object by the first imaging unit and the second imaging unit, the orchestration region component determined in the first extraction step and the second extraction step is used. A generation step of generating three-dimensional shape data to be image-processed by exclusion;
Equipped with,
The first and second extraction steps include a first imaging step and a second imaging step in which the first imaging unit has different angles from the object at the illumination position by the irradiation unit,
An occlusion region is determined from first and second two-dimensional image data obtained in the first imaging step and the second imaging step . 6. The image processing method according to claim 5 , wherein the first and second extraction steps generate luminance value difference data based on the amount of reflected light from the first and second two-dimensional image data, and the average value of the difference data is used as a reference. An image processing method characterized in that image data showing luminance equal to or higher than a threshold value set in the above is determined as a fluctuation region of image data of an object. 7. The image processing method according to claim 6 , wherein the threshold value is a value obtained by separating a standard deviation value from an average value of difference data.

Images