JP2018205030A - Distance measuring device, distance measurement method, and distance measuring program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distance measuring device, a distance measuring method and a distance measuring program capable of measuring a distance with high accuracy. A distance measuring device includes: a matching unit that matches blocks extracted from a target image acquired from reflected light from a target irradiated with a reference pattern and a reference image; In the block extracted from the object image, the weight of each pixel according to the luminance difference between the luminance and the luminance of a pixel other than the pixel is variable according to the luminance of the pixel at the predetermined position, and the pixel at the predetermined position And a variable unit that varies the weight of each pixel according to a spatial position difference between the pixel and a pixel other than the pixel, and a pixel shift amount between the object image and the reference image based on a matching result of the matching unit A detection unit for detecting, a distance calculation unit for calculating a distance between the light emission position of the reference pattern or the shooting position of the object image and the object from the pixel shift amount; Is provided. [Selection] Figure 5

Description

  This case relates to a distance measuring device, a distance measuring method, and a distance measuring program.

  A technique for measuring the distance to an object based on an image acquired by a camera is desired. Thus, for example, a technique for stereo matching using a plurality of cameras is disclosed (for example, see Patent Documents 1 and 2).

JP 2003-269917 A JP 2003-150940 A

  However, in the above technique, since a plurality of images of the object acquired by a plurality of cameras are compared, influences such as the shape of the object and the reflectance are reduced. On the other hand, in the matching that compares the image obtained from the reflected light from the object with a predetermined reference image, the reflected light from the object is greatly affected by the shape of the object and the reflectance. turn into. This makes it difficult to perform accurate distance measurement.

  In one aspect, an object of the present invention is to provide a distance measuring device, a distance measuring method, and a distance measuring program capable of performing distance measurement.

  In one aspect, the distance measuring device includes a matching unit that matches blocks extracted from the object image acquired from the reflected light from the object irradiated with the reference pattern and the reference image, and the matching process. In the block extracted from the object image, the weight of each pixel according to the luminance difference between the luminance and the luminance of the pixels other than the pixel is variable according to the luminance of the pixel at the predetermined position, and A variable unit that varies the weight of each pixel according to a spatial position difference between a pixel and a pixel other than the pixel, and a pixel shift amount between the object image and the reference image based on a matching result of the matching unit And a distance calculation unit that calculates a distance between the light emission position of the reference pattern or the shooting position of the object image and the object from the pixel shift amount. , Comprising a.

  Distance measurement can be performed.

(A) And (b) is a figure showing the outline of a distance measurement technique. It is a figure showing the outline of a distance measurement technique. (A) And (b) is a figure showing the outline of a distance measurement technique. It is a block diagram which illustrates the hardware constitutions of a distance measuring device. It is a functional block diagram of a distance measuring device. It is a figure which illustrates about the object of distance measurement, and the block of matching object. (A)-(d) is a figure illustrated about the weight according to a luminance difference. (A) And (b) is a figure which illustrates about the weight according to the spatial position difference. (A)-(e) is a figure illustrated about the weight width of Gaussian. (A) And (b) is a figure which illustrates about the weight according to the spatial position difference. It is a figure which illustrates a flowchart. (A)-(c) is a figure which illustrates the detection result of the pixel shift | offset | difference amount at the time of setting a weight to a pixel in the case of block matching. 6 is a functional block diagram of a distance measuring device according to Embodiment 2. FIG. It is a figure which illustrates ternary processing. It is a figure which illustrates ternary processing. It is a figure illustrated about a vertically long block and a horizontally long block. It is a figure which illustrates ternary processing. It is a figure which illustrates ternary processing.

  Prior to the description of the examples, distance measurement using triangulation will be described.

  As illustrated in FIG. 1A and FIG. 1B, for example, in a distance measurement technique, a light emitting device 201 provided in a measurement device irradiates an object with an infrared (IR) reference pattern and is provided in the measurement device. The IR camera 202 receives the light reflected from the object, thereby acquiring the object image.

  When the distance between the measuring device and the object changes, the pixel shift amount changes between the object image and the reference image according to the principle of triangulation. The reference image is an image obtained in advance by the IR camera 202 when a reference pattern is irradiated onto a plane parallel to the sensor surface of the IR camera 202 and having a distance from the light emitting device 201 set to a specified value. That is. The distance between the measuring device and the object can be measured by matching the blocks of the same size between the object image and the reference image and detecting the pixel shift amount. The pixel shift amount can be detected by searching for the same pattern in the object image and the reference image by block matching.

  In normal stereo matching, images from two cameras with parallax are compared with each other. On the other hand, in the method of receiving the reflected light of the reference pattern, the reference image represents a plane, but the object is not necessarily a plane. For example, the size and width of the object are not constant. Also, the reflectivity varies depending on the distance and material. For this reason, there is a region where the brightness of the reflected light is lowered. In order to perform block matching, it is preferable that the block has a certain size, but if the block includes an area with low brightness and a narrow width, and an area with high brightness around it, Matching accuracy deteriorates.

  For example, with respect to the back of the chair, the handle portion may have a narrow and blackish color with low reflected brightness. Therefore, when measuring the distance of the chair, the SN ratio is deteriorated at the handle portion having a narrow width, and the matching accuracy may be deteriorated. Next, degradation of matching accuracy will be described in detail.

  FIG. 2 is a diagram for explaining a pixel shift between the reference image and the object image. Originally, a unique random pattern is used to specify the position, but for the sake of simplicity of description, a lattice-point circular pattern is shown (the same applies to FIG. 3 and the like thereafter). In the example of FIG. 2, the reference pattern includes a circular pattern (dot pattern) in a lattice point shape. As illustrated in FIG. 2, it is assumed that the parallel surface 203 moves toward the measuring device while maintaining the parallelism. In this case, since the arrangement of the light receiving patterns 204 obtained by reflection from the parallel plane 203 is shifted in parallel, the pixel shift amounts of the circular patterns are substantially the same. Therefore, deterioration of matching accuracy is suppressed.

  On the other hand, as illustrated in FIG. 3A, it is assumed that a part 205 having a narrower width is arranged on the IR camera 202 side than the parallel plane 203. When the component 205 is disposed between the IR camera 202 and the parallel plane 203, the parallel plane 203 and the component 205 overlap in an image acquired by the IR camera 202. If the width of the component 205 is narrower than the block 206 that is a part of the image acquired by the IR camera 202, the parallel surface 203 and the component 205 overlap in the block 206. Since the distance between the IR camera 202 and the parallel plane 203 is different from the distance between the IR camera 202 and the component 205, the distance from the parallel plane 203 is obtained in the block 206 as illustrated in FIG. 3B. The amount of pixel shift differs between the received light pattern and the received light pattern obtained by reflection from the component 205. That is, the pixel shift amount of each circular pattern is different. In this case, it becomes difficult to specify the amount of pixel shift with respect to the reference image, so that the matching accuracy deteriorates. In particular, since high-luminance information is given priority in matching, when the luminance of reflected light from the component 205 is low, the matching accuracy is significantly deteriorated.

  Therefore, in the following embodiments, a distance measurement device, a distance measurement method, and a distance measurement program that can perform distance measurement with high accuracy by improving the detection accuracy of the pixel shift amount will be described.

  FIG. 4 is a block diagram illustrating a hardware configuration of the distance measuring apparatus 100. As illustrated in FIG. 4, the distance measuring device 100 includes a CPU 101, a RAM 102, a storage device 103, a display device 104, a light emitting device 105, a photographing device 106, and the like. Each of these devices is connected by a bus or the like.

  A CPU (Central Processing Unit) 101 is a central processing unit. The CPU 101 includes one or more cores. A RAM (Random Access Memory) 102 is a volatile memory that temporarily stores programs executed by the CPU 101, data processed by the CPU 101, and the like. The storage device 103 is a nonvolatile storage device. As the storage device 103, for example, a ROM (Read Only Memory), a solid state drive (SSD) such as a flash memory, a hard disk driven by a hard disk drive, or the like can be used. The display device 104 is a liquid crystal display, an electroluminescence panel, or the like, and displays a distance measurement result or the like. The light-emitting device 105 is a device that irradiates a reference pattern to an object for distance measurement, and is, for example, an infrared irradiation device. An example of the reference pattern is a random dot pattern. The imaging device 106 is a device that generates an image by receiving reflected light from an object, and is, for example, an IR camera.

  FIG. 5 is a functional block diagram of the distance measuring device 100. As illustrated in FIG. 5, when the CPU 101 executes the distance measurement program, the block extraction unit 10, the evaluation function calculation unit 20, the minimum value calculation unit 30, the pixel detection unit 40, and the distance calculation are included in the distance measurement device 100. The unit 50 and the like are realized. Each of these functions may be realized by hardware such as a dedicated circuit.

  FIG. 6 is a diagram illustrating the object 60 for distance measurement and the block 70 to be matched. A block 70 is a part of an image acquired by the imaging device 106. As illustrated in FIG. 6, the object 60 includes, for example, a parallel surface 61 that is perpendicular to the optical axis of the light emitting device 105 and parallel to the sensor surface of the imaging device 106, and closer to the imaging device 106 than the parallel surface 61. And an arranged part 62. The component 62 is disposed between the imaging device 106 and the parallel surface 61. In the example of FIG. 6, the axes orthogonal to each other forming the sensor surface of the imaging device 106 are defined as an X axis and a Y axis. The X axis is the direction of a line connecting the light emitting device 105 and the photographing device 106. The Z axis is an axis orthogonal to the X axis and the Y axis. In the example of FIG. 6, the Z axis represents the distance direction between the imaging device 106 and the object 60. The component 62 has a longitudinal direction in the X-axis direction and a narrow width in the Y-axis direction. Thereby, the width of the component 62 in the Y-axis direction is narrower than the width of the block 70 in the Y-axis direction. In this case, the parallel surface 61 and the component 62 overlap in the block 70.

  Subsequently, the operation of the distance measuring apparatus 100 will be described. The block extraction unit 10 acquires an object image from the imaging device 106. Next, the block extraction unit 10 extracts a block 70 including the target pixel position in the target object image. Next, the block extraction unit 10 acquires a reference image stored in a database or the like. In the present embodiment, the reference image refers to the image capturing apparatus 106 in advance when the reference pattern is irradiated onto a plane that is parallel to the sensor surface of the image capturing apparatus 106 and whose distance from the light emitting apparatus 105 is set to a specified value. It is the image that I got. Next, the block extraction unit 10 extracts a range corresponding to the block 70 as a reference block in the reference image.

  Next, the evaluation function calculation unit 20 calculates an evaluation function for the block 70 and the reference block. As the evaluation function, for example, a concept representing the similarity between the luminance value distributions of both objects of matching can be used. In this embodiment, as an example, an evaluation function that has a small value when the similarity of the luminance value distribution is high, such as a sum of squared differences, an absolute sum of differences, or a correlation coefficient normalization, is used.

  Next, the minimum value calculation unit 30 calculates the minimum value of the evaluation function calculated by the evaluation function calculation unit 20 in the repetition of the operations of the block extraction unit 10 and the evaluation function calculation unit 20. The pixel detection unit 40 detects a pixel shift amount obtained from the minimum value calculated by the minimum value calculation unit 30.

  The above operation is repeated in the entire search range including the target pixel position (a predetermined range including a block in the reference image). The pixel detection unit 40 detects a pixel shift amount from the minimum value calculated last by the minimum value calculation unit 30. Next, the distance calculation unit 50 converts the pixel shift amount detected by the pixel detection unit 40 into a distance. For example, triangulation can be used.

  The above operation is repeated for all pixels. By obtaining distances for all the pixels, it is possible to obtain distances between the respective parts of the object 60 and the imaging device 106 and to obtain the shape of the object 60.

  Next, details of the operation of the evaluation function calculation unit 20 will be described. FIG. 7A is a diagram illustrating a block 70 extracted by the block extraction unit 10. In the example of FIG. 7A, the component 62 is located in the center area C in the Y-axis direction, and the parallel plane 61 is located in the upper area U and the lower area L in the Y-axis direction. As illustrated in FIG. 7A, since the positions of the parallel surface 61 and the component 62 are different, in the block 70 where the parallel surface 61 and the component 62 overlap, between the parallel surface 61 and the component 62, The amount of pixel shift in the X-axis direction of dots is different. For pattern matching, it is preferable that the matching target block 70 has an area of a certain size. However, if the block size is large, the amount of pixel shift differs between the parallel surface 61 and the component 62 as in the block 70 of FIG. 7A, and therefore the pixel shift amount varies and cannot be specified. Matching accuracy deteriorates. In particular, when the luminance value of the reflected light from the component 62 is lower than the luminance value of the reflected light from the parallel surface 61, the matching accuracy is significantly deteriorated.

  Therefore, the evaluation function calculation unit 20 changes the weight of each pixel according to the luminance difference when calculating the evaluation function. For example, when the evaluation function is calculated, the evaluation function calculation unit 20 reduces the information weight of the pixel having a large luminance difference. First, as illustrated in FIG. 7B, it is assumed that the luminance of the central region C is lower than the threshold value because the component 62 has a blackish color. A shaded portion in FIG. 7B indicates that the luminance is low. A portion without shading indicates that the luminance is high. The same applies to FIG. 7C, FIG. 7D, FIG. 8 and FIG. In this case, the luminance at the central pixel position P of the block 70 is J (P), the luminance value at the pixel position qa in the central area C is J (qa), and the luminance value at the pixel position qb in the upper area U is J (P). qb). Since the luminance of the central area C where the central pixel position P and the pixel position qa are located is low, the luminance difference {J (qa) −J (P)} is small. On the other hand, the luminance difference {J (qb) −J (P)} increases because the luminance of the upper region U increases. The weight of a pixel having a large luminance difference is reduced according to the luminance difference. For example, the greater the luminance difference, the smaller the weight.

  In the example of FIG. 7C, the component 62 is located in the upper region U and the lower region L, and the parallel surface 61 is located in the center region C. Thereby, the brightness of the central area C is high, and the brightness of the upper area U and the lower area L is low. In this case, since it is originally desired to use only the information of the center region C having high luminance, it is desired to reduce the weight when the luminance difference is large. However, since the variation in luminance increases in the central region C where the luminance is high, it is preferable to loosen the weight when the luminance difference {J (qa) −J (P)} is negative. For example, when the brightness of the central region C is equal to or higher than the threshold value, the degree of weight reduction is reduced according to the brightness difference compared to the case of FIG.

  Next, in the region where the luminance value is originally high, the variation in the luminance value is large. For example, in the region of the parallel plane 61, although the variation in luminance value increases, the luminance value increases at any position. In this case, the weight of the pixel that increases the luminance difference is not reduced. FIG. 7D is a diagram illustrating a case where the part 62 does not overlap the block 70. In the example of FIG. 7D, since the variation of the luminance value is large in any area of the block 70, the luminance difference {J (qa) −J (P)} and the luminance difference {J (qb) −J (P) )} May be large. However, the luminance J (P) at the center pixel position P, the luminance value J (qa) at the pixel position qa, and the luminance value J (qb) at the pixel position qb all have high values. Therefore, in this case, the weight is not reduced for a pixel having a large luminance difference. For example, if the luminance is equal to or higher than the threshold value in any of the central pixel position P, the pixel position qa, and the pixel position qb, any weight of the pixel position qa and the pixel position qb is not reduced.

  In addition, the evaluation function calculation unit 20 changes the weight according to the spatial position difference (distance) when calculating the evaluation function. Let || P-qa || be the distance between the center pixel position P and the pixel position qa. Let || P-qa || be the distance between the center pixel position P and the pixel position qb. For example, the evaluation function calculation unit 20 is set to make the weight of the pixel position qb far from the center pixel position P smaller than the pixel position qa near the center pixel position P. In other words, the evaluation function calculation unit 20 is set so as to decrease the weight of a pixel having a larger spatial position difference from the central pixel position P.

  In this case, since the SN ratio is low when the luminance value J (P) at the central pixel position P is low, it is preferable to use information on a pixel having a larger distance (spatial position difference) than the central pixel position P. Therefore, the evaluation function calculation unit 20 may increase the weight of a pixel having a large spatial position difference. For example, as illustrated in FIG. 8A, when the overall luminance of the block 70 is low, the central pixel position P from the central pixel position P compared to the central pixel position P of the block 70 or the entire block 70 is bright. The weight of the far pixel position qb is also increased. For example, if the luminance is less than the threshold at any of the central pixel position P, the pixel position qa, and the pixel position qb, the weight of any of the pixel position qa and the pixel position qb is increased. Note that, as exemplified in FIG. 8B, when the area around the center region is bright, the weight can be reduced by the luminance difference, and therefore it is preferable to use it together with the weight by the luminance difference.

  Next, an example of calculating weights using a Gaussian distribution will be described. For example, when the thin object (component 62) is a low SN ratio region and the surrounding area is bright, the evaluation function calculation unit 20 weights on the basis of the luminance value in order to adaptively use the information of the low luminance region. A joint bilateral that can be made variable may be adopted. The evaluation function calculation unit 20 uses the spatial position difference Gaussian to reduce the weight of the pixel position farther from the target center pixel position P with a Gaussian distribution corresponding to the distance. Note that the Gaussian width is variable in accordance with the luminance. Next, the evaluation function calculation unit 20 uses the luminance difference Gaussian and sets the weight to a Gaussian distribution according to the luminance difference. The Gaussian width is variable according to the luminance, and the weight is reduced as the luminance is lower. The sign of the luminance difference is also variable.

For example, the weight can be calculated according to the following formula (1). The spatial position difference Gaussian can be expressed as in the following formula (2). The luminance difference Gaussian can be expressed as the following formula (3). Incidentally, W _P of the formula (1) is a normalization term can be expressed as the following equation (4). “I” is the target luminance and is the average value of the absolute differences. “J” is the luminance of the reference image that becomes the luminance weight, and is obtained by applying a low-pass filter of 5 × 5 pixels to the target image, for example. “P” is the center pixel position. “Q” is each pixel position of the element (s). In order to perform normalization according to the following equation (4), the normal Gaussian normalized count (0.5πσ) is omitted in the following equations (2) and (3).

  Subsequently, an example of the widths of the luminance difference Gaussian and the spatial position difference Gaussian will be described. FIG. 9A is a diagram exemplifying the weight width of the luminance difference Gaussian when the luminance difference is positive (in the case where the peripheral luminance is higher than the center). When the luminance value at the center pixel position P is less than the first threshold value (for example, 10) with respect to the MAX luminance value 255, it is very dark and has little variation. Therefore, the weight width corresponding to the luminance difference is reduced. When the luminance value at the center pixel position P is greater than or equal to the second threshold value (for example, 40) larger than the first threshold value with respect to the MAX luminance value 255, the brightness is relatively bright and large. Therefore, the weight width corresponding to the luminance difference is increased. The weight width may be changed stepwise between the first threshold value and the second threshold value. FIG. 9B is a diagram illustrating the weight width of the luminance difference Gaussian when the luminance difference is negative (when the luminance around the center is lower). Since the case where the luminance difference is positive and the case where the luminance difference is positive are more easily affected, it is preferable to reduce the weight width in the case where the luminance difference is positive.

  FIG. 9C illustrates the weight width of the spatial position difference Gaussian. As illustrated in FIG. 9C, when the luminance value at the center pixel position P is less than the first threshold value (for example, 20) with respect to the MAX luminance value 255, it is dark, so that it is illustrated in FIG. In addition, the block weight is set to 9 for example. When the luminance value at the central pixel position P is equal to or larger than the second threshold value (for example, 40) larger than the first threshold value with respect to the MAX luminance value 255, it is relatively bright, and as illustrated in FIG. For example, 3 is set in a range close to the central pixel position P. The weight width may be changed stepwise between the first threshold value and the second threshold value. The number of threshold levels may be increased.

  FIG. 9D is a diagram illustrating a case where the weight width is wide and a case where the weight width is narrow. In FIG. 9D, the horizontal axis represents the difference between the peripheral luminance and the central luminance, and the vertical axis represents the weight. In FIG. 9D, a solid line represents a case where the weight width is wide, and a broken line represents a case where the weight width is narrow. The weight of the vertical axis is “1” as the maximum value. FIG. 9E is a diagram illustrating a case where the weight is changed between a case where the luminance difference is positive and a case where the luminance difference is negative. Also in FIG. 9E, the horizontal axis represents the difference between the peripheral luminance and the central luminance, and the vertical axis represents the weight. The weight of the vertical axis is “1” as the maximum value.

  FIG. 11 is a diagram illustrating a flowchart representing the operation of each function of FIG. First, the block extraction unit 10 acquires an object image from the imaging device 106. Note that the object image may be obtained by removing noise with a low-pass filter. Next, the block extraction unit 10 extracts a block range including the target pixel position of the target image (step S1).

  Next, the block extraction unit 10 acquires a reference image stored in a database or the like. Next, the block extraction unit 10 extracts the same block range as that in step S1 at the target pixel position focused in step S1 in the reference image (step S2).

  Next, the evaluation function calculation unit 20 calculates an evaluation function for the block extracted in step S1 and the block extracted in step S2 (step S3). In step S3, as described above, an evaluation function is calculated after setting weights. Steps S2 to S3 are repeated in the entire search range (a predetermined range including a block in the reference image) including the target pixel position focused on in Step S1.

  Next, the minimum value calculation unit 30 calculates the minimum value of the evaluation function calculated by the evaluation function calculation unit 20 in the repetition of steps S2 to S3. The pixel detection unit 40 calculates a pixel shift amount obtained from the minimum value calculated by the minimum value calculation unit 30 (step S4).

  Next, the distance calculation unit 50 converts the pixel shift amount detected in step S4 into a distance (step S5). Steps S1 to S5 are repeated for all pixels. By obtaining distances for all the pixels, it is possible to obtain distances between the respective parts of the object 60 and the imaging device 106 and to obtain the shape of the object 60.

  FIG. 12A to FIG. 12C are diagrams illustrating examples of detection results of pixel shift amounts when weights are set for pixels during block matching. FIG. 12A shows the detection result when the weight width is widened as illustrated in FIG. 9D. That is, the variation of the weight with respect to the luminance difference is moderated. In this case, the component 62 is detected as surrounded by a dotted circle in FIG. In this way, by setting the weight to the pixel at the time of block matching, the narrow part 62 can be detected. However, since dots representing variations in the amount of pixel shift also appear, false detection tends to increase. In FIG. 12A, dots appear on the left side as surrounded by a dotted line on the left side.

  Next, FIG. 12B shows a detection result when the weight width exemplified in FIG. 9D is narrowed. That is, the variation in weight with respect to the luminance difference is steep. In this case, the part 62 is detected as surrounded by a dotted circle in FIG. Further, no dot representing the variation in the amount of pixel shift appears. However, since the component 62 is detected thinly, the detection accuracy of the component 62 is lower than that in FIG.

  Next, FIG. 12C shows detection results when the weight is changed between the case where the luminance difference is positive and the case where the luminance difference is negative, as illustrated in FIG. That is, when the luminance difference is negative, the weight is relaxed. In this case, no dot representing the variation of the pixel shift amount appears. The part 26 is still detected thickly as enclosed by the dotted circle. Therefore, it can be seen that the detection accuracy of the pixel shift amount is improved by changing the weight between the case where the luminance difference is positive and the case where the luminance difference is negative.

  According to the present embodiment, in the block matching, the weight according to the luminance difference between the luminance at the central pixel position P and the luminance at the peripheral pixels is made variable according to the luminance at the central pixel position P. In this case, it is possible to suppress the deterioration of matching accuracy due to the luminance difference. For example, it becomes possible to detect an area that is narrower than the block width and has a low luminance. Furthermore, the weight according to the spatial position difference between the center pixel position P and the peripheral pixels is made variable according to the luminance of the center pixel position P. Thus, for example, even when the overall luminance of the block is low, the SN ratio can be increased by increasing the weight of the pixel having a large spatial position difference. In this way, the matching accuracy can be further improved. From the above, distance measurement can be performed with high accuracy.

  In the above example, the center pixel position P of the block 70 extracted from the object image is set as the reference position. However, the present invention is not limited to this. For example, a pixel position other than the central pixel position may be used as a reference position for calculating a luminance difference and a spatial position difference. However, basically, since the block matching range is set with the center pixel of the block 70 as the center, it is preferable that the weight is set with reference to the center pixel. The reference position may be a position shifted by several pixels from the center pixel position. However, since the brightness difference is often slightly different from the original reference center position, it is basically desirable to use the center pixel position as the reference position. .

  The difference square sum, the difference absolute value sum, and the like given as the evaluation functions described above have large deviations depending on the luminance level. The amount of processing increases for correlation coefficient normalization. Therefore, in order to suppress the deviation according to the magnitude of the brightness, the brightness difference between adjacent pixels is calculated, and 1 when the difference is larger than the positive threshold, -1 when the difference is smaller than the negative threshold, Other values may be set to 0, and ternarization may be performed. Thereby, the influence of the absolute value of luminance can be suppressed. The adjacent difference can be a difference in the X-axis direction, a difference in the Y-axis direction, or an average of both. Note that noise may be reduced with a low-pass filter or the like before the difference is calculated.

  FIG. 13 is a functional block diagram of the distance measuring apparatus 100a according to the second embodiment. A difference from FIG. 5 is that a differentiation processing unit 80 is further provided. This function is realized by the CPU 101 executing a distance measurement program. Alternatively, each of these functions may be realized by hardware such as a dedicated circuit.

また、隣接差分のフィルタとして、下記式（６）のようなＹ軸方向の隣接差分を算出するための行列を適用することができる。

また、隣接差分のフィルタとして、下記式（７）または下記式（８）のような斜め方向の隣接差分を算出するための行列を適用することができる。

For example, as an adjacent difference filter, a matrix for calculating an adjacent difference in the X-axis direction such as the following formula (5) can be applied.

Further, as a filter for adjacent difference, a matrix for calculating the adjacent difference in the Y-axis direction as in the following formula (6) can be applied.

Further, as an adjacent difference filter, a matrix for calculating an adjacent difference in an oblique direction such as the following formula (7) or the following formula (8) can be applied.

  For example, as illustrated in FIG. 14, the object image is convoluted using the differential images of both the above formula (7) and the above formula (8). Thereby, the object image can be ternarized. For example, as illustrated in FIG. 15, the differential processing unit 80 converts the differential image 1 obtained by ternarizing the target image from which noise has been removed by the low-pass filter using the above equation (7), to the target The image is input to the evaluation function calculation unit 20 as an image. Furthermore, the differential processing unit 80 uses the differential image 2 obtained by ternarizing the object image from which noise has been removed by the low-pass filter by the above equation (8) as an object image to the evaluation function calculating unit 20. input.

  Further, the differentiation processing unit 80 shifts the ternary reference image obtained by ternizing the reference image from which noise has been removed by the low-pass filter by the above equation (7), for each pixel, and inputs it to the evaluation function calculation unit 20. Furthermore, the differentiation processing unit 80 shifts the ternary reference image obtained by ternizing the reference image from which noise has been removed by the low-pass filter by the above formula (8) for each pixel, and inputs it to the evaluation function calculation unit 20. The evaluation function calculation unit 20 calculates the sum of absolute differences of pixel shifts as an evaluation function, and sequentially replaces the pixel shift and the minimum evaluation function at the pixel position that is minimum than the minimum evaluation function before the shift, After all shifts (within the search range), a pixel shift that minimizes the evaluation function at all pixel positions is calculated. As an example of the evaluation function here, with respect to the differential ternarization in the oblique direction, the ternary reference image is shifted, the difference absolute value is taken, and the two kinds of averages in the oblique direction are taken. A convolution process is performed on the average according to the weight for each pixel. The subsequent processing is the same as in the first embodiment.

  According to the present embodiment, by shifting the object image and the reference image into three values, it is possible to suppress a deviation in the evaluation function due to the luminance. Further, the ternarization reduces the types of values, thereby reducing the amount of processing.

  In each of the above examples, one block extracted from the object image is matched with one block extracted from the reference image, but the present invention is not limited to this. For example, as illustrated in FIG. 16, the horizontal block 71 and the vertical block 72 may be overlapped with each other to perform matching processing, and a matching result using a block with a better result may be employed. In this case, as illustrated in FIG. 16, the pixel shift amount can be detected even if there is an area where the object 60 does not exist and the reflected light does not return in any one direction. Furthermore, by setting a weight to each pixel as described in the first embodiment for matching between the horizontal block and the vertical block extracted from the object image and the horizontal block and the vertical block extracted from the reference image, Matching accuracy is improved.

  For example, as illustrated in FIGS. 17 and 18, a ternary process may be performed. As illustrated in FIGS. 17 and 18, the difference from FIGS. 14 and 15 is the contents of the evaluation function. For example, as an example of the evaluation function, the ternary reference image is shifted with respect to the differential ternarization in the oblique direction, the difference absolute value is taken, and the two kinds of averages in the oblique direction are taken. For this average, convolution (summation) of the horizontally long block and vertically long block may be taken according to the weight for each pixel, and the smallest one may be used as the evaluation function. The subsequent processing is the same as in the first embodiment.

  According to the present embodiment, by shifting the object image and the reference image into three values, it is possible to suppress a deviation in the evaluation function due to the luminance. Further, the ternarization reduces the types of values, thereby reducing the amount of processing. In this embodiment, a block that is long in the horizontal direction (X-axis direction) and a block that is long in the vertical direction (Y-axis direction) are used, but the present invention is not limited to this. By using two blocks having different length directions on a plane parallel to the sensor surface of the imaging device 106, it is possible to improve the detection accuracy of the pixel shift amount.

  In each of the above examples, the block extraction unit 10, the evaluation function calculation unit 20, and the minimum value calculation unit 30 respectively extract the object image acquired from the reflected light from the object irradiated with the reference pattern and the reference image. It functions as an example of a matching unit that matches blocks that have been processed. In the block extracted from the object image by the evaluation function calculation unit 20 in the matching, each pixel according to the luminance difference between the luminance and the luminance of the pixels other than the pixel according to the luminance of the pixel at a predetermined position It functions as an example of a variable unit that makes the weight of each pixel variable according to the spatial position difference between the pixel at the predetermined position and the pixel other than the pixel. The pixel detection unit 40 functions as an example of a detection unit that detects a pixel shift amount between the object image and the reference image based on the matching result of the matching unit. The distance calculation unit 50 functions as an example of a distance calculation unit that calculates the distance between the light emission position of the reference pattern or the shooting position of the target object image and the target object from the pixel shift amount.

  Although the embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and changes can be made within the scope of the gist of the present invention described in the claims. It can be changed.

DESCRIPTION OF SYMBOLS 10 Block extraction part 20 Evaluation function calculation part 30 Minimum value calculation part 40 Pixel detection part 50 Distance calculation part 60 Target object 80 Differentiation processing part 100 Distance measuring device

Claims (11)

An object image acquired from reflected light from the object irradiated with the reference pattern, and a matching unit that matches blocks extracted from the reference image, and
In the matching, in the block extracted from the object image, according to the luminance of the pixel at a predetermined position, the weight of each pixel according to the luminance difference between the luminance and the luminance of the pixels other than the pixel is variable, A variable unit that varies the weight of each pixel according to a spatial position difference between a pixel at a predetermined position and a pixel other than the pixel;
A detection unit that detects a pixel shift amount between the object image and the reference image based on a matching result of the matching unit;
A distance measurement device comprising: a distance calculation unit that calculates a distance between the light emission position of the reference pattern or the shooting position of the target object image and the target object from the pixel shift amount.   When the luminance of the pixel at the predetermined position is less than a threshold and the luminance of the pixel other than the pixel is higher than the luminance of the pixel at the predetermined position, the variable unit is a pixel other than the pixel according to the luminance difference. The distance measuring device according to claim 1, wherein the weight of the distance is reduced.   The variable unit, when the luminance of the pixel at the predetermined position is equal to or higher than a threshold value, reduces the degree of decreasing the weight of the pixels other than the pixel at the predetermined position, or does not decrease the weight. Item 3. The distance measuring device according to Item 2.   4. The variable unit according to claim 1, wherein the variable unit reduces a weight of a pixel having a luminance lower than that of the pixel at the predetermined position in accordance with a luminance difference from the pixel at the predetermined position. The distance measuring device described in 1.   In the case where the weight of the pixels other than the pixel is reduced according to the luminance difference between the luminance of the pixel at the predetermined position and the luminance of the pixel other than the pixel, the variable unit is configured to remove the pixel other than the pixel at the predetermined position. 2. The distance measuring device according to claim 1, wherein when the luminance of a pixel other than the pixel is smaller than that of the pixel at a predetermined position, the degree of weight reduction is made smaller than when the luminance of the pixel is larger. .   6. The variable unit according to claim 1, wherein the variable unit reduces the weight of a pixel other than the pixel at the predetermined position, as the pixel has a larger spatial positional difference from the pixel at the predetermined position. The described distance measuring device.   The distance measuring device according to claim 6, wherein the variable unit increases the weight of each pixel according to the spatial position difference if the luminance of the pixel at the predetermined position is less than a threshold value.   The distance measuring device according to claim 1, wherein the pixel at the predetermined position is a central pixel of the block. A light emitting device for emitting the reference pattern;
The distance measuring device according to claim 1, further comprising: an imaging device that acquires the object image from reflected light from the object. The matching unit matches the blocks extracted from the object image acquired from the reflected light from the object irradiated with the reference pattern and the reference image,
In the matching, in the block extracted from the object image, according to the luminance of the pixel at a predetermined position, the weight of each pixel according to the luminance difference between the luminance and the luminance of the pixels other than the pixel is variable, The variable unit makes variable the weight of each pixel according to the spatial position difference between a pixel at a predetermined position and a pixel other than the pixel,
Based on the matching result of the matching unit, the detection unit detects a pixel shift amount between the object image and the reference image,
A distance measurement method, wherein a distance calculation unit calculates a distance between a light emission position of the reference pattern or a shooting position of the object image and the object from the pixel shift amount. On the computer,
Processing for extracting and matching blocks from the object image acquired from the reflected light from the object irradiated with the reference pattern and the reference image, and
In the matching, in the block extracted from the object image, according to the luminance of the pixel at a predetermined position, the weight of each pixel according to the luminance difference between the luminance and the luminance of the pixels other than the pixel is variable, A process for varying the weight of each pixel according to the spatial position difference between a pixel at a predetermined position and a pixel other than the pixel;
A process for detecting a pixel shift amount between the object image and the reference image based on the result of the matching;
A distance measurement program for executing a process of calculating a distance between the light emission position of the reference pattern or the photographing position of the object image and the object from the pixel shift amount.

Images