JP7154527B2 - Displacement measuring device - Google Patents

Description

TECHNICAL FIELD The present invention relates to a displacement measuring device and a displacement measuring method, and for example, to a technique for measuring displacement using a moiré method.

Patent Literature 1 discloses a displacement acquisition device capable of reducing errors due to the inclination of the target surface or the measurement direction when measuring the displacement of the measurement points on the target surface using the sampling moire method. The displacement acquisition device measures the relative positions of the camera and the target surface in advance in a z-direction parallel to the optical axis of the camera and in an x-direction perpendicular to the z-direction. By doing so, a correction coefficient for correcting the error is obtained.

Patent Document 2 discloses a method for measuring out-of-plane displacement (displacement in the z (depth) direction) of an object based on an image obtained from a single camera. In this method, when a camera captures an image of a periodic pattern attached to an object, the pitch of the periodic pattern on the image changes according to the distance between the object and the camera. Out-of-plane displacement is calculated by combining principles. At this time, the pitch of the periodic pattern on the image is calculated based on the phase of moire fringes obtained by the sampling moire method.

Non-Patent Document 1 describes the relationship between the pitch of the reference grid, the pitch and gradient of the sample grid, and the pitch of the moire fringes in the oblique moire created by intersecting the reference grid and the sample grid at a predetermined gradient. A theoretical formula (formula (31)) representing is shown. Further, in the oblique moiré, a theoretical formula (equation (32)) representing the relationship between the pitch of the reference grating, the pitch and gradient of the sample grating, and the gradient of moire fringes is shown.

JP 2019-11984 A WO2017/029905

Shinohara, "Geometry of Moire fringes", Textile Machinery Society, Vol. 37, No. 6, 1984, p. 253-262

As a method for measuring the displacement of an object, a method using moire fringes generated by overlapping periodic patterns with different periods is known. Specifically, this method attaches a marker with a periodic pattern to an object, takes an image of the marker with a camera, and performs computer analysis of the marker image based on the sampling moire method or the like. . A displacement measurement method using such moire fringes is usually used to measure the in-plane displacement of an object. On the other hand, when measuring the displacement of an object in the depth direction, rangefinders using laser light, light waves, or the like are often used.

However, in this case, separate analysis processing using separate hardware is required for measuring the in-plane displacement of the object and for measuring the displacement in the depth direction. As a result, there is a risk that the cost in terms of hardware or time required for displacement measurement will increase. On the other hand, as a method for measuring displacement in the depth direction using a marker image captured by a camera, a method as disclosed in Patent Document 2 is known. However, this method requires complicated image analysis based on the sampling moire method, which may lead to an increase in processing load and an increase in storage capacity.

SUMMARY OF THE INVENTION The present invention has been made in view of the above, and one of its objects is to provide a displacement measuring apparatus and a displacement measuring apparatus capable of measuring the displacement of a marker (and thus an object) in the depth direction with a small processing load or storage capacity. It is to provide a measuring method.

A brief outline of typical inventions disclosed in the present application is as follows.

A displacement measuring device according to a representative embodiment of the present invention includes an imaging unit that captures an image of a marker with a periodic pattern, and a displacement measuring unit that measures the displacement of the marker based on the marker image captured by the imaging unit. , has The displacement measurement unit has an internal marker creation unit, a moire image creation unit, and a moire fringe inclination calculation unit. The internal marker creating unit is a reference periodic pattern that is formed in a direction inclined by a reference inclination with respect to the reference direction when the formation direction of the periodic pattern included in the marker image is set as a reference direction, and whose period is determined in advance. Create an internal marker image containing The moire image creating unit generates a first marker image captured by the imaging unit at a first time point and a second marker image captured by the imaging unit at a second time point. A first moire image and a second moire image are created by superimposing the same created internal marker images. The moiré fringe tilt calculator calculates a first tilt formed by the first moiré fringes included in the first moiré image with respect to the reference direction and a second moiré pattern included in the second moiré image with respect to the reference direction. A second slope formed by the fringes is calculated. Then, the displacement measuring unit performs a predetermined arithmetic process using the first tilt and the second tilt, thereby obtaining a displacement in the optical axis direction of the imaging unit between the first point in time and the second point in time. Calculate the amount of displacement of the marker that has occurred.

Among the inventions disclosed in the present application, the effects obtained by representative ones are briefly described below. According to representative embodiments of the present invention, it is possible to measure the displacement of a marker (and thus an object) in the depth direction with a small processing load or memory capacity.

1(a) is a schematic diagram showing a configuration example of a displacement measurement system according to an embodiment of the present invention, and (b) and (c) are schematic diagrams showing different configuration examples of markers in (a). be. 2 is a block diagram showing a schematic configuration example of a displacement measuring device in FIG. 1; FIG. FIG. 3 is a block diagram showing a detailed configuration example of a main part of a displacement measuring unit in FIG. 2; FIG. 4 is a schematic diagram illustrating an example of processing contents of an internal marker creation unit in FIG. 3; FIG. 4 is a schematic diagram illustrating an example of processing contents of a moiré image creating unit in FIG. 3; FIG. 4 is a block diagram showing a detailed configuration example of a detection unit in FIG. 3; FIG. FIG. 7 is a schematic diagram illustrating an example of detailed processing contents of a moiré fringe inclination calculator in FIG. 6 ; FIG. 7 is a schematic diagram illustrating an example of detailed processing contents of a moiré fringe period calculation unit in FIG. 6 ; FIG. 3 is a diagram showing an example of part of the optical specifications of the imaging unit of FIG. 2; FIG. 2 is a flow diagram showing an example of processing contents in a displacement measuring method according to Embodiment 1 of the present invention; FIG. 7 is a diagram showing an example of operation verification results of the displacement measuring unit of FIGS. 3 and 6; FIG. 4 is a block diagram showing a detailed configuration example of a detector in FIG. 3 in the displacement measuring device according to Embodiment 2 of the present invention; 4 is a block diagram showing a detailed configuration example of a detector in FIG. 3 in the displacement measuring device according to Embodiment 3 of the present invention; FIG. FIG. 10 is a flowchart showing an example of processing contents in a displacement measuring method according to Embodiment 3 of the present invention;

Hereinafter, each embodiment of the present invention will be described with reference to the drawings. It should be noted that the disclosure is merely an example, and those skilled in the art can easily conceive appropriate modifications while keeping the gist of the invention are naturally included in the scope of the present invention. In addition, in order to make the description clearer, the drawings may schematically show the width, thickness, shape, etc. of each part compared to the actual embodiment, but this is only an example, and the interpretation of the present invention is not intended. It is not limited.

In addition, in this specification and each figure, the same reference numerals may be given to the same elements as those described above with respect to the existing figures, and detailed description thereof may be omitted as appropriate.

(Embodiment 1)
《Overview of Displacement Measurement System》
FIG. 1(a) is a schematic diagram showing a configuration example of a displacement measurement system according to Embodiment 1 of the present invention, and FIGS. 1(b) and 1(c) show markers in FIG. 1(a). It is a schematic diagram showing a different configuration example. The displacement measurement system shown in FIG. 1(a) has a marker 1 attached to an object 3 and a displacement measurement device 2 including an imaging unit such as a camera.

The object 3 is, for example, a building installed on the ground at a construction site such as shaft construction, or the ground itself. The displacement measuring device 2 captures an image of the marker 1 attached to the object 3, and measures the displacement of the marker 1 (and thus the displacement of the ground) based on the captured marker image. As a result, it is possible to monitor the displacement of the ground and, for example, detect early signs of an accident such as a landslide. As a result, accidents can be prevented and the safety of workers can be ensured.

In the specification, as shown in FIG. 1( a ), the optical axis direction of the camera (displacement measuring device 2 ) is defined as the Z axis, and one direction (horizontal direction) in the plane direction perpendicular to the Z axis is defined as X A direction perpendicular to the one direction (perpendicular direction) is the Y-axis. The camera (displacement measuring device 2) is installed so that the plane of the marker 1 is the XY plane. A periodic pattern is marked on the surface (XY surface) of the marker 1 as shown in FIG. 1(b) or FIG. 1(c).

The marker 1a shown in FIG. 1B has a striped pattern extending in the Y-axis direction and arranged at equal intervals at a pitch W along the X-axis direction. However, instead of the striped pattern aligned in the X-axis direction, a striped pattern aligned in the Y-axis direction may be used. Alternatively, a marker with stripes aligned in the X-axis direction and a marker with stripes aligned in the Y-axis direction may be used as a set of markers. These are used properly depending on the measurement direction of displacement.

The marker 1b shown in FIG. 1(c) is marked with a checkered pattern in which only the intersections of black level lines, which are arranged in an array with a pitch W at equal intervals, are defined as black levels. In this case, for example, by performing averaging processing along the Y-axis direction using image processing, a striped pattern (that is, a pattern as shown in FIG. 1B) arranged in the X-axis direction can be generated. A stripe pattern arranged in the Y-axis direction can be generated by performing the averaging process along the X-axis direction. As a result, in terms of image processing, the marker 1b can be regarded as a marker with stripes aligned in the X-axis direction, or as a marker with stripes aligned in the Y-axis direction.

A displacement measurement method using such a marker 1 is generally used when measuring the displacement of the marker 1 in the X-axis direction and the Y-axis direction. However, the displacement of the marker 1 (and thus the displacement of the ground) may occur not only in the X-axis direction and the Y-axis direction, but also in the Z-axis direction. Therefore, it is beneficial to use the method of the first embodiment described below.

《Overview of Displacement Measuring Device》
FIG. 2 is a block diagram showing a schematic configuration example of the displacement measuring device in FIG. A displacement measuring device 2 shown in FIG. Prepare. The imaging unit 10 is typically a digital camera or the like. The displacement measuring unit 20 is, for example, an information processing device such as a PC (Personal Computer), or a dedicated image processing device. However, the imaging unit 10 and the displacement measuring unit 20 may be mounted in the same device in the form of an information processing device with a camera, for example.

The imaging unit 10 includes a lens 11 , an image sensor 12 , a calculation unit 13 , a data storage unit 14 and a communication interface 15 . Among them, the calculation unit 13, the data storage unit 14, and the communication interface 15 are connected to each other via a bus. The calculation unit 13, the data storage unit 14 and the communication interface 15 may be implemented in one microcontroller, for example.

A lens 11 converges the light from the marker 1 onto the image sensor 12 . The image sensor 12 is typically a CMOS (Complementary Metal Oxide Semiconductor) image sensor, a CCD (Charge Coupled Device) image sensor, or the like, and includes a plurality of pixels arranged in an array. Each pixel of the image sensor 12 generates an electrical signal corresponding to the amount of light condensed by the lens 11 . The image sensor 12 transmits the electric signal generated by each pixel to the calculation unit 13 .

The data storage unit 14 is, for example, a nonvolatile memory such as a flash memory, and corresponds to a built-in memory within a microcontroller or an external memory such as a memory card. The calculation unit 13 receives the electrical signal from the image sensor 12 to generate a marker image, and stores the generated marker image in the data storage unit 14 . At this time, the calculation unit 13 may store the marker image in the data storage unit 14 after adding information on the imaging time.

The calculation unit 13 includes a processor 13a such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processor), and a RAM (Random Access Memory) 13b. The processor 13a creates a marker image corresponding to the electrical signal from the image sensor 12, for example, by executing a predetermined program developed from the data storage unit 14 to the RAM 13b.

The communication interface 15 transmits and receives data to and from the displacement measurement unit 20 (communication interface 21 therein). As one of them, the communication interface 15 transmits the marker image stored in the data storage unit 14 to the displacement measurement unit 20 . The communication interface 15 and the communication interface 21 are connected by wire or wirelessly. In this case, for example, a form of connection via an external network such as the Internet may be used.

When using an external network, for example, the imaging unit 10 equipped with the communication interface 15 for wireless communication is fixedly installed at the construction site, and the displacement measurement unit 20 is mounted on the in-house server device of the construction company. Morphology is beneficial. In this case, the imaging unit 10 sequentially transmits the captured marker images to the in-house server device via the external network, and the in-house server device performs displacement measurement based on the marker images.

The displacement measurement unit 20 includes a calculation unit 22 , a data storage unit 23 and a communication interface 21 . The computing unit 22, the data storage unit 23, and the communication interface 21 are connected to each other via a bus. For example, when the displacement measurement unit 20 is configured by a dedicated image processing device or the like, the calculation unit 22, the data storage unit 23 and the communication interface 21 may be implemented in one microcontroller. The data storage unit 23 is, for example, a non-volatile memory such as flash memory or hard disk drive. The communication interface 21 receives, for example, the marker image from the communication interface 15 and stores it in the data storage section 23 .

The computing unit 22 includes a processor 22a such as a CPU, GPU, or DSP, and a RAM 22b. The calculation unit 22 measures the displacement of the object 3 by executing predetermined image processing on the marker image stored in the data storage unit 23, for example. At this time, the processor 22a performs displacement measurement by, for example, executing a displacement measurement program expanded from the data storage unit 23 to the RAM 22b. Note that the arithmetic unit 22 is not limited to the processor 22a, and part or all of it may be configured by hardware such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). That is, the calculation unit 22 may be appropriately configured by a software system, a hardware system, or a combination thereof. The same applies to the calculation section 13 in the imaging section 10 .

<<Details of the displacement measurement unit>>
FIG. 3 is a block diagram showing a detailed configuration example of the main part of the displacement measuring unit in FIG. FIG. 4 is a schematic diagram for explaining an example of processing contents of an internal marker creation unit in FIG. FIG. 5 is a schematic diagram for explaining an example of processing contents of a moiré image creating unit in FIG. In the displacement measurement unit 20 of FIG. 3, the processor 22a includes an internal marker creation unit 31, a moiré image creation unit 32, and a detection unit 33 implemented by executing a displacement measurement program. Here, as an example, a marker 1a having a striped pattern as shown in FIG. 1B is used as the marker 1. FIG.

As shown in FIG. 4, the internal marker generator 31 forms markers in directions (X'-axis direction, Y'-axis direction) inclined by a reference inclination γ with respect to the reference direction (X-axis direction, Y-axis direction). An internal marker image 35 is created that includes a reference periodic pattern 30 whose period p is determined in advance. Here, the reference direction (X-axis direction, Y-axis direction) is the direction in which the periodic pattern included in the marker image captured by the imaging unit 10 is formed. It is also the direction in which the pattern is formed. The reference slope γ may be a value other than 0[°], and may be a positive value or a negative value. The internal marker creation unit 31 stores the created internal marker image 35 in the data storage unit 23 .

Specifically, the internal marker creation unit 31 can create the internal marker image 35 using a computer graphics (CG) image, for example. That is, the marker image captured by the imaging unit 10 in FIG. 2 at the stage immediately after the construction of the displacement measurement system as shown in FIG. can be calculated by The internal marker creation unit 31 may create the internal marker image 35 after determining the reference direction (X-axis direction, Y-axis direction) based on the calculated marker image.

In addition, the internal marker creation unit 31 causes the imaging unit 10 in FIG. 2 to actually perform imaging at the stage immediately after the displacement measurement system is constructed (the stage at which no displacement has occurred), and determines the reference inclination of the marker image. The internal marker image 35 may be created by performing γ rotation processing. At this time, the period p of the periodic pattern included in the internal marker image 35 can be calculated by performing a simulation or the like in advance, assuming that no displacement has occurred.

The data storage unit 23 of FIG. 3 stores a measurement marker image 36a captured by the imaging unit 10 at time point [1] and a measurement marker image 36b captured by the imaging unit 10 at time point [2]. ing. The moiré image creating unit 32 creates moire images by the internal marker creating unit 31 as shown in FIG. A moiré image 37 (that is, moiré image [1] and moiré image [2]) is created by superimposing the same internal marker images 35 .

As shown in FIG. 5, the moire image 37 includes moire fringes that change with a period (wavelength) λ. The extending direction of the moire fringes forms an inclination θ with respect to the reference direction (here, the X-axis direction). In the specification, the extending direction of the moiré fringes is called the X ^* axis direction, and the periodic direction of the moiré fringes is called the Y ^* axis direction.

The detection unit 33 in FIG. 3 uses the moire image [1] and the moire image [2] to determine the amount of displacement of the marker 1 (and thus the object 3) occurring between the time points [1] and [2]. To detect. As one of them, the detection unit 33 detects the time point [ 1] and time point [2].

<<Details of detector>>
FIG. 6 is a block diagram showing a detailed configuration example of the detector in FIG. FIG. 7 is a schematic diagram for explaining an example of detailed processing contents of a moire fringe inclination calculator in FIG. FIG. 8 is a schematic diagram for explaining an example of detailed processing contents of a moire fringe period calculation unit in FIG. 6 . The detector 33 shown in FIG. 6 includes a moiré fringe inclination calculator 40 , a moiré fringe period calculator 41 , and a depth direction displacement amount calculator 42 .

The moiré fringe inclination calculator 40 calculates the inclination θ1 formed by the moiré fringes [1] included in the moiré image [1] with respect to the reference direction (for example, the X-axis direction) and the moiré with respect to the reference direction (X-axis direction). Inclination θ2 formed by moiré fringes [2] included in image [2] is calculated. Specifically, as shown in FIG. 7, the moiré fringe inclination calculator 40 first binarizes the luminance of each pixel with "0" or "1" for each moiré image 37. A binarized image 50 is created (step S10). Subsequently, the moire fringe inclination calculator 40 extracts the apex 52 of each rhombus pattern 51 included in the binarized image 50 (step S11).

Next, the moire fringe inclination calculator 40 detects an approximate straight line 53 connecting the extracted vertices 52 (step S12). At this time, the moiré fringe inclination calculator 40 can detect the straight line 53 (that is, the moiré fringes) using, for example, Hough transform. After that, the moiré fringe inclination calculator 40 calculates an inclination θ formed by the straight line 53 (stretching direction of the moiré fringes) with respect to a predetermined reference direction (X-axis direction) (step S13). More specifically, a plurality of straight lines 53 can be detected in the binarized image 50 by step S12. The plurality of straight lines 53 may slightly differ in inclination θ from each other. In this case, the moiré fringe tilt calculator 40 calculates the final tilt θ by, for example, the average value of the tilts θ that are different from each other.

The moiré fringe period calculator 41 in FIG. 6 receives the tilts θ1 and θ2 from the moiré fringe tilt calculator 40, calculates the period λ1 of the moiré fringes [1] in the direction (Y ^* -axis direction) based on the tilt θ1, Calculate the period λ2 of the moire fringes [2] in the direction (Y ^* -axis direction) based on the inclination θ2. Specifically, as shown in FIG. 8, the moire fringe period calculator 41 calculates the luminance I(x, y) of each pixel in the reference direction (X-axis direction, Y-axis direction) in each moire image 37 as , to luminance I ^* (x ^* , y ^* ) of each pixel in directions (X ^* -axis direction, Y ^* -axis direction) forming an inclination θ with respect to the reference direction. Thereby, the luminance distribution of the moire fringes can be represented by a trigonometric function that changes along the Y ^* -axis coordinate (periodic direction of the moire fringes).

Therefore, the moiré fringe period calculator 41 approximates the luminance I ^* of each pixel arranged in the Y ^* axis direction by a trigonometric function (cos function in this example) shown in Equation (1). For this approximation, for example, the method of least squares or the like is used. Also, the trigonometric function may be a sine function. Then, the moiré fringe period calculator 41 calculates the moiré fringe period λ [pixel] based on the approximation formula (1). In equation (1), “y ^* ” is the Y ^* -axis coordinate value of pixels aligned in the Y ^* -axis direction, and the X-axis coordinate value “x ” and the Y-axis coordinate value “y” of the pixels arranged in the Y-axis direction. Also, "A" is the amplitude, "α" is the initial phase, and "B" is the background luminance.

The depth direction displacement amount calculator 42 of FIG. 6 calculates the difference between the period λ1 of the moiré fringes [1] and the period λ2 of the moiré fringes [2] between the time point [1] and the time point [2]. , the amount of displacement of the marker 1 generated in the optical axis direction (depth direction) of the imaging unit 10 is calculated. Specifically, the depth direction displacement amount calculator 42 first uses the period λ of the moiré fringes, the period p of the reference periodic pattern 30 shown in FIG. 4, and the reference inclination β shown in FIG. Then, the marker image cycle w, which is the cycle of the periodic pattern included in the measurement marker image 36 of FIG. 5, is calculated based on the equation (2).

Formula (2) is a theoretical formula (Formula (31)) shown in Non-Patent Document 1 mentioned above. Further, the reference inclination β used in Equation (2) is the inclination formed by the Y′-axis direction (the extending direction of the stripe lines of the periodic pattern) with respect to the X-axis direction, as shown in FIG. 5 . The depth direction displacement amount calculation unit 42 calculates the marker image period w1 and A marker image period w2 is calculated.

Subsequently, the depth direction displacement amount calculation unit 42 calculates the marker image period w [pixels] calculated by the formula (2) and the period W [mm] of the periodic pattern marked on the marker 1a in FIG. , and the known optical specifications of the imaging unit 10 (camera), the imaging distance D between the imaging unit 10 and the marker 1a is calculated. FIG. 9 is a diagram showing an example of a part of the optical specifications of the imaging unit in FIG. 2. FIG. As shown in FIG. 9 , the period W [mm] on the marker plane corresponds to the marker image period w [pixels] on the sensor plane through the lens 11 . The distance between the lens 11 and the sensor surface is approximately equal to the focal length f. Using such a relationship, the depth direction displacement amount calculator 42 calculates an unknown shooting distance D [mm] between the lens 11 and the marker surface.

As an example of the calculation method, the depth direction displacement amount calculation unit 42 first calculates the imaging surface resolution DR [mm/pixel] based on Equation (3). Furthermore, the depth direction displacement amount calculation unit 42 uses the calculated imaging surface resolution DR [mm/pixel] and the known number of pixels PN [pixel] based on the optical specifications of the imaging unit 10 to calculate the following equation (4) The imaging range IR [mm] is calculated based on.
DR=W/w (3)
IR=DR×PN (4)

On the other hand, based on the optical specifications of the imaging unit 10, the angle of view φ [°] of Equation (5) is known in advance using the focal length f and the sensor size SS. The depth direction displacement amount calculation unit 42 uses this angle of view φ [°] and the imaging range IR [mm] of Expression (4) to calculate the shooting distance D [mm] based on Expression (6). . Then, the depth direction displacement amount calculation unit 42 calculates the shooting distance (D[1]) calculated based on the moire image [1] at a certain point [1] and the moire image [ 2], the marker 1a generated in the Z-axis direction (depth direction) between time points [1] and [2] is calculated. Detects the amount of displacement of
φ=2×tan ⁻¹ {SS/(2×f)} (5)
D=IR/{2×tan(φ/2)} (6)

《Displacement measurement method》
FIG. 10 is a flowchart showing an example of processing contents in the displacement measuring method according to Embodiment 1 of the present invention. First, the internal marker creation unit 31 in FIG. 3 creates the internal marker image 35 as shown in FIG. 4 (step S101). Subsequently, the displacement measuring unit 20 in FIG. 3 acquires the measurement marker image [1] by imaging the marker 1 using the imaging unit 10 (step S102). Next, the moire image creating unit 32 in FIG. 3 creates a moire image [1] by superimposing the internal marker image 35 on the measurement marker image [1] as shown in FIG. 5 (step S103).

Then, the detection unit 33 in FIG. 3 (FIG. 6) detects the period [1] of the moire fringes from the moire image [1] through the various processes described with reference to FIGS. 7 and 8 (step S104). . Here, for example, it is assumed that the processes of steps S102 to S104 are performed at the stage immediately after the construction of the displacement measuring system of FIG. In this case, the period [1] of the moire fringes detected in step S104 is the reference period when no displacement occurs.

After that, a standby state is established until the time of displacement measurement (step S105). For example, the displacement measurement time point may be arbitrarily determined by the user, or may be determined periodically (for example, once a day) by the displacement measuring device 2 based on an internal timer or the like. When the displacement measurement time is reached in step S105, the displacement measuring unit 20 performs the same processing as in steps S102 to S104 in steps S106 to S108.

That is, the displacement measuring unit 20 acquires the measurement marker image [n] by capturing an image of the marker 1 using the imaging unit 10 (step S106). The moire image creation unit 32 creates a moire image [n] by superimposing the internal marker image 35 created in step S101 on the measurement marker image [n] (step S107). The detection unit 33 detects the period [n] of moiré fringes in the moiré image [n] (step S108).

After that, in step S109, the detection unit 33 uses formulas (2) to (6) based on the period difference between the period [1] detected in step S104 and the period [n] detected in step S108. By performing such processing, the amount of displacement in the depth direction (Z-axis direction) from the initial time that occurred in the marker 1 (and thus the object 3) is detected (step S109). After that, the processing of steps S105 to S109 is repeatedly executed until displacement measurement becomes unnecessary (step S110). As a result, in this example, the amount of displacement in the depth direction of the marker 1 (and thus the object 3) from the initial point is detected at each displacement measurement point.

<< operation verification result >>
FIG. 11 is a diagram showing an example of operation verification results of the displacement measuring unit of FIGS. 3 and 6. FIG. In this example, measurement marker images 36 (actual marker images) whose marker image period w is shortened by 0.02 times are sequentially input to the displacement measuring unit 20 in FIG. Then, each time, the calculated value of the marker image period w calculated by the equation (2) is shown through the various processes described with reference to FIGS. 3 and 6 . Shortening the marker image period w means increasing the distance in the depth direction (Z-axis direction) between the marker 1 and the displacement measuring device 2 in FIG. 1(a).

As shown in FIG. 11, a value close to the ideal value was obtained as the calculated value of the marker image period w calculated based on Equation (2). In this way, when a calculated value (w) close to the ideal value is obtained, based on the known optical specifications as described in equations (3) to (6), as a result, the depth direction (Z-axis direction ) can be calculated with high accuracy.

<<various modifications>>
Here, the case of using the marker 1a on which a stripe pattern arranged in the X-axis direction as shown in FIG. 1B is used has been described as an example. Alternatively, however, a marker with a striped pattern arranged in the Y-axis direction may be used, or a marker 1b with a checkered pattern as shown in FIG. 1(c) may be used. When the marker 1b is used, the displacement measuring unit 20 may convert the marker image including the checkered pattern into a striped pattern aligned in the X-axis direction or a striped pattern aligned in the Y-axis direction by image processing.

<<Main effects of the first embodiment>>
As described above, by using the displacement measuring apparatus and displacement measuring method of the first embodiment, it is typically possible to measure the displacement of the marker 1 (and thus the object 3) in the depth direction with a small processing load or storage capacity. That is, it becomes possible to detect the displacement amount of the marker 1 (and thus the object 3) as a whole in the depth direction without using complicated processing such as the sampling moire method. As a result, for example, at a construction site such as shaft construction, it is possible to monitor the state of ground displacement and ensure the safety of workers. Moreover, since the displacement in the depth direction can be measured without using a rangefinder using a laser beam, light wave, or the like, the cost can be reduced.

Furthermore, it becomes possible to measure the displacement in the depth direction with high accuracy. That is, for example, in FIG. 8, a method of calculating the period of moire fringes in the X-axis direction or the Y-axis direction is conceivable. In this case, since the X-axis direction or the Y-axis direction is different from the periodic direction of the moire fringes, an error may occur in the calculated period. When the method of Embodiment 1 is used, the periodic direction of the moire fringes can be detected and then the period of the moire fringes can be calculated in that periodic direction, so a highly accurate period can be obtained. It becomes possible to measure with high accuracy.

(Embodiment 2)
<<Details of detector>>
FIG. 12 is a block diagram showing a detailed configuration example of the detector in FIG. 3 in the displacement measuring device according to Embodiment 2 of the present invention. The detector 33 shown in FIG. 12 is not provided with the moiré fringe period calculator 41 and differs from the configuration example shown in FIG. 6 in the processing contents of the depth direction displacement amount calculator 42a. Unlike the configuration example of FIG. 6, the depth direction displacement amount calculation unit 42a receives the inclination θ1 of the moiré fringe [1] and the inclination θ2 of the moiré fringe [2] from the moiré fringe inclination calculator 40, and calculates each moiré fringe. The amount of displacement of the marker 1 in the depth direction is calculated without calculating the period of .

Specifically, the depth direction displacement amount calculation unit 42a uses the inclination θ of the moire fringes, the period p of the reference periodic pattern 30 shown in FIG. 4, and the reference inclination β shown in FIG. A marker image period w, which is the period of the periodic pattern included in the measurement marker image 36 of FIG. 5, is calculated based on the expression (7). Equation (7) is a theoretical equation (equation (32)) shown in Non-Patent Document 1 mentioned above. The depth direction displacement amount calculation unit 42a calculates the marker image period w1 and A marker image period w2 is calculated.

Then, the depth direction displacement amount calculation unit 42a calculates the optical axis direction (depth direction) is calculated. Specifically, the depth direction displacement amount calculation unit 42a calculates the marker image period w and the period of the periodic pattern marked on the marker 1 as described in the formulas (3) to (6) of the first embodiment. A shooting distance D is calculated based on W and the known optical specifications of the imaging unit 10 . Then, the depth direction displacement amount calculator 42a calculates the amount of change in the photographing distance D due to the period difference between the marker image period w1 and the marker image period w2.

<<Main effects of the second embodiment and comparison with the first embodiment>>
As described above, by using the displacement measuring device of the second embodiment, the same effects as those described in the first embodiment can be obtained. Furthermore, as compared with the method of the first embodiment, processing for calculating the period of moire fringes is not required, making it possible to further reduce processing load or storage capacity. On the other hand, from the viewpoint of the measurement accuracy of the displacement amount in the depth direction, the method of the first embodiment is preferable.

Specifically, in the method of the second embodiment, the calculation accuracy of the inclination θ directly affects the measurement accuracy of the displacement amount in the depth direction. The inclination θ is calculated by extracting the vertex 52 from the binarized image 50, as described with reference to FIG. At this time, the position of the vertex 52 may vary depending on various noise components such as quantization errors, so there is a possibility that the calculation accuracy of the slope θ may decrease. On the other hand, in the method of Embodiment 1, even if the calculation accuracy of the slope θ is insufficient, it is possible to calculate the period λ with a certain degree of accuracy by approximating the trigonometric function based on Equation (1). It is possible to measure the amount of displacement in the depth direction based on this cycle λ.

(Embodiment 3)
<<Details of detector>>
FIG. 13 is a block diagram showing a detailed configuration example of the detector in FIG. 3 in the displacement measuring device according to Embodiment 3 of the present invention. The detection unit 33 shown in FIG. 13 includes a depth direction displacement detection unit 60 and a plane direction displacement detection unit 61 . The depth direction displacement detector 60 has the same configuration as in FIG. The planar displacement detector 61 includes a moire fringe phase calculator 65 and a planar displacement amount calculator 66 . The moiré fringe phase calculator 65 calculates a phase α1 of moiré fringes [1] included in moiré image [1] and a phase α2 of moiré fringes [2] included in moiré image [2].

Specifically, when the marker 1a in FIG. 1B is used, the moiré fringe phase calculator 65 calculates, for example, X The luminance _IX of each pixel arranged in the axial direction is approximated by a trigonometric function (cos function in this example) shown in Equation (8). For this approximation, for example, the method of least squares or the like is used. Also, the trigonometric function may be a sine function.
I _X =A _X ×cos {(x/λ _X )×2π+α _X }+B _X (8)

In equation (8), “A _X ” is the amplitude, “x” is the X-axis coordinate value of the pixels arranged in the X-axis direction, and “λ _X ” is the period (wavelength) [pixel] of the moire fringes. , where “α _X ” is the initial phase and “B _X ” is the background luminance. The moiré fringe phase calculation unit 65 approximates the moiré image [1] by Equation (8), and calculates the resulting “α _X ” as the phase α1 of the moiré fringe [1]. Similarly, the moiré fringe phase calculator 65 approximates the moiré image [2] by Equation (8), and calculates the resulting “α _X ” as the phase α2 of the moiré fringe [2].

Based on the phase difference Δα between the phase α1 of the moiré fringes [1] and the phase α2 of the moiré fringes [2] from the moiré fringe phase calculator 65, the planar displacement amount calculator 66 calculates the time point [1] and the time point [2] and the amount of displacement in the surface direction of the marker 1 (here, the X-axis direction) is calculated. Specifically, using the phase difference Δα [rad] and the known pitch (cycle) W [mm] of the marker 1a shown in FIG. ), the displacement amount ΔU (here, the displacement amount ΔX in the X-axis direction) is calculated.
ΔU=(Δα/2π)×W (9)

《Displacement measurement method》
FIG. 14 is a flowchart showing an example of processing contents in a displacement measuring method according to Embodiment 3 of the present invention. In the flow shown in FIG. 14, steps S201 to S203 are added as compared with the flow shown in FIG. Step S201 is inserted between steps S104 and S105. In step S201, the moiré fringe phase calculator 65 detects the phase [1] of the moiré fringes [1] for the moiré image [1].

Steps S202 and S203 are sequentially inserted between steps S109 and S110. In step S202, the moiré fringe phase calculator 65 detects the phase [n] of the moiré fringes [n] for the moiré image [n]. In step S203, the in-plane displacement amount calculation unit 66 calculates the position of the phase [1] of the moiré fringes [1] detected in step S201 and the phase [n] of the moiré fringes [n] detected in step S202. The phase difference is detected, and the amount of displacement generated in the plane direction (here, the X-axis direction) of the marker 1 is detected based on Equation (9).

<<Main effects of the third embodiment>>
As described above, by using the displacement measuring device and the displacement measuring method of the third embodiment, in addition to the various effects described in the first embodiment, based on the marker image acquired using the single imaging unit 10 , both the displacement in the plane direction (X-axis direction or Y-axis direction) and the displacement in the depth direction (Z-axis direction) of the marker 1 (and thus the object 3) can be measured. As a result, low-cost and efficient displacement measurement can be performed.

That is, displacement measurement in the Y-axis direction and the Z-axis direction becomes possible by using a marker with a striped pattern arranged in the Y-axis direction instead of the marker 1a in FIG. 1(b). Also, by using the marker 1b in FIG. 1C instead of the marker 1a in FIG. 1B, displacement measurement in the X-axis direction, the Y-axis direction, and the Z-axis direction becomes possible. When measuring the displacement in the Y-axis direction, the brightness (I _Y ) of each pixel arranged in the Y-axis direction should be approximated by a trigonometric function in the same manner as in Equation (8).

Although the invention made by the inventor has been specifically described above based on the embodiment, the invention is not limited to the embodiment, and can be variously modified without departing from the gist of the invention.

1, 1a, 1b... Marker 2... Displacement measuring device 3... Object 10... Imaging unit 20... Displacement measuring unit 30... Reference periodic pattern 31... Internal marker creating unit 32... Moiré image creating unit 35... Internal marker image 36, 36a, 36b... Marker image for measurement 37... Moire image 40... Moire fringe inclination calculator 41... Moiré fringe period calculator 42, 42a... Depth direction displacement amount calculator 65 Moire fringe phase calculator 66 In-plane displacement amount calculator p, W, λ cycle, w marker image cycle, β, γ, θ slope

Claims (5)

an imaging unit that captures an image of a marker marked with a periodic pattern;
a displacement measuring unit that measures the displacement of the marker based on the marker image captured by the imaging unit;
A displacement measuring device having
The displacement measuring unit
When the forming direction of the periodic pattern included in the marker image is taken as a reference direction, an interior including a reference periodic pattern formed in a direction inclined by a reference inclination with respect to the reference direction and having a predetermined period. an internal marker creation unit that creates a marker image;
Created by the internal marker creating unit for a first marker image captured by the imaging unit at a first time point and a second marker image captured by the imaging unit at a second time point a moire image creation unit that creates a first moire image and a second moire image by superimposing the same internal marker images;
A first tilt formed by the first moire fringes included in the first moire image with respect to the reference direction and a second moire fringe included in the second moire image with respect to the reference direction. a moire fringe tilt calculator that calculates the second tilt and
calculating a first marker image period, which is a period of a periodic pattern included in the first marker image, based on the first inclination, the period of the reference periodic pattern, and the reference inclination; calculating a second marker image period, which is the period of the periodic pattern included in the second marker image, based on the second inclination, the period of the reference periodic pattern, and the reference inclination; Based on the period difference between the one marker image period and the second marker image period, the number of the markers generated in the optical axis direction of the imaging unit between the first time point and the second time point is determined. a depth direction displacement amount calculation unit that calculates the amount of displacement;
has a
Displacement measuring device. In the displacement measuring device according to claim 1,
The moiré fringe tilt calculator calculates the first tilt and the second tilt by detecting the first moiré fringes and the second moiré fringes using a Hough transform, respectively.
Displacement measuring device. In the displacement measuring device according to claim 1,
The depth direction displacement amount calculation unit,
A first photographing distance between the imaging unit and the marker is calculated based on the first marker image period, the period of the periodic pattern marked on the marker, and known optical specifications of the imaging unit. death,
A second photographing distance between the imaging unit and the marker is calculated based on the second marker image period, the period of the periodic pattern marked on the marker, and known optical specifications of the imaging unit. do,
Displacement measuring device. In the displacement measuring device according to claim 1,
The depth direction displacement amount calculation unit uses the following formula, where the first inclination or the second inclination is "θ", the period of the reference periodic pattern is "p", and the reference inclination is "β". calculating the first marker image period or the second marker image period "w" based on

Displacement measuring device. In the displacement measuring device according to claim 2,
The moiré fringe inclination calculator generates a binarized image by binarizing the luminance of each pixel of each of the first moiré image and the second moiré image, and performs the binarization. Calculating the first slope and the second slope by extracting the vertices of the rhombus pattern included in the image and detecting an approximate straight line connecting the extracted vertices using a Hough transform;
Displacement measuring device.

Images