JPH08134958A - Remote support work support image system - Google Patents

Abstract

(57) [Summary] [Purpose] Providing a system that shows the target shape image during work on the design shape image. [Structure] On a heavy machine 2 for remote construction, a movable target 8, a measurement camera 5 for taking a measurement image of a construction target surface 1 on which a design image has been created and the target 8 from different directions, and a three-dimensional target 8 A satellite surveying device 22 for measuring coordinates and transmitting devices 6, 24 for transmitting an image and target coordinates are provided. Control room for remote operation 3
Further, receiving devices 10 and 26, an operation display 14, and a measurement display 15 for displaying a measurement image are provided. Correspondence between a set of two-dimensional image coordinates of an arbitrary point and a three-dimensional coordinate of the arbitrary point from a set of two-dimensional image coordinates of the target in a plurality of measurement images from different directions and the three-dimensional coordinates of the target Calculate the relationship. From the set of the two-dimensional image coordinates of the representative point on the target surface in the plurality of measurement images, the three-dimensional coordinates of each representative point are obtained from the correspondence and the operation image of the target surface 1 is created from the three-dimensional coordinates of the representative points. , The operation image is superimposed on the design image and shown on the operation display.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote construction work support image system, and more particularly, an image for supporting a remote construction work by superimposing and displaying a design image showing a design shape and an operation image showing the current state. Regarding the system.

[0002]

2. Description of the Related Art With the progress of control technology, in order to operate in a dangerous environment or in a place where it is difficult for workers to enter, place only the machinery to be operated in the operating place and There is an increasing tendency for remote control to operate machinery in a remote and safe control room. Ground maintenance and other earthworks are not exceptions to this tendency. For example, as shown in Fig. 3, remote construction is practiced by operating a soil treatment machine such as a heavy machine 2 that performs soil treatment on a work surface 1 from a remote location. Has been adopted by.

[0003]

In the conventional earthwork, a rope indicating the position to be worked and the design shape of the work object is installed as a so-called tension, and the worker uses the tension as a guide, and the current situation and the design. Work while checking the difference with the shape. However, in remote construction, it is extremely difficult to install and check the strut. For this reason, compared to construction using stake,
There is a problem that remote construction tends to be inferior in quality.

Therefore, an object of the present invention is to provide an image system for remote construction support, which enables comparison between the design shape of a work target and the current state.

[0005]

The present inventor can represent the current shape of the construction target surface in earthwork as an image (operation image) by the techniques of satellite surveying and image processing, and use CAD (computer use). (Design) If the design image of the design shape display created by a device etc. and this operation image are appropriately adjusted and superimposed and displayed on the display, it is possible to achieve a function like striking by this superimposed display. did.

Referring to FIG. 1, a remote construction work support image system according to the present invention has three-dimensional coordinates (X, Y,
Z) a measurement image (I _w1 , I _{w2 in} FIG. 5) of the target 8 having the known design image Id (FIGS. 3 and 6) and the construction image Id (FIG. 3 and FIG. 6), which are measured at a plurality of predetermined positions. Take from different direction,
A set of two-dimensional image coordinates (Xa, Ya; Xb, Yb in FIG. 5) of the target 8 on a plurality of measurement images (I _w1 , I _w2 ) is detected, and the set of the detected two-dimensional image coordinates and the known value are detected. From the three-dimensional coordinates of the above, the corresponding relationship between the set of the two-dimensional coordinates of the arbitrary points on the plurality of measurement images and the three-dimensional coordinates of the arbitrary points (Q ^{−1 of the} equation (10)) is calculated,
Representative points (M ₁ , M ₂ ; N ₁ , N _{2 in} FIG. 5) on the construction target surface 1 in the measurement images (I _w1 , I _w2 ) from the predetermined position.
...) two-dimensional image coordinate set (X _M1 , Y _M1 ; X _M2 , Y _M2 ;
...) is detected and the three-dimensional coordinates (X _M , X _M ,
Y _M , Z _M ; ...) is obtained, and the construction target surface 1 viewed from the same viewpoint as the viewpoint of the design image Id from the obtained three-dimensional coordinates of the representative point
Operation image I _w (Fig. 3 and Fig. 6) of
Operation image I _w and design image Id are superimposed and superimposed image Is
(FIGS. 3 and 6).

Preferably, the remote construction work support of the present invention
The image system includes one or more movable targets 8 and a design image Id
Measurement images of the construction target surface 1 and optotype 8 that have already been formed (I_w1, I_w2)
And a measuring camera 5 for photographing the
A satellite that measures the three-dimensional coordinates (X, Y, Z) of the mark 8 to the ground
Surveying device 22 and transmitting device 6 for transmitting measurement images and coordinates
Heavy equipment 2 having: a receiving device 10 for receiving from a transmitting device 6
, Operation display 14 and measurement image (I_w1, I_w2)display
Remote control of heavy equipment 2 with measuring display 15 for
Control room 3; duplicates from different directions on the measurement display 15
Number measurement image (I _w1, I_w2) 2D seat of specific target 8
A set of markers (Xa, Ya; Xb, Yb) and the three-dimensional seat of the specific target 8
A set of two-dimensional coordinates of an arbitrary point from the mark (X, Y, Z) and
Correspondence with the three-dimensional coordinates of the arbitrary point (Q in equation (11)^-1)
Calculation means 16 for calculating a plurality of measurement images (I_w1, I_w2)
Each of a plurality of representative points (M, N, ...) On the target surface 1
Set of two-dimensional image coordinates of (X_M1, Y_M1; X_M2, Y_M2; ......)
And the above correspondence (Q^-1) By the tertiary of the respective representative points
Original coordinate (X_M, Y_M, Z_M; ………)
Target viewed from the same viewpoint as the viewpoint of the design image Id from dimensional coordinates
Operation image I on surface 1_wOperation image creating means 18 for creating
Design image Id and operation image I_wAnd are superimposed and the superimposed image Is and
By the superimposing means 20 which is displayed on the operation display 15.
To configure.

[0008]

[Operation] Section No. in the earthwork for residential land development shown in Fig. 2.
A case of constructing 12 completed slopes M ₀ N ₀ R ₀ S ₀ by remote control will be described. In this case, the points M ₀ , N ₀ , R ₀ , and S ₀ are representative points assumed on the completion slope. The three-dimensional coordinates of each representative point can be expressed as relative coordinates with respect to the reference point 30 from the design drawing. Preferably, the completed slope surface coordinates including the three-dimensional coordinates are stored in the storage device 17 (FIG. 1).
The surface 1 to be constructed in FIGS. 1 and 3 shows the shape of this slope before work, which is being worked.

The three-dimensional coordinates of the representative points M ₀ , N ₀ , R ₀ , and S ₀ can also be expressed as earth coordinates such as longitude, latitude, and elevation from the sea surface. May be stored in the storage device 17. Furthermore, the storage device 17 is used as a component of an appropriate CAD device or other design image production device 19 (FIG. 1), and the three-dimensional coordinates of the representative points M ₀ , N ₀ , R ₀ , S ₀ are used by the CAD device or the like. It may be stored in the device 19 as a system.

In the image system of the present invention, the heavy equipment 2 for construction such as a power shovel used for earthwork is equipped with the following equipment. Operation camera 4: Takes an image of the construction target surface 1. The operator of the remote construction operates while monitoring the output image of this camera. Measurement camera 5: Takes a measurement image of the construction target surface 1 and the optotype 8. Antenna device 7 (see FIGS. 3 and 4): satellite surveying device 22
Antenna of the image transmitter 6, the antenna of the position information transmitter 24, and the like. Image transmitter 6: Sends images from the operation and measurement cameras to the control room. Target 8: Used for calculating the coefficient of stereo image measurement. It preferably has a target position control device 9 (see FIGS. 3 and 4).

The present invention is based on the operation image I _w of the target surface 1 during work.
First, the coordinates of the representative points M, N, R, S, etc. on the target surface 1 are measured by a so-called passive image measurement method that uses a measurement image from the measurement camera 5. After that, for example, a plane or a curved surface defined by a line connecting the representative points is obtained, and the plane or the curved surface is drawn as a perspective view from the same viewpoint as the viewpoint of the design image Id. If necessary, the image of the construction target surface 1 taken by the operation camera 4 can be processed into a perspective view viewed from the same viewpoint as the viewpoint of the design image Id in consideration of the coordinates of the representative point.

The passive image measuring method includes a stereo image measuring method, a three-eye image measuring method, and a multi-eye image measuring method.
Here, the two measurement cameras 5 and 2 are calculated by the equations (1) to (11).
The principle of the stereo image measuring method using the measurement display 15 of the table will be briefly described.

A measurement image of one measurement camera 5 with respect to the target surface 1 at a certain time point is I _w1 in FIG. 5 and is displayed on one measurement display 15, and the other measurement with respect to the same target surface 1 at the same point is performed. It is assumed that the measurement image from the different direction by the use camera 5 is I _w2 in FIG. 5 and is displayed on the other measurement display 15. Specific viewpoint 8
The three-dimensional coordinates of (X, Y, Z), the coordinates of the image 8 ₁ of the target 8 in the measurement image I _w1 of the one measurement display 15 are (Xa, Ya), and the other measurement Display
The coordinates of the image 8 ₂ of the target 8 in the measurement image I _w2 on 15 are (X
b, Yb), the following relational expressions (1) to (11) are established.

[0014]

[Equation 1]

Equation (11) is the three-dimensional coordinates (X, Y,
Z) shows two measurement images I of the arbitrary point viewed from different directions.
It is shown that it can be calculated from the two-dimensional coordinates (Xa, Ya) and (Xb, Yb) on _w1 and I _w2 . As can be seen from the comparison of equations (1), (2) and (9), the three-dimensional coordinates (X, Y, Z) of an arbitrary point and the two-dimensional coordinates (Xa, Ya) and (Xb, Yb) on the measurement image are shown. ) Is defined by a matrix A composed of 12 constants A _{11 to} A ₃₄ of the formula (1) and a matrix B composed of 12 constants B _{11 to} B _{34 of the} formula (2).

According to the observation of the equation (1), if the above-mentioned two-dimensional image coordinates (Xa, Ya) and (Xb, Yb) can be measured for six points which are not on the same plane and whose three-dimensional coordinates are known, The 12 constants A _{11 to} A ₃₄ of the matrix A can be calculated. If the number of points with known three-dimensional coordinates and image coordinates exceeds 6 and is 7 or more, the values of the constants A _{11 to} A ₃₄ can be more accurately determined. The optotype 8 in the present invention has the constants A _{11 to} A _34.
It is for calculating etc.

The three-dimensional coordinates of the heavy equipment 2 itself are calculated by the satellite surveying device 22.
The three-dimensional coordinates of the visual target 8 are determined by measuring with. This is because the relative position of the target 8 with respect to the heavy machine 2 is tracked before and after the movement of the target 8 described below in the embodiments of FIGS. If the measurement camera 5 photographs the target 8 from different directions, the two-dimensional image coordinates of the target 8 can be obtained on the measurement display 15. Six or more optotypes 8 at known positions that are not on the same plane are provided, or one optotype 8 is moved to six or more known positions that are not on the same plane and their two-dimensional image coordinates (Xa, Ya) measuring display 15
If measured above, the constants A _{11 to} A ₃₄ can be calculated by the equation (1). The calculation means 16 of FIG. 1 performs this calculation. By the same operation, the constants B _{11 to} B ₃₄ can also be calculated by the equation (2).

Once the constants A _{11 to} A ₃₄ and the constants B _{11 to} B ₃₄ are calculated, the two-dimensional image coordinates (Xa, Ya) and (Xb, Yb) of the image photographed from different directions about arbitrary points are calculated. If the measurement is performed on the measurement display 15, the term F of the formula (11) and the inverse matrix Q ⁻¹ can be calculated, and therefore the three-dimensional coordinate V (X, Y, Z) of the point is obtained by the relation of V = Q ⁻¹ F. ) Can be calculated. FIG.
The calculation means 16 also performs this calculation.

Representative points M, N, R, and
The creation of the operation image I _w including S, U and V will be described. Figure 5
In, the measurement image I _w1 from the different direction with respect to the target surface 1
And I _w2 on the representative points M ₁ , N ₁ , R ₁ , S ₁ , U ₁ , V ₁ and M ₂ ,
There are _{N 2, R 2, S 2} , U 2, V 2. Two-dimensional image coordinates (X _M1 , Y _M1 ), (X _N1 , Y _N1 ), (X _R1 , Y _R1 ), from those representative points
(X _{S 1} , Y _S1 ), (X _U1 , Y _U1 ), (X _V1 , Y _V1 ) and (X _M2 , Y _M2 ), (X _N2 , Y _N2 ), (X _{R 2} , Y _R2 ), (X _S2 , Y
_S2 ), (X _U2 , Y _U2 ), (X _V2 , Y _V2 ). These two-dimensional image coordinates, wherein the three-dimensional coordinates of the representative points by _{(11) (X M, Y} M, Z M), (X N, Y N, Z N), (X R, Y R, Z _R ),
(X _S , Y _S , Z _S ), (X _U , Y _U , Z _U ) and (X _V , Y _V , Z _V ) are determined.

If the three-dimensional coordinates of the plurality of representative points M, N, etc. are determined, the operation image I _w of the target surface 1 viewed from the same viewpoint as the viewpoint of the design image Id by the operation image creating device 18 of FIG. Can be created. The operation image I _w is most easily represented by their representative points M,
FIG. 3 is a perspective view of a combined structure of planes or curved surfaces that takes into account attributes such as three-dimensional coordinates such as N and their representative points are corners, and the same viewpoint as the viewpoint of the design image Id. .

The operation image I _{w in} FIG. 6 shows an example of a perspective view created from the measurement images I _w1 and I _w2 by the above method. The output image of the operation camera 4 can also be used as the input of the operation image creation device 18. In that case, various adjustments, that is, adjustments for changing to an image viewed from the same viewpoint as the viewpoint of the design image Id,
It is necessary to make an adjustment such that the scale is the same as that of the design image Id, an adjustment to display the representative points M, N, etc. in the three-dimensional coordinates at reasonable positions.

Design image to be superimposed and displayed on the operation image I _w
I _d is, for example, C before the target surface 1 is processed by the heavy machine 2.
The design image of FIG. 6 by the design image creation device 19 such as an AD device
It is created as a design drawing such as Id and stored in the storage device 17. Alternatively, the design drawing of the target surface 1 may be created by handwriting or by a drafting machine, and the result may be stored in the storage device 17.

At any stage during construction of the target surface 1, the design image Id is read from the design image creating device 19 or the storage device 17 to the operation display 14 of FIG. 1, and the operation image I _w is designed by the superimposing means 20. It is superimposed on Id and displayed as a superimposed image Is in FIG. By this superimposed display, it becomes possible to support the comparison between the shape of the target surface 1 and the design shape at an arbitrary time, that is, the confirmation of the difference between the current shape and the design shape during construction. Even in the case of remote construction, it is possible to compare with a design shape similar to that by striking and improve the finish quality.

Therefore, the object of the present invention is to provide a "imaging system for remote construction support capable of comparing the design shape of a work target with the current state".

[0025]

FIG. 3A shows an example in which the work support image system for remote construction of the present invention is applied to a heavy machine 2 which is a power shovel, but the application target of the system of the present invention is not limited to the power shovel. FIG. 3B shows an example of the work image I _w , the design image Id, and the superimposed image Is obtained by superimposing them on each other in this embodiment. In FIG. 3 (A), the optotype 8 is attached to the optotype rod 8a as shown in FIG. 4, and a reference point on the heavy machine 2, for example, the heavy machine 2 is attached.
The relative position of the visual target 8 to the fixed measurement camera 5 can be changed by the visual target position control device 9.

That is, as shown in FIGS. 4 (B) and 4 (C), the optotype position control device 9 is slidable on the support table 9t integrally fixed to the heavy equipment 2 and the table 9t. A three-way positioning unit having a front-rear moving plate 9a, a left-right moving plate 9b, and an up-and-down moving element 9c, and a support arm 9d fixed to the up-and-down moving element 9c. . By monitoring the operation of the positioning unit with an appropriate monitor (not shown), it is possible to monitor the relative position of the optotype rod 8a with respect to the heavy machine 2, and thus the relative position of the optotype rod 8 fixed to the optotype rod 8a. . The optotype rod 8a of FIG. 4D has n optotypes 8 at predetermined positions separated by intervals S ₁ , S ₂ , ..., It is preferable that the optotypes 8 are on the same plane. Not held in place.

[0027]

As described above in detail, the work support image system for remote construction of the present invention superimposes and displays the work image showing the current shape of the construction target surface on the design image. Has a remarkable effect.

(A) Even in remote construction, the work can proceed while comparing the current shape of the work surface with the designed target shape, so that the quality of the finished shape can be improved. (B) Since the database of the conventional mechanical design and drafting system can be used as the designed target shape, special design information for work support is not required. (C) When a CAD device is used for designing, an operation image can be drawn according to the representative point coordinates by simply modifying the CAD software.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of the present invention.

FIG. 2 is an example of a plan view of the ground leveling work.

FIG. 3 is a schematic view of an embodiment of the present invention.

FIG. 4 is an explanatory diagram of an optotype rod.

FIG. 5 is an explanatory diagram of representative points.

FIG. 6 is an explanatory diagram of a process of creating a superposition diagram.

[Explanation of symbols]

1 Target Surface 2 Duplex Machine 3 Control Room 4 Operation Camera 5 Measurement Camera 6 Image Transmitter 7 Antenna Device 8 Visual Target 9 Visual Target Position Control Device 10 Image Receiver 14 Operation Display 15 Measurement Display 16 Calculation Means 17 Storage Device 18 Operation image creating means 19 Design image creating means 20 Superimposing means 22 Satellite surveying device 24 Position information transmitting device 26 Position information receiving device 30 Reference point Id Design image I _w1 , I _w2 measurement image I _w Operation image Is superimposed image.

Claims (5)

[Claims] 1. A measurement image of an optotype with known three-dimensional coordinates and a construction target surface on which a design image has been created is photographed from different directions by a plurality of measurement cameras at predetermined positions, and the images on the plurality of measurement images are viewed. A set of two-dimensional image coordinates of a target is detected, and a set of two-dimensional coordinates on the plurality of measurement images of arbitrary points from the set of detected two-dimensional image coordinates and the known three-dimensional coordinates and a cubic of the arbitrary point. The correspondence with the original coordinates is calculated, the set of two-dimensional image coordinates of the representative point on the construction target surface in the measurement image from the predetermined position is detected, and the set of two-dimensional image coordinates of the detected representative point is detected. Obtain the three-dimensional coordinates of the representative point by the correspondence,
Create an operation image of the construction target surface viewed from the same viewpoint as the viewpoint of the design image from the three-dimensional coordinates of the obtained representative point, and perform a remote construction by superimposing the operation image of the construction target surface and the design image. Work support image system. 2. A one or more movable optotypes, a construction camera for which a design image has been created, and a measurement camera for taking a measurement image of the optotypes from different directions at predetermined positions, and a three-dimensional ground to the optotypes. Heavy equipment for remote construction having a satellite surveying device for measuring coordinates and a transmitting device for transmitting the measurement image and the coordinates; a receiving device for receiving from the transmitting device, an operation display, and measurement for displaying the measurement image Control room for remote operation of the heavy machine having a display for display; a set of two-dimensional coordinates of the specific visual target in a plurality of measurement images from different directions on the measuring display and three-dimensional of the specific visual target Calculating means for calculating the correspondence between the set of the two-dimensional coordinates of the arbitrary point and the three-dimensional coordinates of the arbitrary point from the coordinates; the two-dimensional shape of each of the plurality of representative points on the target surface in the plurality of measurement images. Image coordinates From the three-dimensional coordinates of the respective representative points from the corresponding relationship, the operation image creating means for creating an operation image of the target surface viewed from the same viewpoint as the viewpoint of the design image from the obtained three-dimensional coordinates of the representative points; A work support image system for remote construction, comprising superimposing means for superimposing the design image and the operation image on the operation display. 3. The image system according to claim 2, wherein the heavy equipment is provided with an operation camera for photographing a work surface, and the transmission device sends an output image of the operation camera to the operation image creating means to perform the operation. A work support image system for remote construction, in which an image is created based on the three-dimensional coordinates of the plurality of representative points and the output image of the operation camera. 4. The image system according to claim 2 or 3,
The two measurement cameras are provided at different predetermined positions on the heavy equipment, and the two measurement displays for displaying measurement images of the two measurement cameras are provided in the control room, and the two-dimensional coordinates are provided. A work support image system for remote construction, wherein the calculation means calculates the correspondence relationship between the group and the three-dimensional coordinates by the stereo image measuring method. 5. A work support image system for remote construction according to claim 2, 3 or 4, wherein the plurality of optotypes are attached to one movable optotype rod.