JP6946509B2 - Flight planning method and aircraft guidance system - Google Patents

Description

The present invention relates to a flight planning method and a flight body guidance system for autonomously flying a small unmanned aerial vehicle (UAV: Unmanned Air Vehicle).

In recent years, with the progress of UAV (Unmanned Air Vehicle: small unmanned aerial vehicle), various devices are mounted on the UAV and the required work is performed by remote control or autonomous flight of the UAV. For example, a UAV is equipped with a photogrammetric camera and a scanner, and measurements are performed from the sky to below, or in a place where no one can enter. Further, for the position measurement of the UAV itself, a GPS is mounted on the UAV, and the position of the UAV is measured by the GPS.

When the UAV is made to fly autonomously, a flight plan that defines the measurement range and flight route is created based on known information such as map data or design drawings of the structure. In addition, the UAV is flown based on the flight plan, and images of objects to be measured such as bridges and dams are acquired and measured.

In the case of structures such as bridges and dams, the surface shape is not a constant flat surface, but is inclined, curved, or has irregularities on the surface. However, the flight plan is a two-dimensional flight plan based on map information and information based on drawings, and does not consider the inclination, curvature, and unevenness of the structure. Therefore, the distance between the structure and the UAV cannot be maintained constant.

For example, when performing maintenance on a structure, it is necessary to detect minute cracks of about 0.2 mm in concrete. The resolution of the image corresponds to the distance to the subject, and in order to detect a minute crack of about 0.2 mm, it is necessary to acquire an image of the structure from a close distance of about 2 m to 10 m. However, in the conventional flight plan, the distance between the structure and the UAV cannot be maintained constant, and the shooting is performed at a predetermined distance to avoid collisions, so it is difficult to detect minute cracks, etc. was there.

Japanese Unexamined Patent Publication No. 2014-167413 Japanese Unexamined Patent Publication No. 2015-1450 Japanese Unexamined Patent Publication No. 2015-145784

The present invention creates a three-dimensional flight route that maintains a constant distance between an air vehicle and an object to be measured, and a three-dimensional flight plan creation method that guides the air vehicle based on the three-dimensional flight route. It provides an air vehicle guidance system.

The present invention is based on a flight device equipped with a measurement device that can be remotely operated, a position measurement device capable of acquiring images and capable of distance measurement, angle measurement, and tracking, and a measurement result of the position measurement device. A flying object guidance system having a ground base for controlling flight and a control device provided on the flying object or the ground base. The flying object includes a retroreflector, and the position measuring device is a non-prism. The control device has a non-prism measurement function for performing distance measurement and angle measurement, and a prism measurement function for performing distance measurement and angle measurement for the retroreflector, and the control device is a map information, a drawing, or a measurement object. A schematic flight plan having a flight range set in a plane based on an image including, and having a two-dimensional approximate flight route within the flight range and set on the image acquired by the position measuring device. Create, measure the outline flight route non-prism, calculate the three-dimensional detailed flight route based on the measurement result and the outline flight route, create the detailed flight plan including the detailed flight route, and create the detailed flight plan. And, based on the result of the prism measurement, the present invention relates to a flying object guidance system that controls the flying object so as to keep the distance between the flying device and the surface of the measurement object constant and fly.

Further, in the present invention, the measuring device is a camera unit, and the outline flight plan relates to an air vehicle guidance system including a shooting point and an overlap rate set on the outline flight route.

The present invention also relates to a flying object guidance system in which the measuring device is a laser scanner, and point cloud data is acquired by the laser scanner while the flying object is flying along the detailed flight route.

Further, the present invention relates to the flying object guidance system in which the portion of the detailed flight route that could not be measured by the position measuring device cannot be measured by non-prism is deleted.

The present invention also relates to a flying object guidance system configured such that the position measuring device emits a warning sound based on the detection of the departure of the flying device from the detailed flight route to the measurement target side. be.

Further, in the present invention, the flight device includes a GPS device, and the control of the flying object by the control device is performed by the position information of the flight device acquired by the position measuring device and the flight device acquired by the GPS device. It relates to a flying object guidance system that controls the flight of the flight device by any of the position information of.

Further, in the present invention, the flying device is provided with the measuring device supported so as to be tiltable in an arbitrary direction via a gimbal mechanism and the measuring device, which is tilted integrally with the measuring device and has a known relationship with the measuring device. The present invention relates to a flying object guidance system including the retroreflector.

The present invention is a flight plan creation method using a flight device that can be remotely operated, a control device that remotely controls the flight device, and a position measurement device that can acquire images and can perform non-prism measurement and prism measurement. The flight range in the plane is set based on the map information, the drawing, or the image including the object to be measured, and the two-dimensional approximate flight route is set on the image within the flight range and acquired by the position measuring device. The step of setting, the step of creating a rough flight plan including the rough flight route, the step of non-prism measurement of the rough flight route, the result of non-prism measurement, and the distance between the measurement object and the flight device. The present invention relates to a flight plan creation method including a step of creating a detailed flight plan including a three-dimensional detailed flight route set based on the setting of.

Further, in the present invention, the flight device has a camera unit as a measuring device, and the outline flight plan or the outline flight route includes a shooting point set on the outline flight route and an image acquired by the camera unit. It relates to a flight plan creation method including an overlap rate between adjacent images.

Furthermore, the present invention relates to a flight plan creation method in which a portion of the detailed flight route for which distance measurement data cannot be obtained as a result of non-prism measurement is removed.

According to the present invention, a flying device equipped with a measuring device and capable of remote control, a position measuring device capable of image acquisition and capable of distance measurement, angle measurement, and tracking, and flight based on the measurement results of the position measuring device. An air vehicle guidance system having a ground base for controlling the flight of a body and a control device provided on the air vehicle or the ground base. The air vehicle includes a retroreflector, and the position measuring device is a position measuring device. It has a non-prism measurement function that measures distance and angle with a non-prism and a prism measurement function that measures distance and angle with respect to the retroreflector, and the control device has map information, drawings, or measurement. Approximate flight having a flight range set in a plane based on an image including an object, within the flight range, and having a two-dimensional approximate flight route set on an image acquired by the position measuring device. A plan is created, the outline flight route is measured non-prismically, a three-dimensional detailed flight route is calculated based on the measurement result and the outline flight route, and a detailed flight plan including the detailed flight route is created, and the details are obtained. Based on the results of the flight plan and the prism measurement, the flying object is controlled so as to keep the distance between the flying device and the surface of the measuring object constant, so that the distance between the measuring object and the flying device can be determined. It can be made smaller, and the surface of the object to be measured can be measured from a close distance where minute cracks can be detected.

Further, according to the present invention, a flight plan creating method using a flight device that can be remotely operated, a control device that remotely controls the flight device, and a position measurement device that can acquire images and can perform non-prism measurement and prism measurement. The flight range in the plane is set based on the map information, the drawing, or the image including the object to be measured, and the flight range is within the flight range, and the two-dimensional outline is on the image acquired by the position measuring device. A step of setting a flight route, a step of creating a rough flight plan including the rough flight route, a step of measuring the rough flight route non-prism, a result of non-prism measurement, and between the measurement object and the flight device. Since it has a step of creating a detailed flight plan including a three-dimensional detailed flight route set based on the setting of the distance, the distance between the measurement object and the flight device can be reduced, and minute cracks can be formed. It has an excellent effect that the surface of the object to be measured can be measured from a detectable close distance.

It is a block diagram which shows the flying body guidance system which concerns on this Example. (A) is a perspective view showing a flight device according to this embodiment, and (B) is a perspective view showing an example of a azimuth angle sensor. It is sectional drawing which shows the said flight apparatus. It is a block diagram which shows the control system of the flight apparatus. It is a schematic block diagram which shows an example of the position measuring apparatus which concerns on this Example. It is a figure which shows the schematic structure of the ground base which concerns on this Example, and the relation of said flight apparatus, said position measuring apparatus, said ground base, and remote control. It is a flowchart explaining the process of making a detailed flight plan in this Example. (A) is an explanatory diagram for explaining the setting of the flight range in the present embodiment, and (B) is an explanatory diagram for explaining the setting of the approximate flight route in the present embodiment. It is explanatory drawing explaining the detailed flight route in this implementation.

Hereinafter, examples of the present invention will be described with reference to the drawings.

First, in FIG. 1, the flying object guidance system according to this embodiment will be described.

The aircraft guidance system 1 is mainly composed of one flight device (UAV) 2, a position measurement device 3, a ground base 4, and a remote control device 5. Note that FIG. 1 shows a case where a total station (TS) is used as the position measuring device 3.

The flight device 2 mainly includes a flying object 15 (described later), a shaft 6 as a support member vertically supported by the flying object 15 via a gimbal mechanism, and a camera 7 provided at the lower end of the shaft 6. A GPS device 8 provided at the upper end of the shaft 6, a prism 9 as a retroreflector provided at the lower end of the shaft 6, and an optical axis of the camera 7 provided integrally with the prism 9. It includes a direction angle sensor 10 provided in a known relationship and an air vehicle communication unit 11 that communicates with the ground base 4. The camera 7 functions as a measuring device for taking an aerial photograph or taking a measurement object for photogrammetry.

Here, a reference position is set for the flight device 2, and the relationship between the reference position and the camera 7, the GPS device 8, and the prism 9 is known. The reference position of the flight device 2 is, for example, the center position of the image sensor (not shown) of the camera 7.

The camera 7 is rotatably supported via a horizontal axis (described later), and the optical axis of the camera 7 rotates in a plane parallel to the axis of the shaft 6. The rotation range of the camera 7 includes at least a range from a position where the optical axis of the camera 7 is vertical to a horizontal position.

The shaft 6 is supported by the gimbal mechanism so that the axis of the shaft 6 is vertical. Therefore, when the optical axis of the camera 7 is vertical, the optical axis of the camera 7 and the shaft 6 are supported. It matches the axis.

The optical axis of the prism 9 is also provided so as to be parallel to the axis of the shaft 6 and set to be vertical. Further, the positional relationship between the prism 9 and the camera 7 is also known. The optical axes of the camera 7 and the prism 9 may be supported so as to be vertical, and the axial center of the shaft 6 does not necessarily have to be vertical.

The prism 9 is provided downward, and the prism 9 has an optical property of retroreflecting light incident from the entire lower range. Further, instead of the prism 9, a reflective sticker may be provided at a predetermined position on the shaft 6.

The position measured by the GPS device 8 exists on the axis of the shaft 6, and the position measured by the GPS device 8 is known to the camera 7.

The azimuth sensor 10 detects the orientation of the flight device 2. As the direction angle sensor 10, for example, there is one shown in FIG. 2 (B).

Light receiving sensors 30 (light receiving sensors 30a to 30d in FIG. 2B) are provided at positions that are equally divided around the circumference. Each of the light receiving sensors 30 can receive the ranging light or the tracking light emitted from the position measuring device 3, and determines which of the light receiving sensors 30 has detected the ranging light or the tracking light. By doing so, the direction with respect to the distance measuring light or the tracking light (that is, the direction of the flying device 2 with respect to the position measuring device 3) is detected.

The position measuring device 3 is installed at an arbitrary position, and the position measuring device 3 is leveled so as to be horizontal. The position measuring device 3 can measure a distance by non-prism measurement (measurement without using a prism or a retroreflector) and prism measurement (measurement targeting a prism or a retroreflector), and can measure a horizontal angle and a vertical angle. The angle is measurable.

In the non-prism measurement, it is possible to perform non-prism measurement in a planned range with reference to the installation position of the position measuring device 3.

Further, the position measuring device 3 has a tracking function, and in a state where the prism is being measured, the position measuring device 3 is tracking the prism 9 during the flight of the flying device 2 and the position of the prism 9. Three-dimensional coordinates (slope distance, horizontal angle, vertical right angle) with reference to the installation position of the measuring device 3 are measured. In this embodiment, the total station (TS) is used as the position measuring device 3, but the total station is limited as long as it has a tracking function and can measure an oblique distance, a horizontal angle, and a vertical right angle. It is not something that is done.

The position measuring device 3 is electrically connected to the ground base 4 by wire or wirelessly, and the measured three-dimensional coordinates of the prism 9 (that is, the flying device 2) are input to the ground base 4 as coordinate data. Will be done.

The installation position (absolute coordinates) of the position measuring device 3 can be measured by the following method.

The position measuring device 3 measures the position of the flying device 2 in flight, and the GPS device 8 measures the position coordinates of the two points of the flying device 2. Next, based on the measurement result acquired by the position measuring device 3 and the position coordinates (GPS coordinates) acquired by the GPS device 8, the installation position (GPS coordinates) of the position measuring device 3 is measured by the rear association method. NS. Further, the absolute coordinates can be obtained by converting the GPS coordinates. Therefore, if the GPS coordinates are obtained, the absolute coordinates of the position measuring device 3 can be obtained, and the result of non-prism measurement with reference to the installation position of the position measuring device 3 can also be converted into absolute coordinates.

Furthermore, the position measuring device 3 tracks the prism 9 (that is, the flying device 2), and the measured three-dimensional coordinates of the prism 9 (that is, the three-dimensional coordinates of the flying device 2) are also GPS. It can be converted to coordinates and even absolute coordinates. Therefore, by transmitting the position coordinates of the flight device 2 measured by the position measuring device 3 from the ground base 4 to the flight device 2 in real time, the flight is based on the position coordinates measured by the position measuring device 3. The device 2 can be flown. In the following description, the measurement result of the position measuring device 3 converted into the GPS coordinate system is also referred to as GPS coordinates.

The step of installing the position measuring device 3 at a known point and measuring the installation position of the position measuring device 3 by the rear association method may be omitted.

The ground base 4 is, for example, a PC, and has an arithmetic unit having an arithmetic function, a storage unit for storing data and programs, and a base communication unit. The base communication unit is capable of communicating with the position measuring device 3 and the remote control unit 5, and the remote control unit 5 is capable of wireless communication with the aircraft communication unit 11.

The flight device 2 is provided with a control device as described later. Therefore, by setting the flight plan data having data such as the flight route of the flying object 15 and the shooting distance with respect to the measurement object in the control device, the position data from the position measuring device 3 or the GPS device 8 can be used. Based on the measured position data, the flight device 2 can be made to fly autonomously.

The remote controller 5 can remotely control the flight of the flight device 2 when manually operating it. When performing manual operation, the ground base 4 sets a flight plan based on the result of non-prism measurement, and the remote control aircraft 5 has a flight range and flight so that the flight device 2 is remotely controlled according to the flight plan. Send flight control data about the route. When the flight control data regarding the flight range is transmitted from the ground base 4, the flight control signal transmitted from the remote controller 5 is limited by the flight control data, and the flight device 2 moves within the flight range. Controlled to fly. Further, the remote controller 5 can remotely control the camera 7 and the shutters of the camera 7.

Next, the flight device 2 will be further described with reference to FIGS. 2 (A), 2 (B), and 3.

The flying object 15 has a plurality of propeller frames 17 extending radially and an even number, and a propeller unit is provided at the tip of each propeller frame 17. The propeller unit includes a propeller motor 18 (propeller motors 18a and 18e in FIG. 3 in the figure) attached to the tip of the propeller frame 17 and a propeller 19 (propeller 19 in the figure) attached to the output shaft of the propeller motor 18. 19a to 19h). The propeller motor 18 rotates the propeller 19 so that the flying object 15 flies.

The flying object 15 has a hollow cylindrical main frame 21 at the center, an outer flange 22 extending outward at the upper end of the main frame 21, and an inner flange extending toward the center at the lower end. A flange 23 is provided. A circular hole 24 is formed in the central portion of the inner flange 23.

The propeller frame 17 has a rod shape, is arranged in a plane orthogonal to the axis of the main frame 21, and has a predetermined number (at least 4, preferably 8 pieces, 8 pieces in the drawing) at equal angular intervals in the horizontal direction. Propeller frames 17a to 17h)) are provided. The inner end portion of the propeller frame 17 penetrates the main frame 21 and is fixed to the outer flange 22.

The shaft 6 is provided so as to penetrate the main frame 21 up and down, the shaft 6 is supported by a gimbal 25 so as to be vertical, and the gimbal 25 is provided on the inner flange 23 via a vibration isolator member 26. Has been done.

The gimbal 25 has swing shafts 27a and 27b in two orthogonal directions, and the gimbal 25 swingably supports the shaft 6 in two orthogonal directions. The anti-vibration member 26 absorbs the vibration when the propeller motor 18 and the propeller 19 rotate so that the vibration is not transmitted to the shaft 6.

The tilt sensor 28 is provided at the lower end of the shaft 6 and detects the tilt of the shaft 6 caused by a change in the flight state of the flying object 15. Further, the inclination sensor 28 detects the angle between the vertical line and the axis of the shaft 6 when the shaft 6 is inclined with respect to the vertical direction. The detection result of the tilt sensor 28 is transmitted to the control device 35 (see FIG. 4) described later.

The azimuth sensor 10 detects the orientation of the flying object 15. The orientation of the flying object 15 is, for example, the orientation of the flying object 15 with reference to the position where the position measuring device 3 is installed. In this embodiment, the direction angle sensor 10 shown in FIG. 2B is used. Further, the azimuth sensor may be used as the azimuth sensor 10.

In this embodiment, the case where the azimuth sensor 10 is provided integrally with the prism 9 is shown. The direction angle sensor 10 is provided on the lower surface of the camera holder 32 (described later) via a support member 34. Further, the prism 9 is provided on the lower surface of the azimuth sensor 10. The azimuth sensor 10 will be outlined with reference to FIG. 2 (B).

The light receiving sensors 30a, 30b, 30c, and 30d are provided along the outer peripheral surface of the sensor case 29 formed in a cylindrical shape. The light receiving sensors 30a, 30b, 30c, 30d are arranged at positions where the circumference is divided into four equal parts, and the light receiving sensors 30a, 30b, 30c, 30d are distance measuring light or tracking emitted from the position measuring device 3. It is configured to emit a light receiving signal when it receives light. Further, the orientation of the flight device 2 with respect to the position measuring device 3 is detected by determining at which position the light receiving sensors 30a, 30b, 30c, 30d receive the ranging light or the tracking light. ..

A control box 31 is provided at the lower end of the shaft 6. The control device 35 and the IMU ((Inertial Measurement Unit): inertial measurement unit) 40 (described later) are housed inside the control box 31. The camera holder 32 is provided on the lower surface of the control box 31, and the camera 7 is provided on the camera holder 32 via a horizontal axis 33. The camera 7 is rotatable about the horizontal axis 33, and the camera holder 32 is provided with an imaging direction changing motor (not shown) for rotating the camera 7 via the horizontal axis 33. The reference posture of the camera 7 is such that the optical axis is vertical, and the imaging direction changing motor rotates the camera 7 by a required angle with respect to the vertical according to a command from the control device 35. In FIG. 2A, the optical axis of the camera 7 is horizontal, and in FIG. 3, the optical axis of the camera 7 is vertical for easy understanding.

A digital camera is used as the camera 7, and a still image can be captured and a moving image can also be captured. Further, as the image pickup element, a CCD, a CMOS sensor, or the like, which is an aggregate of pixels, is used, and the position of each pixel in the image pickup element can be specified. For example, the position of each pixel is specified by the Cartesian coordinates with the point where the optical axis of the camera of the image pickup element passes as the origin.

The GPS device 8 is provided at the upper end of the shaft 6. The center of the GPS device 8 (reference position of the GPS device 8) coincides with the axis of the shaft 6, and the optical axis of the prism 9 is parallel to the axis of the shaft 6.

The control box 31, the camera holder 32, the camera 7, the prism 9, and the like function as balance weights. In a state where no external force acts on the shaft 6, that is, in a free state, the weight balance of the control box 31, the camera holder 32, the camera 7, the prism 9, etc. is adjusted so that the shaft 6 is in a vertical state. Set.

In order to stably hold the shaft 6 in the vertical posture, balance assistance is provided so that the shaft 6 can be quickly returned to the vertical posture when the shaft 6 is suddenly tilted (when the posture of the flying object 15 is suddenly changed). Members may be provided. If the balance weight function of the control box 31, the camera holder 32, the camera 7, the prism 9, or the like can sufficiently hold the shaft 6 vertically, the balance assisting member may not be provided.

In the following example, a case where the damper spring 16 is provided as the balance assisting member will be described.

The damper spring 16 is hung between the propeller frame 17 and the shaft 6. At least three, preferably four, damper springs 16 are provided, and the damper springs 16 are preferably provided between the propeller frame 17 extending in parallel with the swing shafts 27a and 27b and the shaft 6. ..

Further, each of the four damper springs 16 exerts tension between the shaft 6 and the propeller frame 17, and the tension is balanced when the flying object 15 is in a horizontal posture (the propeller frame 17 is in a horizontal state). The shaft 6 is set to maintain a vertical state. Further, the tension and the spring constant of the damper spring 16 are set to be small, and when the flying object 15 is tilted, the shaft 6 is oriented in the vertical direction by the action of gravity.

The damper spring 16 is an urging means for urging the shaft 6 to a vertical state, and when the shaft 6 swings or vibrates, it quickly returns to the vertical state and attenuates the vibration. Is. Further, as the urging means, in addition to the damper spring 16, a torsion coil spring that rotates in the return direction when the swing shafts 27a and 27b of the gimbal 25 rotate may be used.

The control system of the flight device 2 will be described with reference to FIG.

The control device 35 and the IMU 40 are housed inside the control box 31.

The control device 35 mainly includes a control calculation unit 36, a clock signal generation unit 37, a storage unit 38, a flight imaging control unit 39, a flight control unit 41, a gyro unit 42, a motor driver unit 43, and the flight body communication unit 11. It is equipped.

The shooting of the camera 7 is controlled by the flight imaging control unit 39, and the image captured by the camera 7 is input to the flight imaging control unit 39 as image data.

A program storage unit and a data storage unit are formed in the storage unit 38. In the program storage unit, an imaging program for controlling the imaging of the camera 7, a flight control program for driving and controlling the propeller motor 18 and controlling flight based on a flight control signal described later, and a detection result of the IMU 40. A flight device position calculation program that calculates the position of the flight device 2 in real time based on the above, a return program that returns the flying object 15 to a predetermined position based on the calculated position, and the acquired data is transmitted to the ground base 4. Also, a communication program for receiving a flight command or the like from the remote controller 5, a data processing program for processing and storing data acquired by the camera 7, and moving image data acquired by the camera 7. Programs such as an image tracking program for tracking using and a flight plan creation program for creating a flight plan are stored.

In the data storage unit, flight plan data for executing autonomous flight, still image data acquired by the camera 7, moving image data, and the position of the flight device 2 measured by the GPS device 8 during flight. Data, position data of the flight device 2 measured by the position measurement device 3 transmitted from the remote controller 5, travel distance data measured by the IMU 40, position data of the flight device 2, and still image data. , The time, position data, etc. at the time of acquiring the moving image data are stored.

The flight image pickup control unit 39 controls the image pickup of the camera 7 based on the control signal emitted from the control calculation unit 36. Examples of the control include selection of a camera angle according to the object to be measured, control of imaging of the camera 7, control of acquiring a still image at predetermined time intervals while acquiring a moving image, and the like. Further, with respect to the camera 7, the shooting time is controlled or synchronously controlled based on the clock signal generated from the clock signal generation unit 37.

The direction angle sensor 10 detects the direction of the flying object 15, inputs the detection result to the control calculation unit 36, and the gyro unit 42 detects the attitude of the flying object 15 in the flight state, and outputs the detection result. Input to the control calculation unit 36.

The flight body communication unit 11 receives a flight control signal from the remote control device 5 when the flight of the flight body 15 is remotely controlled by the remote control device 5, and the flight control signal is used for the control calculation. Input to unit 36. Alternatively, it has a function of transmitting the image data captured by the camera 7 to the ground base 4 on the ground side together with the time of capture.

The control calculation unit 36 converts the position coordinates measured by the position measuring device 3 into GPS coordinates and acquires them as GPS coordinates of the flight device 2. Further, the control calculation unit 36 acquires the GPS coordinates of the flight device 2 measured by the GPS device 8. The flight control signal is calculated based on the obtained GPS coordinates and the flight command transmitted from the remote controller 5, or the flight control signal is calculated based on the flight plan data stored in the storage unit 38 and the GPS coordinates. Then, it is output to the flight control unit 41.

As for which of the GPS coordinates obtained based on the measurement result of the position measuring device 3 and the GPS coordinates measured by the GPS device 8 is used, the GPS coordinates obtained in principle are used. For example, if there is an obstacle between the flight device 2 and the position measurement device 3 and the position measurement device 3 cannot track the flight device 2, the position data from the position measurement device 3 disappears. Therefore, the GPS coordinates measured by the GPS device 8 are used. Further, the position data measured by the IMU 40 is updated in real time or at predetermined time intervals based on the position data acquired by the GPS device 8 or the position measuring device 3.

Further, in an environment where radio waves from artificial satellites are blocked in a building or the like, GPS coordinates obtained based on the measurement result of the position measuring device 3 are used. The absolute coordinates obtained from the GPS coordinates may be used as the position information for flying the flight device 2.

Further, when the flight device 2 is located under a bridge or the like where radio waves from the artificial satellite are blocked, and there is an obstacle between the flight device 2 and the position measurement device 3, that is, the position measurement. GPS coordinates may not be obtained from either the device 3 or the GPS device 8. In this case, the position where the acquisition of GPS coordinates is interrupted, for example, the moving distance and the moving direction (that is, the current position of the flying device 2) from the position where the position measuring device 3 cannot track the flying device 2. The IMU 40 measures, and based on the measurement result, the control calculation unit 36 outputs a flight control signal for returning the flight device 2 to a position where GPS coordinates can be acquired. Here, the position information of the flight device 2 until it is restored is acquired by the IMU 40.

When both the measurement result from the position measuring device 3 and the measurement result from the GPS device 8 are obtained, the priority to be used may be determined in advance. Since the measurement accuracy is better in the position measuring device 3, when the accuracy is prioritized, it is preferable to give priority to the measurement result by the position measuring device 3.

Further, the control calculation unit 36 executes the control necessary for acquiring an image based on the required program stored in the storage unit 38.

When a flight control signal is input from the control calculation unit 36, the flight control unit 41 drives the propeller motors 18a to 18h to a required state via the motor driver unit 43 based on the flight control signal. ..

Next, the position measuring device 3 will be described with reference to FIG.

The position measuring device 3 mainly includes a measuring control device 45, a camera unit 46 (see FIG. 1), a distance measuring unit 47, a horizontal angle detector 48, a vertical right angle detector 49, a horizontal rotation drive unit 51, and a vertical rotation drive unit. It includes 52, a display unit 53, an operation unit 54, and the like.

A digital camera is used as the camera unit 46, and a still image can be captured and a moving image can also be captured. Further, as the image pickup element, a CCD, a CMOS sensor, or the like, which is an aggregate of pixels, is used, and the position of each pixel in the image pickup element can be specified. Further, the camera unit 46 is a camera capable of zooming by optical or digital processing or both optical and digital processing. A measurement object is detected from a moving image or a still image captured by the camera unit 46, and the measurement object is collimated (image tracking).

The distance measuring unit 47 emits distance measuring light through the camera unit 46, and further receives reflected light from the measurement object via the camera unit 46 to perform distance measuring. Further, the distance measuring unit 47 has three modes as measurement modes: a non-prism measurement mode, a prism measurement mode, and a tracking measurement mode for tracking the measurement object (prism) while performing prism measurement. Either of them can measure the distance to the object to be measured. In the tracking measurement mode, in addition to the distance measuring light, the tracking light is emitted via the camera unit 46.

The horizontal angle detector 48 detects the horizontal angle in the collimation direction of the camera unit 46. Further, the vertical angle detector 49 detects the vertical angle in the collimation direction of the camera unit 46. The detection results of the horizontal angle detector 48 and the vertical angle detector 49 are input to the measurement control device 45.

The display unit 53 is, for example, a touch panel, and the collimation position can be adjusted by sliding a finger while touching the display unit 53. Further, the operation unit 54 can perform various operations such as changing the measurement mode, setting the measurement conditions, and finely adjusting the collimation position. Further, the display unit 53 can also serve as the function of the operation unit 54.

The measurement control device 45 mainly includes an arithmetic processing unit 55, an image pickup control unit 56, a distance measurement control unit 57, a position measurement storage unit 58, a position measurement communication unit 59, a tracking control unit 60, a motor drive control unit 61, and image processing. It has a part 62 and the like.

The image pickup control unit 56 sets the magnification of the camera unit 46, the timing of shooting, and the like based on a command from the arithmetic processing unit 55. Further, the image pickup control unit 56 controls the camera unit 46 according to a set magnification, shooting timing, and the like.

Based on the measurement mode selection command from the arithmetic processing unit 55, the distance measuring control unit 57 executes measurement in any of the non-prism measurement mode, the prism measurement mode, and the tracking measurement mode. To decide. Further, the distance measuring control unit 57 controls the measurement by the distance measuring unit 47 according to the determined measurement mode. Here, in the non-prism measurement mode, the position measuring device 3 executes measurement with a structure such as a bridge or a dam as a measurement object. In the tracking measurement mode, the object to be measured is the prism 9, and the position of the flight device 2 is measured while tracking the flight device 2.

The position measurement storage unit 58 is provided with a measurement program for performing distance measurement in each measurement mode of non-prism measurement mode, prism measurement mode, and tracking measurement mode, a tracking program that receives tracking light and performs tracking, and image processing. A program such as an image tracking program for tracking, a communication program for communicating with the flight device 2 and the ground base 4, and the like are stored. Further, the position measurement storage unit 58 stores the measurement results (distance measurement, angle measurement) of the measurement object and the image acquired by the camera unit 46.

The position measurement communication unit 59 transmits the result of measuring the measurement object (prism 9) in the tracking measurement mode (oblique distance, vertical angle, horizontal angle of the prism 9) to the ground base 4 in real time. The prism tracking and the image tracking for detecting and tracking the measurement object from the captured image are performed in parallel at the same time, and the prism tracking is preferentially executed.

The tracking control unit 60 calculates the difference between the center of the image sensor and the light receiving position of the prism 9 based on the light receiving position on the image sensor when the tracking light reflected by the prism 9 is received. Further, the tracking control unit 60 transmits a control signal to the motor drive control unit 61 so that the deviation between the center of the image sensor and the light receiving position of the prism 9 is 0 based on the calculation result.

The motor drive control unit 61 causes the camera unit 46 to collate the object to be measured, or causes the camera unit 46 to track the object to be measured based on a control signal from the tracking control unit 60. The horizontal rotation drive unit 51 and the vertical rotation drive unit 52 are controlled to rotate the camera unit 46 in the vertical direction or the horizontal direction.

Further, the image processing unit 62 performs predetermined image processing such as extracting feature points and edges from the image acquired by the camera unit 46, for example. An object to be measured is detected by image processing, and the direction angle of the object to be measured is determined based on the position of the object to be measured in the image pickup element and the detection results of the horizontal angle detector 48 and the vertical angle detector 49. It is calculated. When image tracking is performed, this calculated azimuth is used.

FIG. 6 is a diagram showing a schematic configuration of the ground base 4, and the relationship between the flight device 2, the position measuring device 3, the ground base 4, and the remote control device 5.

The ground base 4 has a control device 63 having a calculation function, a base storage unit 64, and a base communication unit 65.

The control device 63 has a clock signal generation unit (not shown). The control device 63 associates image data, shutter time data, and coordinate data received via the remote controller 5 with clock signals, processes them as time-series data based on the clock signals, and processes the base storage unit 64. Save to. Further, the control device 63 creates a schematic flight plan (described later), and creates detailed flight plan data (described later) based on the schematic flight plan data.

In the base storage unit 64, a flight range, a flight route, and the like are set based on map information obtained from the Internet or the like or an image acquired by the camera unit 46, and a rough flight plan is created. Based on the program and the measurement result of the flight range obtained by the position measuring device 3, the outline flight plan is modified and the detailed flight plan is created. The flight of the flight device 2 is performed based on the detailed flight plan. A flight control program for creating flight control data for control, a communication program for data communication between the remote controller 5 and the position measuring device 3, and the flight at two or more positions transmitted from the flight device 2. A program that calculates the GPS coordinates of the installation position of the position measuring device 3 based on the GPS coordinates of the device 2, and the measurement result of the position measuring device 3 (oblique distance, vertical angle, horizontal angle of the prism 9) is obtained from the position. Various programs such as a program for converting to GPS coordinates based on the GPS coordinates of the installation position of the measuring device 3 are stored.

Regarding the work of converting the measurement result of the position measuring device 3 into GPS coordinates based on the GPS coordinates of the installation position of the position measuring device 3, the measurement result of the position measuring device 3 is transmitted to the flight device 2 as it is. Then, it may be executed by the control device 35 of the flight device 2.

Further, various data such as an image acquired by the flight device 2, measurement data (coordinate data) measured by the position measurement device 3, time when the image was acquired, position coordinates, detailed flight plan data, and the like are stored in the base storage unit. It is stored in 64.

The base communication unit 65 performs wired communication or wireless communication between the ground base 4 and the remote control device 5.

The detailed flight plan data is transmitted to the flight device 2 via the remote controller 5 or the base communication unit 65, and the detailed flight plan data is stored in the storage unit 38. The flight control unit 41 makes the flight device 2 autonomously fly based on the detailed flight plan data. Alternatively, the detailed flight plan data is stored in the base storage unit 64 of the ground base 4, and the flight control signal created based on the detailed flight plan is transmitted to the flight device 2, so that the flight device 2 can perform the detailed flight plan data. It may fly autonomously.

When inspecting structures such as bridges and dams, it is necessary to detect minute cracks of, for example, about 0.2 mm. Therefore, in order to detect the crack from the image taken by the flight device 2, the flight device 2 is flown in a state of being close to the object to be measured, and the image is set to a shooting distance (offset) of about 2 m to 10 m. Need to get.

First, the measurement range is set based on the existing map information, photographs, blueprints, etc., and the flight route is set on the map. Further, in setting the flight route, for example, when taking an aerial photograph, the overlap rate, the shooting point, and the like are taken into consideration. As a result, a flight plan (outline flight plan) including flight conditions, shooting conditions, etc. is created. The outline flight plan is based on two-dimensional information such as a map, and is two-dimensional flight plan data, and does not correspond to unevenness, inclination, and curvature of the surface of the object to be measured. Therefore, when the flight device 2 is made to fly autonomously, there is a risk that cracks cannot be detected due to fluctuations in the shooting distance. Further, when the flight device 2 is simply flown in a state of being close to the measurement object, there is a possibility that the flight device 2 and the measurement object come into contact with each other.

In order to fly the flight device 2 while maintaining an appropriate shooting distance between the flight device 2 and the object to be measured, a three-dimensional flight plan corresponding to the unevenness of the surface of the object to be measured (detailed flight plan). Need to be created.

Hereinafter, referring to the flowchart of FIG. 7 and FIGS. 8 (A), 8 (B), and 9, a detailed flight plan creation process for flying the flight device 2 in a state of being close to the measurement object. Will be described. The detailed flight plan is prepared in advance prior to the inspection work of structures such as bridges and dams.

STEP: 01 First, a structure such as a bridge or a dam is set as a measurement target, and a measurement point required for setting the measurement range 68 is determined based on a drawing or a photograph. As the measurement point, as shown in FIG. 8A, there is a corner point 67 or the like of the measurement object 66 (in the figure, the upper part of the pier), and a closed space is formed by the measurement points.

The position measuring device 3 is installed at a predetermined position, for example, a known point with respect to the measuring object 66. The measurement points determined by the position measuring device 3 are measured in the non-prism measurement mode. The measurement range 68 is set based on the measurement result, a photograph, and known map information. A flight range for flying the flight device 2 is set based on the measurement range 68. The flight range is set so as to sufficiently cover the measurement range 68, and is preferably set to be larger than the measurement range 68.

In this embodiment, as shown in FIG. 8A, the upper part of the pier is the measurement object 66, and the corner point 67 of the measurement object 66 is defined from the photograph acquired by the camera unit 46. The portion that is detected and surrounded by the corner points 67 may be set as the measurement range 68. Further, FIG. 8A shows a case where the measurement range 68 and the flight range are set to be the same.

STEP: 02 Next, the position measuring device 3 sets a rough flight route 69 (see FIG. 8B) within the measuring range 68. The approximate flight route 69 is set in consideration of the overlap ratio between adjacent images in the traveling direction and between the adjacent approximate flight routes 69, and the shooting distance with respect to the measurement object 66. In addition, a shooting position (shooting point) and the like are set on the rough flight route 69, and a rough flight plan is created. The approximate flight route 69 may be transferred on the captured image or directly set on the captured image. Further, the outline flight plan may be created manually via the operation unit 54, or may be automatically created based on the program stored in the position measurement storage unit 58.

Here, the shooting position by the flight device 2 may be set according to the moving speed of the flight device 2 and the timing of shooting using an interval timer or the like, or is measured by the position measurement device 3 or the GPS device 8. It may be set using the coordinate data.

STEP: 03 After creating the rough flight plan, the rough flight route 69 set by the position measuring device 3 is scanned and measured in the non-prism measurement mode. By scanning and measuring the approximate flight route 69, the three-dimensional coordinates (GPS coordinates) of the surface of the measurement object 66 on the approximate flight route 69 are measured. Further, the unevenness, curvature, and inclination on the approximate flight route 69 can be measured based on the three-dimensional coordinates. Further, the portion where the measured value was not obtained in the process of scanning measurement of the approximate flight route 69 by the position measuring device 3, that is, the portion of the approximate flight route 69 which is a blind spot due to an obstacle (blind spot portion). To detect.

STEP: 04 The measurement control device 45 modifies the rough flight plan based on the scan result of the rough flight route 69 by the position measuring device 3. That is, the proximity to the measurement object 66 so that the distance between the flight device 2 and the surface of the measurement object 66 is constant corresponding to the unevenness, curvature, and inclination of the surface (measurement surface) of the measurement object 66. Calculate the detailed flight route including the movement in the separation direction. Further, the measurement control device 45 deletes the blind spot portion from the rough flight plan and creates the detailed flight route without the blind spot portion.

The shooting point setting, the overlap rate setting, and the distance setting between the measurement object 66 and the flight device 2 may be set based on the detailed flight route.

As described above, the detailed flight route including the movement in the proximity and separation direction with respect to the measurement object 66 is created, and the detailed flight route in which the blind spot portion is deleted is created. As a result, as shown in FIG. 9, a three-dimensional detailed flight route 71 that maintains a constant distance from the surface of the measurement object 66 is created. Further, the outline flight plan is revised based on the detailed flight route 71. For example, the position of the shooting point may be changed, the number of shooting points may be increased or decreased, the orientation of the flight device 2 at the shooting point may be corrected, and the like. A detailed flight plan is created by modifying the outline flight plan.

When the radio wave from the satellite can be received, the flight device 2 autonomously flies based on the detailed flight route 71 set based on the position information acquired by the GPS device 8.

Next, a case where the flight device 2 is autonomously flown based on a detailed flight plan in a place where radio waves from satellites cannot be received will be described. The detailed flight plan is set in the flight device 2 in advance.

With the flight device 2 tracked by the position measuring device 3, the control calculation unit 36 reads the detailed flight route 71 from the detailed flight plan stored in the storage unit 38, and further, the position measuring device 3 Sends the measurement result to the flight device 2. The flight device 2 acquires the current position information (GPS coordinates) of the flight device 2 by receiving the measurement result of the position measurement device 3.

In addition, the coordinates of the position where the position measuring device 3 is installed are unknown (when it is not a known point), the flying device 2 is flown in a space where radio waves from satellites can be received, and the GPS device 8 makes two GPS points. Get the coordinates. Further, by measuring the position of the flight device 2 at each of the two points with the position measuring device 3, the GPS coordinates and the absolute coordinates of the installation position of the position measuring device 3 can be obtained by the rear association method.

Next, the detailed flight route 71 is compared with the current position of the flight device 2, and a flight control signal is issued to the flight control unit 41 so that the current position moves on the detailed flight route 71, and the flight control is performed. The unit 41 controls the drive of the propeller motor 18 via the motor driver unit 43 based on the flight control signal.

Next, the shooting point (target position) is read from the storage unit 38, and the control calculation unit 36 determines whether or not the target position and the current position match. When the deviation becomes 0 or the allowable range, the required work such as shooting by the camera 7 is executed. When the required work is completed, the next shooting point (target position) is read from the detailed flight plan, and the flight of the flight device 2 is controlled so that the target position and the current position are sequentially matched.

The above process is repeated until the flight device 2 finishes flying the detailed flight route 71 of the detailed flight plan, and when the scheduled work is completed, the flight device 2 returns to the original standby position.

During the autonomous flight of the flight device 2, the flight device 2 may deviate from the detailed flight route 71 due to the influence of wind or the like. In this case, particularly when the flight device 2 approaches the measurement object 66, the flight device 2 is moved onto the detailed flight route 71 at an early stage in order to avoid contact between the flight device 2 and the measurement object 66. It is necessary to restore.

In this embodiment, when the position measuring device 3 detects that the flight device 2 has left the detailed flight route 71 toward the measurement target 66 side, the position measuring device 3 emits a warning sound. .. When the warning sound is emitted, the operator causes the ground base 4 to execute a return process for returning the flight device 2 to the detailed flight route 71. The return process includes, for example, returning the flight device 2 to a point where it first deviates on the detailed flight route 71.

By executing the return process, contact between the flight device 2 and the measurement object 66 is prevented.

It should be noted that the position measuring device 3 moves to the ground base 4 when the flight device 2 approaches the measurement object 66, instead of causing the operator to execute the return process to the ground base 4 based on the warning sound. A warning signal may be issued to the ground base 4, and the ground base 4 may automatically execute the return process based on the warning signal.

As described above, in the present embodiment, the two-dimensional outline flight route 69 of the outline flight plan created based on map information, photographs, etc. is scanned and measured by the position measuring device 3, and the outline flight route 69 is used. The unevenness, curvature, and inclination of the surface (measurement surface) of the object to be measured 66 are measured, and the three-dimensional detailed flight route 71 including movement in the proximity and separation directions corresponding to the measurement surface is created.

Therefore, the distance (offset) between the measurement surface and the flight device 2 can be made constant regardless of the unevenness, curvature, and inclination of the measurement surface of the measurement object 66, so that the flight device 2 can be measured. It is possible to fly in a state of being close to the surface of the object 66, and it is possible to take an image of the surface of the object 66 to be measured from a close distance where minute cracks can be detected. Thereby, the crack detection accuracy can be improved.

Further, in the detailed flight plan, when the approximate flight route 69 is scanned and measured by the position measuring device 3, the portion where the measurement object 66 cannot be scanned, that is, the portion which is a blind spot due to an obstacle or the like is detected. Then, by deleting the portion of the detailed flight route 71 corresponding to the blind spot portion from the detailed flight plan, the detailed flight route 71 having no blind spot portion is created.

Therefore, during the autonomous flight of the flight device 2, the tracking by the position measuring device 3 is not interrupted, so that it is possible to avoid a state in which the position information cannot be obtained by the position measuring device 3, and the flight device 2 is stable and reliable. Can perform autonomous flight.

In this embodiment, as the measuring device, the camera 7 is mounted on the flying device 2, and the camera 7 acquires an image of the object to be measured. However, the flying device 2 is equipped with the camera 7. Other measuring devices may be mounted instead. For example, a laser scanner may be mounted on the flight device 2 as a measurement device, and the laser scanner may acquire point cloud data of a measurement target while the flight device 2 is flying along the detailed flight route 71. good. Alternatively, the measuring device may be a spectrum camera for investigating the geology and the growth state of the crop.

Further, it goes without saying that the photogrammetry of the measurement object 66 may be performed based on the image taken by the camera 7 of the flight device 2. In this case, the camera 7 functions as a measuring device.

1 Air vehicle guidance system 2 Air vehicle 3 Position measurement device 4 Ground base 5 Remote control device 7 Camera 8 GPS device 9 Prism 10 Directional angle sensor 11 Air vehicle communication unit 15 Air vehicle 30 Light receiving sensor 36 Control calculation unit 38 Storage unit 39 Flight Imaging control unit 41 Flight control unit 45 Measurement control device 46 Camera unit 47 Distance measurement unit 56 Imaging control unit 57 Distance measurement control unit 58 Position measurement storage unit 59 Position measurement communication unit 64 Base storage unit 65 Base communication unit 66 Measurement target 69 Outline flight route 71 Detailed flight route

Claims (8)

Based on a remotely controllable flying device equipped with a measuring device and a flying object, a position measuring device capable of acquiring images and capable of distance measurement, angle measurement, and tracking, and measurement results of the position measuring device. a flight vehicle guidance system having a terrestrial base for controlling the flight of the flying object, and a control device provided in the aircraft, or the ground base, the flying body comprises a retro-reflector, the position measurement The device has a non-prism measurement function for performing distance measurement and angle measurement with a non-prism, and a prism measurement function for performing distance measurement and angle measurement for the retroreflector. Alternatively , it has a flight range set in a plane based on an image including a measurement object measured by the measuring device, is within the flight range, and is set on an image acquired by the position measuring device. A rough flight plan with a dimensional rough flight route was created, the rough flight route was measured non-prismically, and based on the measurement results and the rough flight route, the part that could not be non-prism measured by the position measuring device was deleted. A three-dimensional detailed flight route is calculated, a detailed flight plan including the detailed flight route is created, and based on the result of the detailed flight plan and the prism measurement, between the flight device and the surface of the measurement object. A flying object guidance system that controls the flying object so that it keeps flying at a constant level. The flying object guidance system according to claim 1, wherein the measuring device is a camera unit, and the schematic flight plan includes a shooting point and an overlap rate set on the schematic flight route. The flying object guidance system according to claim 1, wherein the measuring device is a laser scanner, and point cloud data is acquired by the laser scanner while the flying object is flying along the detailed flight route. The position measuring device according to any one of claims 1 to 3 , which is configured to emit a warning sound based on the detection of the departure of the flight device from the detailed flight route to the measurement target side. The aircraft guidance system described. The flight device includes a GPS device, and the control of the flying object by the control device is performed by either the position information of the flight device acquired by the position measuring device or the position information of the flight device acquired by the GPS device. The flying object guidance system according to any one of claims 1 to 4 , which controls the flight of the flight device. The flight device includes the measuring device supported so as to be tiltable in an arbitrary direction via a gimbal mechanism, and the retroreflector provided in a known relationship with the measuring device, which is tilted integrally with the measuring device. The air vehicle guidance system according to any one of claims 1 to 5. A flight device capable of remote operation and provided with a flying object and a measuring device, a control device for remotely operating the flying device, and a position measuring device capable of image acquisition and non-prism measurement and prism measurement were used. A flight planning method in which a flight range in a plane is set based on map information, drawings, or an image including a measurement object measured by the measuring device , and the flight range is within the flight range and the position is measured. A step of setting a two-dimensional rough flight route on an image acquired by the device, a step of creating a rough flight plan including the rough flight route, a step of non-prism measurement of the rough flight route, and a non-prism measurement. A detailed flight plan including a three-dimensional detailed flight route set so that the portion of the non-prism measurement result for which distance measurement data cannot be obtained is deleted based on the result and the setting of the distance between the measurement object and the flight device. A flight planning method that has a process of creating. The measuring device is a camera unit , and the outline flight plan or the outline flight route has an overlap rate between a shooting point set on the outline flight route and adjacent images in an image acquired by the camera unit. The flight plan creation method according to claim 7 , including the above.

Images