CN106909141A - Obstacle detection positioner and obstacle avoidance system - Google Patents

Abstract

The present invention provides an obstacle detection and positioning device, comprising an image acquisition unit, used to acquire the scene image ahead of the obstacle; an image processing unit, used to acquire sequence images from the scene images acquired by the image acquisition unit, and from The optical flow is extracted from the sequence images, and finally the relative distance between the obstacle and the image acquisition unit is obtained according to the optical flow; the image acquisition unit and the image processing unit are both set at the position where the scene image is acquired. The present invention also provides an obstacle avoidance system, including the above-mentioned obstacle detection and positioning device, used to obtain the relative distance between the obstacle and the image acquisition unit; an intelligent body control module, used to obtain the relative distance between the obstacle and the image acquisition unit , and accordingly plan the route to avoid obstacles to realize the obstacle avoidance function. The technical solution of the invention can realize image-based obstacle detection and positioning, and image information collection and optical flow extraction can be performed at the position of image collection.

Description

Obstacle detection and positioning device and obstacle avoidance system

technical field

The invention relates to the technical field of machine vision applications, in particular to an obstacle detection and positioning device and an obstacle avoidance system using the obstacle detection and positioning device.

Background technique

Optical flow refers to the movement of brightness or grayscale on the imaging plane in a time-varying image. When animals (especially insects) move, the surrounding environment will form an image sequence on the retina, and the continuous change of the image sequence will cause the "flow" of the brightness on the retina, so it is vividly called the movement of the image brightness on the two-dimensional imaging plane for optical flow. Relative motion is a necessary condition for the generation of optical flow. Biological research results show that insects use optical flow for navigation, flight and landing, which shows that optical flow contains rich three-dimensional motion and scene information. A large number of researches have been carried out in the field of optical flow application technology at home and abroad, mainly focusing on the direction of moving target detection and tracking, robot visual navigation and obstacle avoidance, and low-altitude terrain following flight control of aircraft.

In the research and development of the optical flow application algorithm, due to the large amount of calculation of the optical flow extraction algorithm, the sequence images collected by the camera carried by the agent are usually sent back to the ground computer through a wireless link for optical flow calculation. To a certain extent, it limits the engineering transformation of optical flow applications. Moreover, the PX4 optical flow sensor, as an emerging hardware system based on image optical flow applications, can be used together with ultrasonic ranging devices to measure the downward-looking optical flow of micro-aircraft, and assist in the realization of fixed-point flight without GPS , this technology has been applied to the Ar.drone four-rotor aircraft, and the PX4 optical flow sensor can also be used compatible with the open source flight control Pixhawk, but due to the small number of imaging units, the measurement coverage and the distance are also small, it cannot adapt to the long-distance forward vision. Application of range detection.

Contents of the invention

The object of the present invention is to provide an obstacle detection and positioning device capable of extracting optical flow at the image acquisition site and realizing obstacle detection and positioning based on the image, and an obstacle avoidance system using the obstacle detection and positioning device.

In order to solve the above problems, the present invention proposes an obstacle detection and positioning device, which includes:

an image acquisition unit, configured to acquire a scene image in front of the obstacle;

An image processing unit, configured to acquire a sequence of images from the scene image acquired by the image acquisition unit, and extract the optical flow from the sequence of images, and finally obtain the distance between the obstacle and the image acquisition unit according to the optical flow. relative distance;

The image acquisition unit is connected to the image processing unit to transmit the image information of the scene containing obstacles to the image processing unit;

Both the image acquisition unit and the image processing unit are set at the position where the scene image is acquired.

Preferably, the obstacle detection and positioning device is connected to the ground monitoring and control workstation to transmit the relative distance between the obstacle and the image acquisition unit to the ground monitoring and control workstation.

Preferably, the obstacle detection and positioning device is provided with a wireless network card, and the obstacle detection and positioning device is wirelessly connected to the ground monitoring and control workstation.

Preferably, the ground monitoring and control workstation controls the image processing unit through wireless remote login via a wireless network card.

Preferably, the obstacle detection and positioning device of the present invention further includes a mobile power supply, the mobile power supply is respectively connected to the image acquisition unit and the image processing unit to provide power, specifically, the mobile power supply is connected to the image acquisition unit and the image processing unit MicroUSB power supply can be used between the image processing units.

Preferably, the image acquisition unit includes a video camera;

The image processing unit includes a Pcduino microcomputer, a loaded Linux operating system and Opencv application layer software Opencv application layer software.

Preferably, the application layer software in the image processing unit includes:

A sequence image acquisition module, configured to acquire sequence images in the scene images acquired by the image acquisition unit;

A correction module, used to correct the sequence images collected by the sequence image collection module and filter out noise;

An optical flow estimation module, used to acquire and extract the optical flow in the sequence images corrected by the correction module;

The depth estimation module is used to obtain the relative distance between the obstacle and the image acquisition unit.

Preferably, there is a fixed time interval between each of the sequence images read by the module sequence image acquisition module, and the time interval can be set by the staff in combination with the actual situation, and preset in the Opencv application layer software.

Preferably, the noise filtering of the correction module adopts a Gaussian filtering method;

The optical flow estimation module extracts the optical flow using a binomial fitting method;

The depth estimation module is based on center of expansion (FOE) and remaining time to collision (TTC).

The present invention also provides an obstacle avoidance system, comprising:

The above-mentioned obstacle detection and positioning device is used to obtain the relative distance between the obstacle and the image acquisition unit;

The intelligent body control module is used to obtain the relative distance between the obstacle and the image acquisition unit, and accordingly plans a route to avoid the obstacle to realize the obstacle avoidance function;

The obstacle detection and positioning device is connected to the agent control module to transmit the relative distance between the acquired obstacle and the image acquisition unit to the agent control module.

Using the obstacle detection and positioning device of the present invention and the obstacle avoidance system using the obstacle detection and positioning device can collect image information and extract optical flow at the position of image collection, and finally obtain the obstacle according to the optical flow. The relative distance from the image acquisition unit realizes image-based obstacle detection and positioning. Due to the adoption of the Pcduino microcomputer, the obstacle detection and positioning device of the present invention is light in weight, small in size and strong in adaptability, and can be directly installed on robots, unmanned vehicles, and unmanned aerial vehicles for early data collection and accumulation. As a basis, realize the functions of navigation, obstacle avoidance, and target tracking, and improve the level of autonomy and intelligence of the agent based on vision; and the Opencv application layer software can be run on the Pcduino microcomputer, which is not only low in cost, easy to obtain, but also easy to operate.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a functional block diagram of the obstacle detection and positioning device of the present invention in Embodiment 1.

FIG. 2 is a functional block diagram of the obstacle avoidance system of the present invention in Embodiment 1.

FIG. 3 is a scene image acquired by the image acquisition unit in Embodiment 1.

FIG. 4 is a schematic diagram of the optical flow extracted by the image processing unit in Embodiment 1.

FIG. 5 is an explanatory diagram of the relative distance between the obstacle in FIG. 4 and the image acquisition unit.

detailed description

Embodiments of the present invention will be described below with reference to the drawings. Elements and features described in one drawing or one embodiment of the present invention may be combined with elements and features shown in one or more other drawings or embodiments. It should be noted that representation and description of components and processes that are not relevant to the present invention and known to those of ordinary skill in the art are omitted from the drawings and descriptions for the purpose of clarity.

The present invention provides an obstacle detection and positioning device, the obstacle detection and positioning device includes:

an image acquisition unit, configured to acquire a scene image in front of the obstacle;

The image processing unit is used to collect sequence images from the scene image obtained by the image acquisition unit, and extract the optical flow from the sequence images, and finally obtain the relative distance between the obstacle and the image acquisition unit according to the optical flow;

The above-mentioned image acquisition unit is connected to the image processing unit to transmit the image information of the scene containing obstacles to the image processing unit;

Both the image acquisition unit and the image processing unit are arranged at the position where the scene image is acquired.

As a preferred embodiment, the obstacle detection and positioning device can be connected to the ground monitoring and control workstation to transmit the relative distance between the obstacle and the image acquisition unit to the ground monitoring and control workstation. A wireless network card is set in the obstacle detection and positioning device, and the obstacle detection and positioning device is wirelessly connected to the ground monitoring and control workstation. At the same time, the ground monitoring and control workstation wirelessly logs in and controls the image processing unit through the wireless network card. It is also possible to set a mobile power supply in the obstacle detection and positioning device of the present invention, and connect the mobile power supply to the image acquisition unit and the image processing unit to provide power, specifically, the mobile power supply, the image acquisition unit and the image processing unit It can be powered by MicroUSB. The image acquisition unit includes a camera; the image processing unit includes a Pcduino microcomputer, a loaded Linux operating system, and Opencv application layer software Opencv application layer software. The camera can be connected with the Pcduino microcomputer through the USB interface. In order to realize the collection of image information, the extraction of optical flow and the acquisition of the relative distance between obstacles and the image acquisition unit, the Opencv application layer software of the image processing unit can be respectively set to collect the sequence in the scene image acquired by the image acquisition unit. The sequence image acquisition module of the image; the correction module used to correct the sequence images collected by the sequence image collection module and filter out the noise; the optical flow estimation module used to obtain and extract the optical flow in the sequence images corrected by the correction module ; The depth estimation module used to obtain the relative distance between the obstacle and the image acquisition unit. Specifically, as a preferred implementation, wherein the correction module uses Gaussian filtering to filter out noise; the optical flow estimation module extracts optical flow using a binomial fitting method; the depth estimation module uses the expansion center and the remaining collision time. . And the sequence images read by the module sequence image acquisition module can be selected as a fixed time interval, the time interval can be set by the staff in combination with the actual situation, and preset in the Opencv application layer software.

The present invention also provides an obstacle avoidance system, which includes:

The above-mentioned obstacle detection and positioning device is used to obtain the relative distance between the obstacle and the image acquisition unit;

The intelligent body control module is used to obtain the relative distance between the obstacle and the image acquisition unit, and accordingly plans the route to avoid the obstacle to realize the obstacle avoidance function;

The obstacle detection and positioning device is connected to the agent control module to transmit the relative distance between the acquired obstacle and the image acquisition unit to the agent control module.

Example 1:

As shown in Figure 1, as a kind of preferred embodiment, in the present embodiment, image acquisition unit adopts video camera; Image processing unit adopts Pcduino microcomputer and Opencv application layer software running on this Pcduino microcomputer. The Pcduino microcomputer of the image processing unit is wirelessly connected with the ground monitoring and control workstation through WIFI, and transmits the relative distance between the obstacle and the image acquisition unit to the ground monitoring and control workstation. In this embodiment, the mobile power supply provided in the obstacle detection and positioning device is selected as a 5V mobile power supply and is powered by MicroUSB with the Pcduino microcomputer of the image processing unit. Here, the 5V mobile power supply can also be used with the camera using MicroUSB powered by. At the same time, the camera and the Pcduino microcomputer of the image processing unit can be connected by a USB interface to transmit the collected image information to the Pcduino microcomputer. Image acquisition signal for time interval information, etc.

As shown in Figure 2, in order to realize the application of the relative distance between the obstacle and the image acquisition unit, such as depth estimation, obstacle avoidance command solution, navigation algorithm, tracking algorithm, etc., the above algorithms and corresponding The control instructions are stored and run on the Pcduino microcomputer in the form of optical flow applications. Since each application can be realized by the program stored and run on the Pcduino microcomputer, and the Pcduino microcomputer is small in size, it can be loaded on autonomous intelligent bodies such as robots, unmanned vehicles, and unmanned aerial vehicles to realize sequential image-based light Streaming intelligent body autonomous navigation, autonomous obstacle avoidance, moving target detection and tracking and other technological applications. As a high-performance mini-PC platform, Pcduino has a TTL serial port, so the result data of executing the depth estimation, obstacle avoidance command solution, navigation algorithm, etc. and the corresponding control commands can be sent to the smart body control unit.

In addition, the Linux operating system can be stored and run on the Pcduino microcomputer, which provides an environment for algorithm development, debugging, and operation, and supports various open source software. As a preferred implementation manner, in this embodiment, the obstacle detection and positioning device of the obstacle avoidance system adopts the same components, structures and connection methods as in FIG. 1 , which will not be repeated here.

If the obstacle is a passing vehicle, the camera and the Pcduino microcomputer can be selectively installed on the flyover to collect the scene image obliquely below, as shown in Figure 3, which is the scene image acquired by the image acquisition unit at this time, that is, the camera. The optical flow in two adjacent images can be estimated by the FarneBack optical flow extraction module in the Opencv application layer software, as shown in Figure 4, which is a schematic diagram of the extracted optical flow, where Figure 5 shows the obstacles and images Get the relative distance illustration map of the unit. Based on the optical flow information, a moving vehicle can be detected, and the relative distance between the vehicle and the camera can be further estimated. The closer the distance, the darker the black.

Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made hereto without departing from the spirit and scope of the invention as defined by the appended claims. Moreover, the scope of the present application is not limited to the specific embodiments of the procedures, devices, means, methods and steps described in the specification. Those of ordinary skill in the art will readily appreciate from the disclosure of the present invention that existing and future devices that perform substantially the same function or obtain substantially the same results as the corresponding embodiments described herein can be used in accordance with the present invention. The developed process, device, means, method or steps. Accordingly, the appended claims are intended to include within their scope such processes, means, means, methods or steps.

Claims (10)

1. An obstacle detection and positioning device, characterized in that, comprising: an image acquisition unit, configured to acquire a scene image in front of the obstacle; An image processing unit, configured to acquire a sequence of images from the scene image acquired by the image acquisition unit, and extract the optical flow from the sequence of images, and finally obtain the distance between the obstacle and the image acquisition unit according to the optical flow. relative distance; The image acquisition unit is connected to the image processing unit to transmit the image information of the scene containing obstacles to the image processing unit; Both the image acquisition unit and the image processing unit are set at the position where the scene image is acquired. 2. The obstacle detection and positioning device according to claim 1, characterized in that: the obstacle detection and positioning device is connected to the ground monitoring and control workstation to transmit the relative distance between the obstacle and the image acquisition unit to the Ground monitoring and control workstation. 3. The obstacle detection and positioning device according to claim 2, characterized in that: the obstacle detection and positioning device is provided with a wireless network card, and the obstacle detection and positioning device is wirelessly connected to the ground monitoring and control workstation. 4 . The obstacle detection and positioning device according to claim 3 , wherein the ground monitoring and control workstation controls the image processing unit through wireless remote login via a wireless network card. 5 . The obstacle detection and positioning device according to claim 1 , further comprising a mobile power supply, which is respectively connected to the image acquisition unit and the image processing unit to provide power. 6. The obstacle detection and positioning device according to any one of claims 1 to 5, characterized in that: the image acquisition unit includes a camera; The image processing unit includes a Pcduino microcomputer, a loaded Linux operating system and Opencv application layer software Opencv application layer software. 7. The obstacle detection and positioning device according to claim 6, characterized in that: the application layer software in the image processing unit comprises: A sequence image acquisition module, configured to acquire sequence images in the scene images acquired by the image acquisition unit; A correction module, used to correct the sequence images collected by the sequence image collection module and filter out noise; An optical flow estimation module, used to acquire and extract the optical flow in the sequence images corrected by the correction module; The depth estimation module is used to obtain the relative distance between the obstacle and the image acquisition unit. 8. The obstacle detection and positioning device according to claim 7, characterized in that: there is a fixed time interval between the sequence images read by the module sequence image acquisition module. 9. The obstacle detection and positioning device according to claim 7, characterized in that: The noise filtering of the correction module adopts a Gaussian filtering method; The optical flow estimation module extracts the optical flow using a binomial fitting method; The depth estimation module is performed according to the expansion center and the remaining collision time. 10. An obstacle avoidance system, comprising: The obstacle detection and positioning device described in claims 1 to 9 is used to obtain the relative distance between the obstacle and the image acquisition unit; The intelligent body control module is used to obtain the relative distance between the obstacle and the image acquisition unit, and accordingly plans a route to avoid the obstacle to realize the obstacle avoidance function; The obstacle detection and positioning device is connected to the agent control module to transmit the relative distance between the acquired obstacle and the image acquisition unit to the agent control module.