CN110091875A - Deep learning type intelligent driving context aware systems based on Internet of Things - Google Patents

Abstract

The invention discloses a deep learning intelligent driving environment perception system based on the Internet of Things, which belongs to the field of intelligent perception systems. The deep learning intelligent driving environment perception system based on the Internet of Things includes a perception system and an intelligent driving vehicle. The perception system includes a perception layer and an intelligent driving environment. Cognitive layer, intelligent driving vehicles are equipped with a decision-making layer and a control layer, the perception layer is used for data collection, and the decision-making layer is used to process information and route planning from the cognitive layer, process them with algorithms, and output adjustments to the control layer Instructions for vehicle speed and direction, the control layer receives instructions from the decision-making layer, and controls the vehicle's brakes, accelerator and gear position. This solution improves the implementation of unmanned driving, and uses the perception system to collect all moving and stationary obstacles on the road , to send the data information of these obstacles to all intelligent driving vehicles driving on the road, while improving the safety and stability of unmanned driving, it reduces technical difficulty and production costs.

Description

Deep Learning Intelligent Driving Environment Perception System Based on Internet of Things

technical field

The invention relates to the field of intelligent perception systems, more specifically, to a deep learning type intelligent driving environment perception system based on the Internet of Things.

Background technique

As the future research direction of automobiles, unmanned driving has a profound impact on the automobile industry and even the transportation industry. The advent of unmanned vehicles will liberate human hands, reduce the frequency of traffic accidents, and ensure people's safety. . At the same time, with the breakthrough and continuous advancement of core technologies such as artificial intelligence and sensor detection, unmanned driving will become more intelligent, and at the same time, it will be able to realize the industrialization of unmanned vehicles.

Unmanned driving technology, especially the Internet and non-traditional auto companies joining this field, has made this technology fall into a "red sea" before it is widely used. But the question is, can this kind of intelligent robot-like driverless car improve and completely improve people's ability to travel by car like the invention of a car to replace a horse-drawn carriage? The main reason for the poor road traffic conditions in China's large and medium-sized cities is that there are too many vehicles. Moreover, under the conditions of our country, accidents and congestion caused by human factors account for a considerable proportion. A large number of traffic accidents such as rear-end collisions and contact scratches can be summarized as driving dependent on the driver. "Independent Behavior Ability" mode, that is, discovery-judgment-action within the visual range. So there will be factors such as untimely discovery, untimely response, and insufficient reaction time to cause accidents. However, in the intelligent unmanned driving mode that still continues the artificial "individual independent behavior ability", of course, the above-mentioned problems still exist, and they have not been fundamentally resolved. In theory, the possibility of the above-mentioned accidents still exists. At the same time, the two main problems currently restricting the mass production of unmanned vehicles are technical difficulty and cost. cost.

Contents of the invention

1. Technical problems to be solved

In view of the problems existing in the prior art, the purpose of the present invention is to provide a deep learning type intelligent driving environment perception system based on the Internet of Things, which improves the implementation of unmanned driving, and utilizes the perception system to collect all moving and stationary objects on the road. Obstacles, the data information of these obstacles is sent to all intelligent driving vehicles driving on the road, which improves the safety and stability of unmanned driving while reducing technical difficulty and production costs.

2. Technical solution

In order to solve the above problems, the present invention adopts the following technical solutions.

The deep learning type intelligent driving environment perception system based on the Internet of Things includes a perception system and an intelligent driving vehicle. The perception system includes a perception layer and a cognitive layer. The intelligent driving vehicle is provided with a decision-making layer and a control layer. The perception layer is used for data collection, and the perception layer includes radar unit, inertial navigation unit, positioning unit and camera unit, the cognitive layer is used for data analysis, and the decision-making layer is used for information and route planning from the cognitive layer , use the algorithm to process, and output the instructions to adjust the speed and direction of the vehicle to the control layer, the control layer receives the instructions from the decision-making layer, and controls the brakes, accelerator and gear of the vehicle. This solution improves the implementation of unmanned driving , use the perception system to collect all moving and stationary obstacles on the road, and send the data information of these obstacles to all intelligent driving vehicles driving on the road, improving the safety and stability of unmanned driving while reducing technical difficulty and production costs.

Further, the radar unit, inertial navigation unit, positioning unit and camera unit can be installed on the street lamps on both sides of the road, and the original street lamp equipment is used to install and design the perception system without adding additional large-scale hardware facilities, which can save A large number of hardware configurations save costs and do not take up space. The cognitive layer includes the analysis of pedestrians, vehicles, traffic objects, traffic signs and lane lines.

Further, the radar unit includes a laser radar and a millimeter-wave radar, and the millimeter-wave radar is a device for measuring the distance, angle and relative speed of vehicles around the car by sending and receiving radio waves. Currently, it is widely used as vehicle radar. It is not easily affected by bad weather such as fog, rain and snow, as well as dust and dirt, and can detect vehicles stably. In this system, the millimeter-wave radar is based on a multi-target detection algorithm to detect moving obstacles on the driveway and sidewalk in a fixed area The distance and speed of the object.

Further, the laser radar is forbidden, and the environment of its scanning is fixed. The laser radar first obtains the environmental data and stores them in the computer in the form of an array, preprocesses the acquired environmental data, and removes information such as trees and the ground. The distance information and reflection intensity information of the radar are simultaneously processed by the environmental data segmentation and clustering of the non-planar algorithm to extract the rectangular outline features of the obstacles. The Kalman filter algorithm is used to continuously predict and track dynamic obstacles.

Further, wireless communication is used between the perception system and the intelligent driving vehicle, and the perception system sends the identified data information to all nearby intelligent driving vehicles, so that each vehicle can clearly control all obstacles, other vehicles and other vehicles around it. The direction and speed of travel of the vehicle.

Further, the sensor data fusion of the radar unit, the inertial navigation unit, the positioning unit and the camera unit includes space fusion, time fusion and sensor data fusion algorithms to establish an accurate laser radar coordinate system, a three-dimensional world coordinate system, and a millimeter wave coordinate system , is the key to realize the spatial fusion of multi-sensor data. Spatial fusion of laser radar and millimeter-wave radar is to convert the measured values of different sensor coordinate systems into the same coordinate system; the information of laser radar and millimeter-wave radar needs to be fused in space, and the sensor also needs to collect data synchronously in time , to achieve time fusion. The sampling frequency of the two sensors is different. In order to ensure the reliability of the data, based on the low-sampling rate sensor, the low-frequency sensor collects a frame of image and selects the data cached on the high-frequency sensor for a frame to complete a joint sampling. Frame radar and visual fusion data, thus ensuring the time synchronization of millimeter-wave radar data and camera data; the core key of sensor data fusion is to adopt a suitable fusion algorithm, and the sensor data fusion algorithm of this system adopts extended Kalmar filter algorithm .

Further, the radar unit, inertial navigation unit, positioning unit and camera unit are installed on adjacent street lamps at a fixed distance, and the positioning design is rationalized and standardized.

Further, the radar unit, inertial navigation unit, positioning unit and camera unit on the adjacent street lamps are connected in parallel. When a single power-consuming component is damaged, it will not affect the normal operation of other power-consuming components, ensuring that the entire The stability of system operation.

3. Beneficial effect

Compared with the prior art, the present invention has the advantages of:

(1) This solution improves the realization of unmanned driving, uses the perception system to collect all moving and stationary obstacles on the road, and sends the data information of these obstacles to all intelligent driving vehicles driving on the road. The safety and stability of unmanned driving are reduced while reducing technical difficulty and production costs.

(2) The radar unit, inertial navigation unit, positioning unit and camera unit can be installed on the street lamps on both sides of the road, and the original street lamp equipment is used for the installation and design of the perception system, without additional large-scale hardware facilities, which can save a lot of hardware Configuration, cost saving, and does not take up space. The cognitive layer includes the analysis of pedestrians, vehicles, traffic objects, traffic signs and lane lines.

(3) The radar unit includes lidar and millimeter-wave radar. The millimeter-wave radar is a device that measures the distance, angle and relative speed of vehicles around the car by sending and receiving radio waves. Currently, it is widely used as vehicle radar. It is not easily affected by bad weather such as fog, rain and snow, as well as dust and dirt, and can detect vehicles stably. In this system, the millimeter-wave radar is based on a multi-target detection algorithm to detect moving obstacles on the driveway and sidewalk in a fixed area The distance and speed of the object.

(4) The laser radar is prohibited, and the scanning environment is fixed. The laser radar first obtains the environmental data and stores them in the computer in the form of an array, preprocesses the acquired environmental data, and removes information such as trees and the ground. The distance information and reflection intensity information are simultaneously processed by the non-planar algorithm of environmental data segmentation and clustering to extract the circumscribed rectangular outline features of obstacles. The laser radar uses the multi-hypothesis tracking model algorithm to perform data association on the obstacle information of two consecutive frames, and uses Carl The Mann filtering algorithm continuously predicts and tracks dynamic obstacles.

(5) Wireless communication is used between the perception system and the intelligent driving vehicle, and the perception system sends the identified data information to all nearby intelligent driving vehicles, so that each vehicle can clearly control all obstacles around it, other vehicles and other vehicles. direction of travel and speed of travel.

(6) The sensor data fusion of radar unit, inertial navigation unit, positioning unit and camera unit includes space fusion, time fusion and sensor data fusion algorithm to establish accurate lidar coordinate system, three-dimensional world coordinate system and millimeter wave coordinate system. It is the key to realize the spatial fusion of multi-sensor data. Spatial fusion of laser radar and millimeter-wave radar is to convert the measured values of different sensor coordinate systems into the same coordinate system; the information of laser radar and millimeter-wave radar needs to be fused in space, and the sensor also needs to collect data synchronously in time , to achieve time fusion. The sampling frequency of the two sensors is different. In order to ensure the reliability of the data, based on the low-sampling rate sensor, the low-frequency sensor collects a frame of image and selects the data cached on the high-frequency sensor for a frame to complete a joint sampling. Frame radar and visual fusion data, thus ensuring the time synchronization of millimeter-wave radar data and camera data; the core key of sensor data fusion is to adopt a suitable fusion algorithm, and the sensor data fusion algorithm of this system adopts extended Kalmar filter algorithm .

(7) The radar unit, inertial navigation unit, positioning unit and camera unit are installed on adjacent street lamps at a fixed distance, and the positioning design is rationalized and standardized.

(8) The radar unit, inertial navigation unit, positioning unit and camera unit on adjacent street lamps are connected in parallel. When a single power-consuming component is damaged, it will not affect the normal operation of other power-consuming components, ensuring the operation of the entire system stability.

Description of drawings

Fig. 1 is a scheme diagram of the environment perception system of the present invention;

Fig. 2 is the unmanned driving scheme figure of the present invention;

Fig. 3 is the laser radar recognition algorithm flowchart of the present invention;

Fig. 4 is a high-precision electronic construction diagram of the adjacent area of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention; obviously, the described embodiments are only part of the embodiments of the present invention, not all embodiments, based on The embodiments of the present invention and all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

In the description of the present invention, it should be noted that the orientations or positional relationships indicated by the terms "upper", "lower", "inner", "outer", "top/bottom" etc. are based on the orientations shown in the drawings Or positional relationship is only for the convenience of describing the present invention and simplifying the description, but does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only, and should not be understood as indicating or implying relative importance.

In the description of the present invention, it should be noted that, unless otherwise specified and limited, the terms "installed", "set with", "sleeved/connected", "connected", etc. should be understood in a broad sense, such as " Connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediary, and it can be an internal connection between two components. connectivity. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention in specific situations.

Example 1:

Please refer to Figure 1. The deep learning intelligent driving environment perception system based on the Internet of Things includes a perception system and an intelligent driving vehicle. The perception system includes a perception layer and a cognitive layer. The intelligent driving vehicle is equipped with a decision-making layer and a control layer. The perception layer It is used for data collection, and the perception layer includes radar unit, inertial navigation unit, positioning unit and camera unit, the external computer is used as the cognitive layer for data analysis, and the decision-making layer set on the intelligent driving vehicle is used to transmit the cognitive layer The information and route planning are processed by algorithms, and the instructions for adjusting the speed and direction of the vehicle are output to the control layer. The control layer set on the intelligent driving vehicle receives the instructions of the decision-making layer and controls the brakes, accelerator and gear position of the vehicle. Improve the realization of unmanned driving, and improve the perception system and intelligent driving vehicles at the same time, so that the two can cooperate perfectly, and reduce technical difficulty and production costs while improving the safety and stability of unmanned driving.

The radar unit, inertial navigation unit, positioning unit and camera unit can be installed on the street lamps on both sides of the road, and the original street lamp equipment is used for the installation and design of the perception system, without additional large-scale hardware facilities, which can save a lot of hardware configuration and save The cognitive layer includes the analysis of pedestrians, vehicles, traffic objects, traffic signs and lane lines.

The radar unit includes lidar and millimeter-wave radar. Please refer to Figure 3. The millimeter-wave radar is a device that measures the distance, angle and relative speed of vehicles around the car by sending and receiving radio waves. Currently, it is widely used as vehicle radar. It is not easily affected by bad weather such as fog, rain and snow, as well as dust and dirt, and can detect vehicles stably. In this system, the millimeter-wave radar is based on a multi-target detection algorithm to detect moving obstacles on the driveway and sidewalk in a fixed area The distance and speed of the object.

The laser radar is prohibited, and its scanning environment is fixed. The laser radar first obtains the environmental data and stores them in the computer in the form of an array, preprocesses the acquired environmental data, removes information such as trees and ground, and performs distance information of the laser radar, At the same time, the reflection intensity information is processed by the non-planar algorithm of the environmental data segmentation and clustering process, and the circumscribed rectangular outline features of the obstacle are extracted. The lidar adopts the multi-hypothesis tracking model algorithm to perform data association on the obstacle information of two consecutive frames, and uses the Kalman filter algorithm Continuous prediction and tracking of dynamic obstacles.

Please refer to Figure 1. Wireless communication is used between the perception system and the intelligent driving vehicle. The perception system sends the identified data information to all nearby intelligent driving vehicles, so that each vehicle can clearly control all obstacles, other vehicles and other vehicles around it. The direction and speed of travel of the vehicle.

The sensor data fusion of radar unit, inertial navigation unit, positioning unit and camera unit includes space fusion, time fusion and sensor data fusion algorithm, and establishes accurate lidar coordinate system, three-dimensional world coordinate system and millimeter wave coordinate system, which is the realization of multi-sensor The key to spatial fusion of data. Spatial fusion of laser radar and millimeter-wave radar is to convert the measured values of different sensor coordinate systems into the same coordinate system; the information of laser radar and millimeter-wave radar needs to be fused in space, and the sensor also needs to collect data synchronously in time , to achieve time fusion. The sampling frequency of the two sensors is different. In order to ensure the reliability of the data, based on the low-sampling rate sensor, the low-frequency sensor collects a frame of image and selects the data cached on the high-frequency sensor for a frame to complete a joint sampling. Frame radar and visual fusion data, thus ensuring the time synchronization of millimeter-wave radar data and camera data; the core key of sensor data fusion is to adopt a suitable fusion algorithm, and the sensor data fusion algorithm of this system adopts extended Kalmar filter algorithm .

Please refer to Figure 4. The radar unit, inertial navigation unit, positioning unit and camera unit are installed on adjacent street lights at a fixed distance, and the positioning design is rationalized and standardized.

The radar unit, inertial navigation unit, positioning unit and camera unit on adjacent street lights are connected in parallel. When a single power-consuming component is damaged, it will not affect the normal operation of other power-consuming components, which is convenient for maintenance. The settings can facilitate relevant maintenance personnel to quickly and accurately find the location of the fault, speed up the maintenance process, and ensure the stability of the entire system operation.

Compared with the traditional technology, this solution improves the implementation of unmanned driving, and improves the perception system and intelligent driving vehicles at the same time. The perception system is used to collect all moving and stationary obstacles on the road, and the data information of these obstacles Sent to all intelligent driving vehicles on the road, while improving the safety and stability of unmanned driving, it reduces technical difficulty and production costs.

The above description is only a preferred embodiment of the present invention; however, the scope of protection of the present invention is not limited thereto. Anyone familiar with the technical field within the technical scope disclosed in the present invention, according to the technical solution of the present invention and its improved concept to make equivalent replacements or changes shall fall within the scope of protection of the present invention.

Claims (10)

1. the deep learning type intelligent driving context aware systems based on Internet of Things, it is characterised in that: including sensory perceptual system and intelligence Vehicle can be driven, the sensory perceptual system includes sensing layer and cognition layer, is provided with decision-making level and control on the intelligent driving vehicle Preparative layer, the sensing layer is acquired for data, and sensing layer includes radar cell, inertial navigation unit, positioning unit and camera shooting Unit, the cognition layer are analyzed for data, and the decision-making level uses algorithm for that will recognize the information and route planning that layer transmits It is handled, and the instruction to control layer output adjustment speed and direction, the control layer receives the instruction of decision-making level, and controls Brake, throttle and the gear of vehicle.

2. the deep learning type intelligent driving context aware systems according to claim 1 based on Internet of Things, feature exist In: it is described on the street lamp that the radar cell, inertial navigation unit, positioning unit and camera unit are mountable to both sides of the road Cognition layer includes the analysis to pedestrian, vehicle, traffic article, traffic mark and lane line.

3. the deep learning type intelligent driving context aware systems according to claim 1 based on Internet of Things, feature exist In: the radar cell includes laser radar and millimetre-wave radar.

4. the deep learning type intelligent driving context aware systems according to claim 3 based on Internet of Things, feature exist In: the laser radar is to forbid, and the environment strafed is to fix.

5. the deep learning type intelligent driving context aware systems according to claim 4 based on Internet of Things, feature exist In: the laser radar is obtained environmental data first and is stored in computer with array form, to the environmental data of acquisition into Row pretreatment, rejects trees, and the information such as ground carry out range information, the Reflection intensity information of laser radar non-planar simultaneously The environmental data of algorithm divides clustering processing, extracts barrier boundary rectangle contour feature.

6. the deep learning type intelligent driving context aware systems according to claim 5 based on Internet of Things, feature exist In: laser radar carries out data correlation using obstacle information of the multiple hypotheis tracking model algorithm to two continuous frames, and utilizes card Kalman Filtering algorithm is continuously predicted and is tracked to dynamic barrier.

7. the deep learning type intelligent driving context aware systems according to claim 1 based on Internet of Things, feature exist In: using wireless communication between the sensory perceptual system and intelligent driving vehicle.

8. the deep learning type intelligent driving context aware systems according to claim 1 based on Internet of Things, feature exist In the Data Fusion of Sensor of: the radar cell, inertial navigation unit, positioning unit and camera unit include Space integration, Fusion in Time and Data Fusion of Sensor algorithm, and Data Fusion of Sensor algorithm is using extension karr horse filtering algorithm.

9. the deep learning type intelligent driving context aware systems according to claim 2 based on Internet of Things, feature exist In: on the adjacent street lamp that the radar cell, inertial navigation unit, positioning unit and camera unit are installed on fixed range.

10. the deep learning type intelligent driving context aware systems according to claim 9 based on Internet of Things, feature exist In: radar cell, inertial navigation unit, positioning unit on the adjacent street lamp and the connection that parallel connection is used between camera unit Mode.