[go: up one dir, main page]

CN112477873B - An assisted driving and vehicle safety management system based on Internet of Vehicles - Google Patents

An assisted driving and vehicle safety management system based on Internet of Vehicles Download PDF

Info

Publication number
CN112477873B
CN112477873B CN202011476325.9A CN202011476325A CN112477873B CN 112477873 B CN112477873 B CN 112477873B CN 202011476325 A CN202011476325 A CN 202011476325A CN 112477873 B CN112477873 B CN 112477873B
Authority
CN
China
Prior art keywords
vehicle
module
information
image
unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011476325.9A
Other languages
Chinese (zh)
Other versions
CN112477873A (en
Inventor
王骥
凌敏
袁志杰
林晓棠
罗颖杰
梁程锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong Ocean University
Original Assignee
Guangdong Ocean University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong Ocean University filed Critical Guangdong Ocean University
Priority to CN202011476325.9A priority Critical patent/CN112477873B/en
Publication of CN112477873A publication Critical patent/CN112477873A/en
Application granted granted Critical
Publication of CN112477873B publication Critical patent/CN112477873B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R16/00—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
    • B60R16/02—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
    • B60R16/023—Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for transmission of signals between vehicle parts or subsystems
    • B60R16/0231—Circuits relating to the driving or the functioning of the vehicle
    • B60R16/0232—Circuits relating to the driving or the functioning of the vehicle for measuring vehicle parameters and indicating critical, abnormal or dangerous conditions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29—Geographical information databases
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/953—Querying, e.g. by the use of web search engines
    • G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0968—Systems involving transmission of navigation instructions to the vehicle
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y20/00—Information sensed or collected by the things
    • G16Y20/10—Information sensed or collected by the things relating to the environment, e.g. temperature; relating to location
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y20/00—Information sensed or collected by the things
    • G16Y20/20—Information sensed or collected by the things relating to the thing itself
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y20/00—Information sensed or collected by the things
    • G16Y20/40—Information sensed or collected by the things relating to personal data, e.g. biometric data, records or preferences
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/10—Detection; Monitoring
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/20—Analytics; Diagnosis
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/50—Safety; Security of things, users, data or systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/60—Positioning; Navigation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02—Services making use of location information
    • H04W4/021—Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02—Services making use of location information
    • H04W4/029—Location-based management or tracking services
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B60W2040/0827—Inactivity or incapacity of driver due to sleepiness

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Signal Processing (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Data Mining & Analysis (AREA)
  • Transportation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Physics (AREA)
  • Biomedical Technology (AREA)
  • Computer Security & Cryptography (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Atmospheric Sciences (AREA)
  • Environmental & Geological Engineering (AREA)
  • Toxicology (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention discloses an auxiliary driving and vehicle safety management system based on the Internet of vehicles, which comprises a user side system, a user side data storage module, a user side communication module, a user side display module, a user side data processing unit and a user side positioning module, wherein the user side system comprises a user side data storage module, a user side communication module and a user side positioning module; the vehicle-end system comprises an in-vehicle data collection module, a driver driving state detection module, a lane identification detection module, a traffic road identification detection module, a vehicle-end data storage module and a vehicle-end communication module; the server-side system comprises a server-side data storage module, a server-side data processing module and a server-side communication module; the invention has the advantages of great development prospect, low cost for the hardware part, no erosion of hardware equipment by the harsh environment outside the vehicle, adoption of a front-end and rear-end separated mode in the software aspect, separation of the logic and view layers of data, enhanced expansibility and low development and upgrading cost of the software.

Description

Auxiliary driving and vehicle safety management system based on Internet of vehicles
Technical Field
The invention belongs to the field of telemedicine, and particularly relates to an auxiliary driving and vehicle safety management system based on the Internet of vehicles.
Background
Currently, the traffic industry in China is rapidly developed, and vehicles become a part of daily life of people; the increase of the traffic flow causes road blockage, so that the working efficiency of the whole traffic transportation system is low, and the problem of traffic accidents is more serious; the passive protection of the airbag, crash barrier, of the type of vehicles today has not met the needs of the individual. With the development of the construction and service capability of Beidou system in the traffic industry of China. The technology of the Internet of vehicles starts to enter the rising period, and the urgent need is to create a practical and functional Internet of vehicles product which is of great significance.
Disclosure of Invention
Aiming at the problems and defects of similar software in function and design on the basis of a Beidou satellite positioning system, the invention aims at providing a vehicle networking service with richer content and more complete function for a user, obtains the position and the running track of a vehicle by analyzing and processing the positioning service provided by the Beidou navigation positioning satellite, and collects data such as temperature, humidity, smoke concentration, alcohol concentration, distance of surrounding obstacles and the like in the vehicle by using a sensor; monitoring the driving state of a driver in real time by using a camera, and collecting the information of the road surface in front of the vehicle in real time; based on an OpenCV face spectrogram, the states of a driver are monitored by face detection and face features, whether fatigue driving exists or the situation of distraction such as mobile phone watching exists or not is judged, objects such as vehicles, roads and traffic lights are identified by combining a computer vision technology, functions of monitoring the running states of the vehicles, inquiring historical tracks, planning geofences, searching places, planning paths, forecasting weather and the like are realized on an autonomously built Web application, and a set of internet-of-vehicles products with perfect functions is provided.
The invention provides an auxiliary driving and vehicle safety management system based on the Internet of vehicles, which comprises the following components:
The client system comprises a client data storage module, a client communication module, a client display module, a client data processing unit, a client positioning module,
The user terminal data storage module is connected with the user terminal communication module, the user terminal data processing unit and the user terminal positioning module,
The user terminal data processing unit is connected with the user terminal display module,
Wherein the user side display module comprises,
A vehicle integrated information display unit for displaying vehicle integrated information of the target vehicle,
A driver information display pre-warning unit for displaying the driver information of the target vehicle and pre-warning according to the driver information,
A location searching unit for inputting the target location, displaying the target location on the user side display module,
A path planning unit for inputting the target location, displaying a path reaching the target location on the user side display module,
A geofence unit for selecting a target area, a display module at the user end for displaying the target area, a vehicle positioning module at the user end for displaying the movement condition of the target vehicle at the target area in real time,
A vehicle history track unit for inquiring and displaying the history motion track of the target vehicle,
A user identification unit of the user terminal; user identification and login for the user-side system;
The vehicle-end system comprises an in-vehicle data collection module, a driver driving state detection module, a lane identification detection module, a traffic road surface identification detection module, a vehicle-end data storage module, a vehicle-end communication module,
The vehicle-end data storage module is connected with the in-vehicle data collection module, the driver driving state detection module, the lane identification detection module, the traffic road surface identification detection module and the vehicle-end communication module,
Wherein the in-vehicle data collection module is used for collecting the environmental information of the target vehicle, positioning the vehicle and obtaining the movement track information of the target vehicle,
The driver driving state detection module is used for collecting and detecting the driver state image of the target vehicle through an internal camera arranged on the target vehicle,
The lane recognition detection module is used for collecting and detecting the lane condition of the road surface during the running process of the target vehicle through an external camera arranged on the target vehicle,
The traffic road surface recognition detection module is used for detecting and recognizing the road surface condition of the road surface through the external camera;
The server-side system comprises a server-side data storage module, a server-side data processing module, a server-side communication module,
The server-side data storage module is connected with the server-side data collection module and the server-side data processing module.
Preferably, the target area comprises a plurality of areas, wherein each area comprises an overlapping part and a non-overlapping part; and adding or deleting the target area through the geofence unit.
Preferably, the historical motion trail comprises a plurality of motion trail, the motion trail information is collected through the positioning function of the in-vehicle data collection module, the motion trail information is transmitted to the server-side data storage module through the vehicle-side communication module, the historical motion trail is generated through the server-side data processing module, and the vehicle historical trail unit displays and inquires the historical motion trail through the user-side communication module based on the server-side data storage module.
Preferably, the in-vehicle data collection module comprises a smoke sensor unit, a temperature sensor unit, a vehicle positioning unit, a data conversion unit,
Wherein,
The smoke alcohol sensor unit is used for detecting internal gas information of the target vehicle;
The temperature and humidity sensor unit is used for detecting internal temperature and humidity information of the target vehicle;
the vehicle positioning unit is used for positioning the target vehicle in real time and obtaining the motion trail information;
The data conversion unit is used for converting the internal gas information, the internal temperature and humidity information and the movement track information into digital signals;
The in-vehicle data collection module is configured to transmit the digital signal to the server-side data storage module through the vehicle-side communication module based on the digital signal, process the digital signal based on the server-side data processing module to obtain a processing result, transmit the processing result to the client-side system through the server-side communication module, and display the processing result through the vehicle comprehensive information display unit, where the vehicle comprehensive information includes the processing result, and the vehicle-side communication module is an EC204G wireless communication module.
Preferably, the smoke alcohol sensor unit comprises an mp2 smoke sensor module and an mp3 alcohol sensor module;
the data conversion unit is an mcu converter;
the temperature and humidity sensor unit is a DHT11 temperature and humidity sensor;
the vehicle positioning unit is an ATK1218-BD Beidou module;
And the vehicle positioning unit obtains the movement track information through the data conversion unit.
Preferably, the driver driving state detection module includes a face recognition unit;
The driving state monitoring module is used for obtaining a driver face image of the target vehicle through the internal camera, carrying out eye positioning on the driver face image through the face recognition unit to obtain a driver face eye positioning image, carrying out recognition processing on the driver face eye positioning image to obtain driver information, and transmitting the driver information to the driver information display early warning unit through the vehicle end communication module.
Preferably, the driver face eye positioning image includes a left eye positioning image and a right eye positioning image, wherein the left eye positioning image obtains a left eye positioning image by selecting three left eye positioning points, and the right eye positioning image obtains a right eye positioning image by selecting three right eye positioning points;
The face recognition unit is used for recognizing the left eye positioning image and the right eye positioning image based on a threshold value by setting the threshold value of the face eye positioning image of the driver, wherein when the left eye positioning image and the right eye positioning image do not meet the threshold value, early warning information is output, and when the left eye positioning image and the right eye positioning image meet the threshold value, normal information is output;
The driver information comprises early warning information and normal information;
and the driver information display early warning unit is used for carrying out early warning according to the early warning information.
Preferably, the lane recognition detection module is configured to collect an original lane image of the road surface through the external camera, perform gray level image processing on the original lane image for several times to obtain an original target image, obtain target lane image data through gaussian blur processing, image contour processing and hough direct detection processing on the original target image, transmit the target lane image data to the user terminal system through the vehicle terminal communication module, and display the target lane image data through the vehicle comprehensive information display unit, where the vehicle comprehensive information further includes the target lane image data.
Preferably, the traffic road surface recognition detection module is configured to obtain an original road surface image through the external camera, recognize a road surface object image of the original road surface image based on YOLO, construct a road surface condition model based on the road surface object image according to the original road surface image, obtain the road surface condition, transmit the road surface condition to the user side system through the vehicle side communication module, and display the road surface condition through the vehicle comprehensive information display unit, where the vehicle comprehensive information further includes the road surface condition.
Preferably, the client system further comprises a convenient living module;
the convenient life module comprises a weather condition display unit and a current life index unit;
The user side system displays the temperature information, the temperature sensing information, the wind direction information and the wind force information of the area where the user side system is based on the weather condition display unit of the current day through the user side positioning module;
and the user side system displays the living information of the region through the user side positioning module based on the current living index unit.
The invention has the positive progress effects that:
The invention has great development prospect, the cost of the hardware part is lower, and the hardware equipment is arranged in the vehicle and cannot be corroded by the harsh environment outside the vehicle. The durability of the hardware is greatly increased. The architecture in the aspect of software adopts a front-end and rear-end separated mode, the mode realizes the characteristic of high cohesion and low coupling, the logic and the view layer of data are separated, the development is more efficient, the expansibility is enhanced, and the development and upgrading cost of the software is low.
The invention has the greatest characteristics in terms of the computer vision recognition technology, and in the prior art, when a traffic accident occurs, passive protective measures such as an air bag or an anti-collision fence of an automobile are difficult to better ensure the life safety of people, but if the computer vision recognition technology is combined, the detection and judgment of some behaviors of a driver, such as the detection of the fatigue state of the driver, are carried out in the driving process of the driver, and whether the driver is in an abnormal state or not. If some signs of fatigue appear, the user is reminded to achieve the purpose of preventing the accident, and the occurrence of traffic accidents is reduced. And the data in the vehicle is received in real time, so that a user can look up the real-time condition in the vehicle, and the driving safety is ensured.
Drawings
FIG. 1 is a block diagram of a system according to the present invention;
FIG. 2 is a diagram of the overall architecture of the system according to the present invention;
FIG. 3 is a flow chart of a system implementation according to the present invention;
FIG. 4 is a diagram of a smoke alcohol sensor unit according to the present invention, wherein 4a is a mq2 smoke sensor and 4b is a mq3 alcohol sensor;
FIG. 5 shows an ATK1218-BD Beidou module of the present invention;
FIG. 6 shows a DHT11 temperature and humidity sensor according to the present invention;
FIG. 7 is an EC204G wireless communication module according to the present invention;
FIG. 8 is a hardware flow chart of the present invention;
FIG. 9 is a software flow chart according to the present invention;
FIG. 10 is a diagram of the detection process according to the present invention;
FIG. 11 is a schematic diagram of test data according to the present invention;
FIG. 12 is a schematic view of a location search according to the present invention;
FIG. 13 is a schematic diagram of a path plan according to the present invention;
FIG. 14 is a schematic view of real-time vehicle positioning according to the present invention;
fig. 15 is a schematic view of a ready-to-enter fence area according to the present invention;
Fig. 16 is a schematic view of an out-of-fence area in accordance with the present invention;
FIG. 17 is a schematic view of a vehicle history track according to the present invention;
FIG. 18 is a graphical illustration of a screen vehicle history track interface in accordance with the present invention;
FIG. 19 is a weather index chart according to the present invention;
FIG. 20 is a diagram of a life index according to the present invention;
FIG. 21 is a circuit diagram of a system according to the present invention;
fig. 22 shows the fencing function of the present invention.
Detailed Description
The following description of the preferred embodiments of the present invention will be given with reference to the accompanying drawings, in order to explain the technical solution of the present invention in detail, but it is not intended to limit the present invention to the scope of the embodiments described.
As shown in fig. 1 to 21, the present embodiment provides a driving assistance and vehicle safety management system based on the internet of vehicles, including:
The client system comprises a client data storage module, a client communication module, a client display module, a client data processing unit, a client positioning module,
The user terminal data storage module is connected with the user terminal communication module, the user terminal data processing unit and the user terminal positioning module,
The user terminal data processing unit is connected with the user terminal display module,
Wherein the user side display module comprises,
A vehicle integrated information display unit for displaying vehicle integrated information of the target vehicle,
A driver information display pre-warning unit for displaying the driver information of the target vehicle and pre-warning according to the driver information,
A location searching unit for inputting the target location, displaying the target location on the user side display module,
A path planning unit for inputting the target location, displaying a path reaching the target location on the user side display module,
A geofence unit for selecting a target area, a display module at the user end for displaying the target area, a vehicle positioning module at the user end for displaying the movement condition of the target vehicle at the target area in real time,
A vehicle history track unit for inquiring and displaying the history motion track of the target vehicle,
A user identification unit of the user terminal; user identification and login for the user-side system;
The vehicle-end system comprises an in-vehicle data collection module, a driver driving state detection module, a lane identification detection module, a traffic road surface identification detection module, a vehicle-end data storage module, a vehicle-end communication module,
The vehicle-end data storage module is connected with the in-vehicle data collection module, the driver driving state detection module, the lane identification detection module, the traffic road surface identification detection module and the vehicle-end communication module,
Wherein the in-vehicle data collection module is used for collecting the environmental information of the target vehicle, positioning the vehicle and obtaining the movement track information of the target vehicle,
The driver driving state detection module is used for collecting and detecting the driver state image of the target vehicle through an internal camera arranged on the target vehicle,
The lane recognition detection module is used for collecting and detecting the lane condition of the road surface during the running process of the target vehicle through an external camera arranged on the target vehicle,
The traffic road surface recognition detection module is used for detecting and recognizing the road surface condition of the road surface through the external camera;
The server-side system comprises a server-side data storage module, a server-side data processing module, a server-side communication module,
The server-side data storage module is connected with the server-side data collection module and the server-side data processing module.
The target area comprises a plurality of areas, wherein each area comprises a superposition part and a non-superposition part; and adding or deleting the target area through the geofence unit.
The historical motion trail comprises a plurality of motion trail, the motion trail information is collected through the positioning function of the in-vehicle data collection module, the motion trail information is transmitted to the server-side data storage module through the vehicle-side communication module, the historical motion trail is generated through the server-side data processing module, and the historical motion trail unit of the vehicle is displayed and inquired through the user-side communication module based on the server-side data storage module.
The in-vehicle data collection module comprises a smoke sensor unit, a temperature sensor unit, a vehicle positioning unit and a data conversion unit,
Wherein,
The smoke alcohol sensor unit is used for detecting internal gas information of the target vehicle;
The temperature and humidity sensor unit is used for detecting internal temperature and humidity information of the target vehicle;
the vehicle positioning unit is used for positioning the target vehicle in real time and obtaining the motion trail information;
The data conversion unit is used for converting the internal gas information, the internal temperature and humidity information and the movement track information into digital signals;
The in-vehicle data collection module is configured to transmit the digital signal to the server-side data storage module through the vehicle-side communication module based on the digital signal, process the digital signal based on the server-side data processing module to obtain a processing result, transmit the processing result to the client-side system through the server-side communication module, and display the processing result through the vehicle comprehensive information display unit, where the vehicle comprehensive information includes the processing result, and the vehicle-side communication module is an EC204G wireless communication module.
The smoke alcohol sensor unit comprises an mp2 smoke sensor module and an mp3 alcohol sensor module;
the data conversion unit is an mcu converter;
the temperature and humidity sensor unit is a DHT11 temperature and humidity sensor;
the vehicle positioning unit is an ATK1218-BD Beidou module;
And the vehicle positioning unit obtains the movement track information through the data conversion unit.
Preferably, the driver driving state detection module includes a face recognition unit;
The driving state monitoring module is used for obtaining a driver face image of the target vehicle through the internal camera, carrying out eye positioning on the driver face image through the face recognition unit to obtain a driver face eye positioning image, carrying out recognition processing on the driver face eye positioning image to obtain driver information, and transmitting the driver information to the driver information display early warning unit through the vehicle end communication module.
The driver face eye positioning image comprises a left eye positioning image and a right eye positioning image, wherein the left eye positioning image is used for obtaining a left eye positioning image by selecting three left eye positioning points, and the right eye positioning image is used for obtaining a right eye positioning image by selecting three right eye positioning points;
The face recognition unit is used for recognizing the left eye positioning image and the right eye positioning image based on a threshold value by setting the threshold value of the face eye positioning image of the driver, wherein when the left eye positioning image and the right eye positioning image do not meet the threshold value, early warning information is output, and when the left eye positioning image and the right eye positioning image meet the threshold value, normal information is output;
The driver information comprises early warning information and normal information;
and the driver information display early warning unit is used for carrying out early warning according to the early warning information.
The lane recognition detection module is used for collecting an original lane image of the road surface through the external camera, carrying out gray level image processing on the original lane image for a plurality of times to obtain an original target image, carrying out Gaussian blur processing, image contour processing and Hough direct detection processing on the original target image to obtain target lane image data, transmitting the target lane image data to the user side system through the vehicle side communication module, and displaying the target lane image data through the vehicle comprehensive information display unit, wherein the vehicle comprehensive information further comprises the target lane image data.
The traffic road surface recognition and detection module is used for obtaining an original road surface image through the external camera, recognizing a road surface object image of the original road surface image based on the YOLO, constructing a road surface condition model based on the road surface object image according to the original road surface image, obtaining the road surface condition, transmitting the road surface condition to the user side system through the vehicle side communication module, and displaying the road surface condition through the vehicle comprehensive information display unit, wherein the vehicle comprehensive information further comprises the road surface condition.
The user side system further comprises a convenient living module;
the convenient life module comprises a weather condition display unit and a current life index unit;
The user side system displays the temperature information, the temperature sensing information, the wind direction information and the wind force information of the area where the user side system is based on the weather condition display unit of the current day through the user side positioning module;
and the user side system displays the living information of the region through the user side positioning module based on the current living index unit.
The design thought and the technical constitution of the technical scheme of the application are explained in detail as follows:
Aiming at the problems and defects of similar software in function and design on the basis of a Beidou satellite positioning system, the invention aims at providing a vehicle networking service with richer content and more complete function for a user, obtains the position and the running track of a vehicle by analyzing and processing the positioning service provided by the Beidou navigation positioning satellite, and collects data such as temperature, humidity, smoke concentration, alcohol concentration, distance of surrounding obstacles and the like in the vehicle by using a sensor; monitoring the driving state of a driver in real time by using a camera, and collecting the information of the road surface in front of the vehicle in real time; based on an OpenCV face spectrogram, the states of a driver are monitored by face detection and face features, whether fatigue driving exists or the situation of distraction such as mobile phone watching exists or not is judged, objects such as vehicles, roads and traffic lights are identified by combining a computer vision technology, functions of monitoring the running states of the vehicles, inquiring historical tracks, planning geofences, searching places, planning paths, forecasting weather and the like are realized on an autonomously built Web application, and a set of internet-of-vehicles products with perfect functions is provided.
The whole structure of the invention is as follows: the cloud server is matched as a data transfer station, stm32f103 is adopted as a main control chip by hardware equipment, vehicle positioning data and in-vehicle environment data are collected through a serial port, GPIO and an analog-to-digital converter, and TCP protocol communication can be carried out through an EC204G module; the web client is connected with the cloud server through the IP and the port number, and the web client and the cloud server are connected with each other simultaneously to realize remote connection. Thereby realizing communication between data and transmission between data. The communication module is designed based on TCP, webSocket protocol and Netty framework.
The concrete implementation of the frame is as follows: the cloud server is provided with a network service program written based on a Netty framework, and waits for the connection of the client. And after the service end program of the Web application establishes connection with the cloud server according to the IP address and the port number, carrying out data processing and storage on the response of the cloud server. And the server program of the Web application sends the processed data to the browser for display through the WebSocket connection.
In terms of hardware, we use stm32rct6, ARM-Cortex-M3 kernel microprocessor as the main control CPU for motion, highest operating frequency 72MHz,128KB SRAM memory. The chip has rich peripherals such as A/D conversion, serial ports and the like, and has high cost performance. The concentration can be obtained by using the mq2 and mq3 modules, converting the analog signals sent by the analog-to-digital conversion acquisition module into digital signals, and processing the digital signals. And in the aspect of positioning, an ATK1218-BD positioning module is used, original Beidou positioning original data is received through a serial port, and the data format of a protocol is analyzed to obtain longitude and latitude. And acquiring the temperature and humidity in the vehicle through the DHT11 module, and finally transmitting the data to a server through the EC204G module. A high-definition camera is built on the raspberry party, and through face recognition and WiFi communication, a server can obtain real-time video and perform image recognition by accessing ip and port numbers.
In terms of software, the Web application is mainly developed by adopting Java EE and Vue. The server uses IntelliJ IDEA as a development platform, adopts Spring Boot as a core framework, combines SPRING DATA and MyBatisPlus as a database interaction framework and uses Netty as a communication framework for development. The client uses WebStorm as a development platform, adopts Vue.js as a core framework, elementUI as a UI framework, and combines Express and database interaction for development. The cloud server program is developed by using a Netty framework and is installed in an environment where the CentOS7 can access the external network. The database aspect employs MySQL of the relational data management system. Through the application, the user can realize the functions of monitoring the running state of the vehicle, inquiring the historical track, planning the geofence, searching the place, planning the path, forecasting the weather and the like
The server is selected from an ali cloud server, and a remote server adopts an ali cloud server of an ali bus company, which provides expandable computing capacity and has unique IP of a public network, so that communication with the server can be realized in theory as long as the internet can be connected anywhere. Through the Arian cloud server, the application program can be rapidly deployed, and resources such as a CPU, a memory, a hard disk and the like are used. Our server uses Linux Centos systems to build contact with hardware devices and Web application server side programs using Netty, process and NIO streaming techniques. After the user accesses the application program, connection with the cloud server is automatically established according to the IP address and the port. After successful connection, the user can easily establish contact with the car equipment through the transfer station of the remote server.
In the technical selection of the front end aspect, the Vue framework has the advantages of light weight framework, simplicity, easiness in learning, bidirectional data binding, componentization, view, separation of data and structure, virtual DOM and high running speed. The front-end developer is provided with a quick and attractive interface for constructing the page.
In the computer vision recognition technology, the Python OpenCV library is adopted, and in the aspect of computer vision recognition, python has great advantages compared with other languages, meanwhile, the YOLO framework which is a computer vision recognition framework provided by Python is adopted, and models belonging to the people can be trained through yolov so as to realize the corresponding vision recognition functions.
The alcohol smoke detection module is used for collecting alcohol concentration and smoke concentration in the vehicle. The module has high sensitivity, quick response, good stability and long service life. The anti-interference device has good anti-interference performance, and can accurately remove interference information of the irritating nonflammable smoke. The acquisition of the MQ-2 smoke concentration is realized, and the acquisition of signals can be completed only by realizing an ADC0832 acquisition function. However, the signal collected by the ADC0832 is only an original signal, and is to be converted into an actual smoke concentration, and correction and formula conversion are also required according to the characteristics of MQ-2, so as to obtain an actual concentration value.
The vehicle positioning unit is used for positioning the vehicle position, adopts a GPS and Beidou dual mode for positioning, can perform various parameter setting and data receiving and transmitting through a serial port, can store internal FLASH, and is convenient to use. Through ATK-S1218-BD GPS/big Dipper module, any singlechip (3.3V/5V power) can realize GPS/big Dipper location very conveniently.
The ATK1218-BD GPS/Beidou module has the following characteristics:
The module adopts an S1216F8-BD module, has small volume and excellent performance.
The module can set various parameters through the serial port, can store the internal FLASH, and is convenient to use.
The module is provided with an IPX interface and can be connected with various active antennas
The module is compatible with 3.3V/5V level, and is convenient to connect various singlechip systems.
The module is provided with a rechargeable backup battery and can be powered down to maintain ephemeris data
The temperature and humidity sensor unit is used for collecting the temperature and humidity in the vehicle, and the module is applied to a special digital module acquisition technology and a temperature and humidity sensing technology, so that the product is ensured to have extremely high reliability and excellent long-term stability. The DATA is used for communication and synchronization between the microprocessor and the DHT11, and adopts a single bus DATA format, wherein the communication time is about 4ms, the DATA is divided into a fractional part and an integer part, the specific format is described below, the current fractional part is used for later expansion, and the current zero-reading operation flow is as follows:
One complete data transmission is 40bit, high-order first-out. The data format is that after a last 8-bit user MCU of a result obtained by 8bit humidity integer data+8 bit humidity decimal data+8 bi temperature integer data+8 bit temperature decimal data+8 bit check data and the check sum data is equal to 8bit humidity integer data+8 bit humidity decimal data+8 bi temperature decimal data transmits a start signal once, the DHT11 is converted into a high-speed mode from a low-power mode, after waiting for the end of a host start signal, the DHT11 transmits a response signal, transmits 40bit data and triggers a signal acquisition, and a user can select to read part of data.
EC 20R 2.1 is an LTE Cat4 wireless communication module deduced by remote communication, and adopts LTE 3GPP Rel.11 technology to support a maximum downlink rate of 150Mbps and a maximum uplink rate of 50Mbps; meanwhile, a mobile telecommunication UMTS/HSPA+UC20 module and a multi-network LTE EC20/EC21/EC25/EG25-G module are compatible in packaging, so that seamless switching between a 3G network and a 4G network is realized. The module provides serial communication, and the serial port can be used for AT command or data transmission, supports 9600, 19200, 38400, 57600, 115200, 230400, 460800 and 921600bps baud rate, and can be used for serial communication with mcu, wherein the default baud rate is 115200 bps.
Hardware principle part
In-vehicle data collection module: the mq2 and mq3 sensors are used, and the sensors belong to tin dioxide semiconductor gas-sensitive materials. When in contact with smoke, a change in surface conductivity is caused if the barrier at the grain boundaries changes upon tuning of the smoke. By using this, it is possible to obtain information on the presence of such smoke, and the greater the concentration of smoke, the greater the conductivity, and the lower the output resistance, the greater the analog signal output. The mcu may convert the analog signal to a digital signal, which is converted to a concentration value. The DHT11 temperature and humidity sensor for collecting temperature and humidity adopts a single bus data format to communicate with a processor, the communication time is about 4ms, the data is divided into a decimal part and an integer part, and the data bits are read through time sequence control. The ATK1218 module Beidou module used in the positioning aspect outputs GPS/Beidou positioning data by adopting NMEA0183 protocol by default, the module can be configured through SkyTraq protocol, the mcu receives the output original data through the serial port, and the data format of NMEA0183 protocol is analyzed to obtain the positioning data. The vehicle data is transmitted to the server in real time, an EC204G module is used, and the module is controlled by using AT instructions through a serial port, so that a main control board is connected with a cloud server through a TCP protocol, and the transmission of the vehicle data is performed in real time.
In the driver driving state detection module: the method is characterized in that a high-definition camera is built by using a raspberry pie 4B, face detection is carried out on data transmitted back by the camera in the vehicle by using an opencv library of Python, 6 mark points are marked on the left eye and the right eye of the face image respectively based on facial makeup image data carried by the opencv, and when the mark points of the left eye and the right eye are in a certain state outside the limit of the user through a threshold value set by the user, a warning is sent out so as to achieve the aim of better detecting the driving state of the driver.
In the lane recognition detection module: and carrying out lane line detection on the transmitted data by using an opencv library of Python. After the processes of denoising, image contour, hough straight line detection and the like are carried out on the image through multiple gray level image processing and Gaussian blur, a considerable lane contour line set can be obtained.
In the traffic pavement recognition and detection module: the object is identified and detected based on YOLO, and the speed YOLOv is faster than that of other frames, so that the object can be more consistent with the project. The yolov training model is pre-trained, the original accuracy is kept, the recognition speed is improved, meanwhile, due to the limitation of the performance of an operating machine, the original frame is modified, the recognition data amount is reduced through a data screening method, and meanwhile, the recognition accuracy can be kept well, so that the recognition speed is further improved.
The software principle part:
In the module for planning the fences, the front end is responsible for providing a button for adding the fences, when a user selects to add the fences, the shape and the size of the fences can be controlled by using a mouse, and finally, the longitudes and latitudes corresponding to the fences are collected and sent to a background java client, and after some preprocessing is carried out on the data by the java, the data are added into a MySQL server, so that the fencing storage function is realized.
In the fence viewing module, after a user selects a fence, the system automatically starts to judge whether the running track of the current vehicle enters or exits the fence through a series of algorithms, and if the vehicle enters or exits the fence area, the browser sends a corresponding notification to the user.
In the history track module, the principle of storing the history track is judged through the connection condition of the browser to the rear-end java, when the browser is successfully connected with the client of the java, the corresponding history track starts to be stored, the stored history track is ended after the browser is disconnected with the client of the java, the stored data are sent to the client of the java at one time, and then the client of the java stores the data in MySQL for later viewing by a user.
System test scheme
Test purpose:
The report aims at the stability, the availability, the rendering time of the webpage UI and some unknown errors of the project, discovers the possible performance problems in the existing system, proposes a feasible proposal to reduce the subsequent working risk as much as possible, and provides guarantee for maintaining stable operation.
Test range:
And performing system tests according to the project development description, the software requirement specification and the corresponding design documents, wherein the system tests comprise functional tests, performance tests, user access and safety control tests and user interface tests. The main functions are that the remote server performs Lina sister and receives data transmitted by hardware to perform some processing, and the data is correctly realized on a UI interface.
Test environment:
Configuration: the server adopts an Arian cloud server, a CentrOS7.364 bit operating system and a memory 4G;
application software: a fire fox browser, a google browser and other popular browsers;
A test procedure, as shown in fig. 10;
test data, as shown in fig. 11.
The system of the invention realizes the functions
Searching places: as shown in fig. 12, in the location search module, a user may search for a corresponding location in a search field, and landmarks of the corresponding location in an area that will be displayed on a map provide user-selected viewing.
Path planning: as shown in fig. 13, the user may select a starting location and a travel mode in the path planning module, and then may search for some relevant paths to achieve the navigation effect.
Real-time vehicle positioning: as shown in fig. 14, in the vehicle positioning module, we can accept the longitude and latitude of the hardware, then send the data of the longitude and latitude to the server, and then the client receives the data and displays the data in the browser. Realizing real-time positioning of vehicle
Fence function: as shown in fig. 22, the user can add or delete corresponding pens according to his own needs in the pens; as shown in fig. 15, the successfully added fence can be selected for viewing, and when a vehicle enters the fence, the user is prompted that the vehicle enters the fence range area; as shown in fig. 16, when the vehicle leaves the fence, the user is prompted that the vehicle has left the fence-wide area.
Looking at the vehicle history track, as shown in figures 17-18,
The user can view the historical driving tracks in the interface, and can also screen the historical driving tracks according to keywords to search the historical tracks which are wanted.
In-vehicle data real-time monitoring
In the in-car data monitoring module, the client can receive data transmitted by hardware, and display the data on a browser for a user to view. The data acquired by the hardware is sent to an Ali cloud server, and then the java client processes the data sent by the hardware by connecting the corresponding IP address and port number, and then sends the processed data to a browser for display to a user.
Weather and life index as shown in fig. 19 and 20, in the weather and life index module, a user can obtain weather information such as temperature, somatosensory temperature, wind direction, wind power and the like of a region where the user is located, and can view the current life index. Providing a detailed living guide for the user.
Computer vision recognition
In a driver fatigue detection module, driver image data of a user can be received through a camera, then the data can be detected in real time through Python, the Python passes opencv, then the incoming image is processed, finally the processed image is transmitted back, and whether the driver sleeps or not is known through a plurality of algorithms for judging eyes.
In the traffic road surface detection module, the image data on the street can be sent to the Python again, then the Python carries out image recognition through opencv, and vehicles, pedestrians, traffic lights and the like on the road are detected.
In the route detection module, python recognizes and detects a lane line on a road through opencv. And feeding back the processed result to the user.
The invention has great development prospect, the cost of the hardware part is lower, and the hardware equipment is arranged in the vehicle and cannot be corroded by the harsh environment outside the vehicle. The durability of the hardware is greatly increased. In the aspect of software, the architecture of the invention adopts a front-end and back-end separated mode, the mode realizes the characteristic of high cohesion and low coupling, the logic and the view layer of data are separated, the development is more efficient, the expansibility is enhanced, and the development and upgrading cost of the software is low.
The invention has the greatest characteristics in terms of the computer vision recognition technology, and in the prior art, when a traffic accident occurs, passive protective measures such as an air bag or an anti-collision fence of an automobile are difficult to better ensure the life safety of people, but if the computer vision recognition technology is combined, the detection and judgment of some behaviors of a driver, such as the detection of the fatigue state of the driver, are carried out in the driving process of the driver, and whether the driver is in an abnormal state or not. If some signs of fatigue appear, the user is reminded to achieve the purpose of preventing the accident, and the occurrence of traffic accidents is reduced. And the data in the vehicle is received in real time, so that a user can look up the real-time condition in the vehicle, and the driving safety is ensured.
The above embodiments are only illustrative of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention, and various modifications and improvements made by those skilled in the art to the technical solutions of the present invention should fall within the protection scope defined by the claims of the present invention without departing from the design spirit of the present invention.

Claims (8)

1. An assisted driving and vehicle safety management system based on the internet of vehicles, comprising:
The client system comprises a client data storage module, a client communication module, a client display module, a client data processing unit, a client positioning module,
The user terminal data storage module is connected with the user terminal communication module, the user terminal data processing unit and the user terminal positioning module,
The user terminal data processing unit is connected with the user terminal display module,
Wherein the user side display module comprises,
A vehicle integrated information display unit for displaying vehicle integrated information of the target vehicle,
A driver information display pre-warning unit for displaying the driver information of the target vehicle and pre-warning according to the driver information,
A location searching unit for inputting a target location, displaying the target location on the user side display module,
A path planning unit for inputting a target place, a display module at the user end for displaying a path reaching the target place,
A geofence unit for selecting a target area, a display module at the user end for displaying the target area, a vehicle positioning module at the user end for displaying the movement condition of the target vehicle at the target area in real time,
A vehicle history track unit for inquiring and displaying the history motion track of the target vehicle,
A user identification unit of the user terminal; user identification and login for the user-side system;
The vehicle-end system comprises an in-vehicle data collection module, a driver driving state detection module, a lane identification detection module, a traffic road surface identification detection module, a vehicle-end data storage module, a vehicle-end communication module,
The vehicle-end data storage module is connected with the in-vehicle data collection module, the driver driving state detection module, the lane identification detection module, the traffic road surface identification detection module and the vehicle-end communication module,
Wherein the in-vehicle data collection module is used for collecting the environmental information of the target vehicle, positioning the vehicle and obtaining the movement track information of the target vehicle,
The driver driving state detection module is used for collecting and detecting the driver state image of the target vehicle through an internal camera arranged on the target vehicle,
The lane recognition detection module is used for collecting and detecting the lane condition of the road surface during the running process of the target vehicle through an external camera arranged on the target vehicle,
The traffic road surface recognition detection module is used for detecting and recognizing the road surface condition of the road surface through the external camera;
The server-side system comprises a server-side data storage module, a server-side data processing module, a server-side communication module,
The server-side data storage module is connected with the server-side data processing module and the server-side communication module;
The historical motion trail comprises a plurality of motion trail, the motion trail information is collected through the positioning function of the in-vehicle data collection module, the motion trail information is transmitted to the server-side data storage module through the vehicle-side communication module, the historical motion trail is generated through the server-side data processing module, and the historical motion trail unit of the vehicle is displayed and inquired through the user-side communication module based on the server-side data storage module.
2. A driving assistance and vehicle safety management system based on the internet of vehicles according to claim 1,
The target area comprises a plurality of areas, wherein each area comprises a superposition part and a non-superposition part; and adding or deleting the target area through the geofence unit.
3. A driving assistance and vehicle safety management system based on the internet of vehicles according to claim 1,
The in-vehicle data collection module comprises a smoke sensor unit, a temperature sensor unit, a vehicle positioning unit and a data conversion unit,
Wherein,
The smoke sensor unit is used for detecting internal gas information of the target vehicle;
the temperature sensor unit is used for detecting internal temperature and humidity information of the target vehicle;
the vehicle positioning unit is used for positioning the target vehicle in real time and obtaining the motion trail information;
The data conversion unit is used for converting the internal gas information, the internal temperature and humidity information and the movement track information into digital signals;
The in-vehicle data collection module is configured to transmit the digital signal to the server-side data storage module through the vehicle-side communication module based on the digital signal, process the digital signal based on the server-side data processing module to obtain a processing result, transmit the processing result to the client-side system through the server-side communication module, and display the processing result through the vehicle comprehensive information display unit, where the vehicle comprehensive information includes the processing result, and the vehicle-side communication module is an EC204G wireless communication module.
4. A driving assistance and vehicle safety management system based on the internet of vehicles according to claim 1,
The driver driving state detection module comprises a face recognition unit;
The driving state detection module is used for obtaining a driver face image of the target vehicle through the internal camera, carrying out eye positioning on the driver face image through the face recognition unit to obtain a driver face eye positioning image, carrying out recognition processing on the driver face eye positioning image to obtain driver information, and transmitting the driver information to the driver information display early warning unit through the vehicle end communication module.
5. A driving assistance and vehicle safety management system based on the internet of vehicles according to claim 4,
The driver face eye positioning image comprises a left eye positioning image and a right eye positioning image, wherein the left eye positioning image is used for obtaining a left eye positioning image by selecting three left eye positioning points, and the right eye positioning image is used for obtaining a right eye positioning image by selecting three right eye positioning points;
The face recognition unit is used for recognizing the left eye positioning image and the right eye positioning image based on a threshold value by setting the threshold value of the face eye positioning image of the driver, wherein when the left eye positioning image and the right eye positioning image do not meet the threshold value, early warning information is output, and when the left eye positioning image and the right eye positioning image meet the threshold value, normal information is output;
The driver information comprises early warning information and normal information;
and the driver information display early warning unit is used for carrying out early warning according to the early warning information.
6. A driving assistance and vehicle safety management system based on the internet of vehicles according to claim 1,
The lane recognition detection module is used for collecting an original lane image of the road surface through the external camera, carrying out gray level image processing on the original lane image for a plurality of times to obtain an original target image, carrying out Gaussian blur processing, image contour processing and Hough direct detection processing on the original target image to obtain target lane image data, transmitting the target lane image data to the user side system through the vehicle side communication module, and displaying the target lane image data through the vehicle comprehensive information display unit, wherein the vehicle comprehensive information further comprises the target lane image data.
7. A driving assistance and vehicle safety management system based on the internet of vehicles according to claim 1,
The traffic road surface recognition and detection module is used for obtaining an original road surface image through the external camera, recognizing a road surface object image of the original road surface image based on the YOLO, constructing a road surface condition model based on the road surface object image according to the original road surface image, obtaining the road surface condition, transmitting the road surface condition to the user side system through the vehicle side communication module, and displaying the road surface condition through the vehicle comprehensive information display unit, wherein the vehicle comprehensive information further comprises the road surface condition.
8. A driving assistance and vehicle safety management system based on the internet of vehicles according to claim 1,
The user side system further comprises a convenient living module;
the convenient life module comprises a weather condition display unit and a current life index unit;
The user side system displays the temperature information, the temperature sensing information, the wind direction information and the wind force information of the area where the user side system is based on the weather condition display unit of the current day through the user side positioning module;
and the user side system displays the living information of the region through the user side positioning module based on the current living index unit.
CN202011476325.9A 2020-12-15 2020-12-15 An assisted driving and vehicle safety management system based on Internet of Vehicles Active CN112477873B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011476325.9A CN112477873B (en) 2020-12-15 2020-12-15 An assisted driving and vehicle safety management system based on Internet of Vehicles

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011476325.9A CN112477873B (en) 2020-12-15 2020-12-15 An assisted driving and vehicle safety management system based on Internet of Vehicles

Publications (2)

Publication Number Publication Date
CN112477873A CN112477873A (en) 2021-03-12
CN112477873B true CN112477873B (en) 2024-04-30

Family

ID=74917513

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011476325.9A Active CN112477873B (en) 2020-12-15 2020-12-15 An assisted driving and vehicle safety management system based on Internet of Vehicles

Country Status (1)

Country Link
CN (1) CN112477873B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113085888A (en) * 2021-04-21 2021-07-09 金陵科技学院 Intelligent networked automobile driving-assisting safety information detection system
CN113682112A (en) * 2021-09-15 2021-11-23 深圳地平线机器人科技有限公司 Method and device for controlling gas in vehicle
CN114218209A (en) * 2021-10-28 2022-03-22 北京蜂云科创信息技术有限公司 Hadoop-based vehicle big data processing method and system
CN114512000A (en) * 2022-01-11 2022-05-17 海南省设计研究院有限公司 A smart integrated pole with continuous traffic information detection function
CN114938272B (en) * 2022-05-13 2024-07-23 武汉理工大学 Ship-to-shore data communication encryption device and method based on SM2 encryption algorithm
CN115202789A (en) * 2022-07-19 2022-10-18 东软睿驰汽车技术(大连)有限公司 Method, device and electronic device for layered display of driving assistance information

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014017764A1 (en) * 2012-07-27 2014-01-30 에스케이플래닛 주식회사 Information providing method, and device and system for same
CN111724408A (en) * 2020-06-05 2020-09-29 广东海洋大学 Validation experiment method of abnormal driving behavior algorithm model based on 5G communication

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011140993A1 (en) * 2010-05-12 2011-11-17 北京星河易达科技有限公司 Intelligent traffic safety system based on comprehensive state detection and decision method thereof

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014017764A1 (en) * 2012-07-27 2014-01-30 에스케이플래닛 주식회사 Information providing method, and device and system for same
CN111724408A (en) * 2020-06-05 2020-09-29 广东海洋大学 Validation experiment method of abnormal driving behavior algorithm model based on 5G communication

Also Published As

Publication number Publication date
CN112477873A (en) 2021-03-12

Similar Documents

Publication Publication Date Title
CN112477873B (en) An assisted driving and vehicle safety management system based on Internet of Vehicles
CN106846863B (en) Accident black spot warning system and method based on augmented reality and cloud intelligent decision
CN103247186B (en) Scene-based driver assistant system realizing method
EP4276412A1 (en) Positioning method and related apparatus
CN109556621B (en) Route planning method and related equipment
US9430941B2 (en) Harvesting traffic information from mobile devices
EP4539011A1 (en) Trajectory prediction method as well as apparatus therefor, medium, program product, and electronic device
CN119579882A (en) System and method for actively selecting and labeling images for semantic segmentation
CN109005239A (en) Intelligent information exchange method, system and device for assisting vehicle travel
CN103824457B (en) Based on express way driving safety information query system and the querying method of VANET
CN113923775B (en) Method, device, equipment and storage medium for evaluating quality of positioning information
US12097856B2 (en) System and method for adjusting a yielding space of a platoon
KR102363687B1 (en) Small mobility path generation system using user experience data and method
CN113076830A (en) Environment passing area detection method and device, vehicle-mounted terminal and storage medium
CN201903966U (en) Monitoring system of networking automobiles
CN208016006U (en) Wisdom traffic system based on wisdom lamp stand
CN114722931A (en) Vehicle data processing method, device, data acquisition device and storage medium
CN104916099A (en) System and method for maintaining vehicle health
KR101422767B1 (en) Mobile device and method for classifying user's action and method for creating hierarchical tree model
CN108282744A (en) Wisdom traffic system based on wisdom lamp stand
KR101716653B1 (en) Method for providing detecting missing drone
CN203149847U (en) Road condition sharing server
CN104715601A (en) Intelligent traffic glasses and working method thereof
US12462572B2 (en) Method and system for gathering image training data for a machine learning model
CN205124003U (en) Portable tourism terminal based on cloud calculates

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
OL01 Intention to license declared
OL01 Intention to license declared