CN117292552A - High-speed road condition analysis system and method based on machine vision - Google Patents
High-speed road condition analysis system and method based on machine vision Download PDFInfo
- Publication number
- CN117292552A CN117292552A CN202311589326.8A CN202311589326A CN117292552A CN 117292552 A CN117292552 A CN 117292552A CN 202311589326 A CN202311589326 A CN 202311589326A CN 117292552 A CN117292552 A CN 117292552A
- Authority
- CN
- China
- Prior art keywords
- road condition
- image
- module
- condition image
- speed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
- G08G1/0145—Measuring and analyzing of parameters relative to traffic conditions for specific applications for active traffic flow control
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Molecular Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Databases & Information Systems (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Signal Processing (AREA)
- Biodiversity & Conservation Biology (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of road traffic, and discloses a system and a method for analyzing high-speed road conditions based on machine vision, wherein a highway is divided into a plurality of sections according to monitoring bayonets as nodes, and real-time road condition images shot by monitoring equipment in each section are obtained; preprocessing the real-time road condition image to obtain a preprocessed high-speed road condition image, scaling the preprocessed high-speed road condition image to be uniform in size, and obtaining a standardized road condition image; inputting the standardized road condition image into a road condition analysis model, and analyzing the high-speed road condition through the road condition analysis model to obtain a road condition analysis result; when the road condition analysis result is of a congestion type, the traffic condition and interval information of the congested road section are sent to a traffic command platform; the invention collects the high-speed road condition images, analyzes the high-speed road condition, timely knows the high-speed congestion condition, provides support for relieving the congestion, and improves the travel quality of users.
Description
Technical Field
The invention relates to the technical field of road traffic, in particular to a high-speed road condition analysis system and method based on machine vision.
Background
Expressway, namely expressway for short, refers to a highway special for automobile to run at high speed; the expressway pavement comprises a main road, a ramp and an auxiliary lane, wherein the main road is a roadway, and is sequentially arranged into a passing lane, a fast lane and a slow lane from left to right according to different numbers; with the gradual development of traffic conditions nowadays, the quantity of automobile conservation in China is increased year by year, expressway networks cover all areas of the country, and with the popularization of automobiles, people travel more and more depending on the automobiles, which inevitably leads to the situation of road congestion; the expressway is congested due to excessive vehicle driving and traffic accidents, so that the user traveling experience is poor, and meanwhile, the traffic command platform can only timely know the congestion condition and carry out corresponding traffic congestion road condition prompt, so that research on the system and the method for analyzing the high-speed road condition based on machine vision has important significance for improving the traveling quality of the user.
Disclosure of Invention
The invention aims to solve the problems and designs a system and a method for analyzing high-speed road conditions based on machine vision.
The invention provides a high-speed road condition analysis method based on machine vision, which comprises the following steps:
dividing the expressway into a plurality of intervals according to the monitoring bayonets as nodes, and acquiring real-time road condition images shot by monitoring equipment in each interval;
preprocessing the real-time road condition image to obtain a preprocessed high-speed road condition image, scaling the preprocessed high-speed road condition image to be uniform in size, and obtaining a standardized road condition image;
inputting the standardized road condition image into a road condition analysis model, and analyzing the high-speed road condition through the road condition analysis model to obtain a road condition analysis result;
and when the road condition analysis result is of a congestion type, transmitting traffic conditions and interval information of the congested road sections to a traffic command platform.
Optionally, in a first implementation manner of the first aspect of the present invention, preprocessing the real-time road condition image to obtain a preprocessed high-speed road condition image includes:
acquiring a real-time road condition image shot by monitoring equipment, performing distortion correction processing on the real-time road condition image, and projecting the point coordinates after distortion removal to a pixel screen to obtain a first road condition image;
sequentially carrying out bilateral filtering and Canny edge detection on the first road condition image, extracting edge characteristics of the first road condition image, and obtaining a second road condition image;
and performing target detection on the second road condition image by adopting the probability Hough transform to obtain a preprocessed high-speed road condition image.
Optionally, in a second implementation manner of the first aspect of the present invention, performing bilateral filtering and Canny edge detection processing on the first road condition image sequentially, extracting edge features of the first road condition image, and obtaining a second road condition image includes:
adopting a bilateral filter to perform noise reduction treatment on the first road condition image to obtain a noise-reduced first road condition image;
respectively carrying out a Sobel horizontal operator and a Sobel vertical operator on the first road condition image after noise reduction to obtain image gradient information, wherein the image gradient information comprises a horizontal gradient component and a vertical gradient component;
performing non-maximum value inhibition processing and double threshold screening on the first road condition image after noise reduction based on the image gradient information to obtain pixel information, wherein the pixel information comprises strong edge pixels, weak edge pixels and pixel points needing to be inhibited;
and carrying out edge tracking on the first road condition image after noise reduction based on the pixel information, extracting edge characteristics, and obtaining a second road condition image.
Optionally, in a third implementation manner of the first aspect of the present invention, performing target detection on the second road condition image by using the probability hough transform to obtain a preprocessed high-speed road condition image, where the method includes:
traversing edge points in the second road condition image, carrying out Hough transformation on the edge points, and selecting a point with the largest value in a Hough space to obtain the maximum value;
determining left and right endpoints along a straight line direction based on the maximum value to obtain a detection line segment, and obtaining an intersection point of line segments adjacent to a certain angle threshold value of the detection line segment to obtain a candidate point set;
calculating the distance of the candidate points in the candidate point set by adopting Euclidean distance, combining the points with the minimum distance value, and carrying out hierarchical clustering to obtain vanishing points;
and selecting a target detection area of the high-speed road surface in the second road condition image based on the vanishing point to obtain a preprocessed high-speed road condition image.
Optionally, in a fourth implementation manner of the first aspect of the present invention, inputting the standardized road condition image into a road condition analysis model, and analyzing the high-speed road condition by the road condition analysis model to obtain a road condition analysis result, including:
inputting the standardized road condition image into a road condition analysis model, extracting features through a lightweight backbone network formed by residual units of an attention mechanism of the road condition analysis model, and outputting three scale feature images, wherein the sizes of the output feature images are 52×52, 26×26 and 13×13 respectively;
the three size feature images are output to an ECSK-Residual module, and the feature images with the scale of 13 multiplied by 13 and the channel number of 24 are output after the ECSK-Residual module and a plurality of convolution operations;
performing Concat operation on the deep features after upsampling and the middle layer features, and outputting a feature map with the scale of 26 multiplied by 26 and the channel number of 24 after convolution operation;
performing Concat operation on the middle layer features after upsampling and the shallow layer features, and outputting feature images with the dimensions of 52 multiplied by 52 and the channel number of 24 after convolution operation;
and dividing S multiplied by S grids on the feature map obtained after the three feature maps are fused, classifying road conditions through a Softmax classifier, and outputting a road condition analysis result.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the standardized road condition image is input into a road condition analysis model, and feature extraction is performed through a lightweight backbone network formed by residual units of an attention mechanism of the road condition analysis model, and three scale feature graphs are output, including:
inputting the standardized road condition image into a road condition analysis model, wherein the standardized road condition image is H multiplied by W multiplied by C, and the size and the channel number are kept unchanged after 2-layer convolution processing;
the processed image generates a 1 multiplied by C feature map through a global average pooling layer, the feature map becomes 1 multiplied by C/r after being input into a 1 st full-connection layer, the feature map is restored to 1 multiplied by C after being input into a 2 nd full-connection layer after nonlinear transformation by using an activation function ReLU, wherein r is a scaling coefficient, and the value is 16;
the 1 multiplied by C images processed by the full connection layer generates a weight coefficient matrix through a Sigmoid function, and then the weight coefficient is multiplied by the corresponding channel number to generate an H multiplied by W multiplied by C characteristic diagram;
and adding the convolution output image and the H multiplied by W multiplied by C characteristic diagram to obtain the corresponding size characteristic.
Optionally, in a sixth implementation manner of the first aspect of the present invention, when the road condition analysis result is a congestion type, after sending traffic conditions and interval information of a congested road section to a traffic guidance platform, the method further includes:
acquiring upstream and downstream road condition images of a congestion zone, and extracting spatial information of traffic data by adopting a graph SAGE (graph SAGE) graph neural network;
taking the output result of the GraphSAGE map neural network and the time information of the upstream and downstream road condition images as the input of an LSTM network, predicting the traffic road conditions in time sequence and space through the LSTM network, and outputting the prediction result;
and sending the prediction result to a traffic command platform.
The invention provides a machine vision-based high-speed road condition analysis system, which comprises an image acquisition module, an image preprocessing module, a road condition analysis module and an information sending module, wherein the image acquisition module is used for dividing a highway into a plurality of sections according to a monitoring bayonet as a node and acquiring real-time road condition images shot by monitoring equipment in each section;
the image preprocessing module is used for preprocessing the real-time road condition image to obtain a preprocessed high-speed road condition image, scaling the preprocessed high-speed road condition image, and adjusting the preprocessed high-speed road condition image to be of a uniform size to obtain a standardized road condition image;
the road condition analysis module is used for inputting the standardized road condition image into a road condition analysis model, and analyzing the high-speed road condition through the road condition analysis model to obtain a road condition analysis result;
and the information sending module is used for sending the traffic condition and the interval information of the congested road section to the traffic command platform when the road condition analysis result is of the congestion type.
Optionally, in a first implementation manner of the second aspect of the present invention, the image preprocessing module includes a distortion correction sub-module, an edge detection sub-module and a target detection sub-module, where the distortion correction sub-module is configured to obtain a real-time road condition image captured by a monitoring device, perform distortion correction processing on the real-time road condition image, and project a point coordinate after de-distortion to a pixel screen to obtain a first road condition image;
the edge detection sub-module is used for sequentially carrying out bilateral filtering and Canny edge detection processing on the first road condition image, extracting edge characteristics of the first road condition image and obtaining a second road condition image;
and the target detection sub-module is used for carrying out target detection on the second road condition image by adopting the probability Hough transformation to obtain a preprocessed high-speed road condition image.
Optionally, in a second implementation manner of the second aspect of the present invention, the road condition analysis module includes a feature extraction sub-module, a first convolution sub-module, a second convolution sub-module, a third convolution sub-module, and a feature map sub-module, where the feature extraction sub-module is configured to input the standardized road condition image into a road condition analysis model, perform feature extraction through a lightweight backbone network formed by residual units of an attention mechanism of the road condition analysis model, and output three scale feature maps;
the first convolution submodule is used for outputting three size feature graphs to the ECSK-Residual module, and outputting feature graphs with 13X 13 scale and 24 channel number after the ECSK-Residual module and a plurality of convolution operations;
the second convolution sub-module is used for performing Concat operation on the deep features after upsampling and the middle layer features, and outputting feature images with the dimensions of 26 multiplied by 26 and the channel number of 24 after convolution operation;
the third convolution sub-module is used for performing Concat operation on the middle layer features after up-sampling and the shallow layer features, and outputting feature images with the dimensions of 52 multiplied by 52 and the channel number of 24 after convolution operation;
and the feature map dividing sub-module is used for dividing S multiplied by S grids on the feature map obtained after the three feature maps are fused, classifying road conditions through a Softmax classifier and outputting a road condition analysis result.
According to the technical scheme provided by the invention, the expressway is divided into a plurality of intervals according to the monitoring bayonets as nodes, and the real-time road condition image shot by the monitoring equipment in each interval is obtained; preprocessing the real-time road condition image to obtain a preprocessed high-speed road condition image, scaling the preprocessed high-speed road condition image to be uniform in size, and obtaining a standardized road condition image; inputting the standardized road condition image into a road condition analysis model, and analyzing the high-speed road condition through the road condition analysis model to obtain a road condition analysis result; when the road condition analysis result is of a congestion type, the traffic condition and interval information of the congested road section are sent to a traffic command platform; according to the invention, the high-speed road condition images are collected, the high-speed road condition is analyzed, the high-speed congestion condition is known in time, the support is provided for relieving the congestion, and the travel quality of the user is improved.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention.
Fig. 1 is a schematic diagram of a first embodiment of a machine vision-based high-speed road condition analysis method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a second embodiment of a machine vision-based high-speed road condition analysis method according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a third embodiment of a machine vision-based high-speed road condition analysis method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a high-speed road condition analysis system based on machine vision according to an embodiment of the present invention.
Detailed Description
The terms "first," "second," "third," "fourth" and the like in the description and in the claims and in the above drawings, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
For easy understanding, the following describes a specific flow of an embodiment of the present invention, please refer to fig. 1 for a first embodiment of a machine vision-based high-speed road condition analysis method, which specifically includes the following steps:
step 101, dividing a highway into a plurality of sections according to monitoring bayonets as nodes, and acquiring real-time road condition images shot by monitoring equipment in each section;
102, preprocessing a real-time road condition image to obtain a preprocessed high-speed road condition image, scaling the preprocessed high-speed road condition image to be uniform in size, and obtaining a standardized road condition image;
step 103, inputting the standardized road condition image into a road condition analysis model, and analyzing the high-speed road condition through the road condition analysis model to obtain a road condition analysis result;
in the embodiment, an upstream road condition image and a downstream road condition image of a congestion zone are obtained, and a graph SAGE (graphical SAGE) graph neural network is adopted to extract space information of traffic data; taking the output result of the GraphSAGE map neural network and the time information of the upstream and downstream road condition images as the input of an LSTM network, predicting the traffic road conditions in time sequence and space through the LSTM network, and outputting the prediction result; and sending the prediction result to the traffic command platform.
And 104, when the road condition analysis result is the congestion type, transmitting the traffic condition and the interval information of the congested road section to the traffic command platform.
In the embodiment, a highway is divided into a plurality of intervals according to monitoring bayonets as nodes, and real-time road condition images shot by monitoring equipment in each interval are obtained; preprocessing the real-time road condition image to obtain a preprocessed high-speed road condition image, scaling the preprocessed high-speed road condition image to be uniform in size, and obtaining a standardized road condition image; inputting the standardized road condition image into a road condition analysis model, and analyzing the high-speed road condition through the road condition analysis model to obtain a road condition analysis result; when the road condition analysis result is of a congestion type, the traffic condition and interval information of the congested road section are sent to a traffic command platform; the invention collects the high-speed road condition images, analyzes the high-speed road condition, timely knows the high-speed congestion condition, provides support for relieving the congestion, and improves the travel quality of users.
Referring to fig. 2, a second embodiment of a machine vision-based high-speed road condition analysis method according to an embodiment of the present invention is shown, and the method includes:
step 201, acquiring a real-time road condition image shot by monitoring equipment, performing distortion correction processing on the real-time road condition image, and projecting the point coordinates after distortion removal to a pixel screen to obtain a first road condition image;
step 202, sequentially performing bilateral filtering and Canny edge detection processing on a first road condition image, and extracting edge characteristics of the first road condition image to obtain a second road condition image;
in the embodiment, a bilateral filter is adopted to perform noise reduction treatment on the first road condition image, so as to obtain a noise-reduced first road condition image; respectively carrying out a Sobel horizontal operator and a Sobel vertical operator on the first road condition image after noise reduction to obtain image gradient information, wherein the image gradient information comprises a horizontal gradient component and a vertical gradient component; performing non-maximum value inhibition processing and double threshold screening on the noise-reduced first road condition image based on the image gradient information to obtain pixel information, wherein the pixel information comprises strong edge pixels, weak edge pixels and pixel points needing to be inhibited; and carrying out edge tracking on the first road condition image after noise reduction based on the pixel information, extracting edge characteristics, and obtaining a second road condition image.
And 203, performing target detection on the second road condition image by adopting probability Hough transformation to obtain a preprocessed high-speed road condition image.
In this embodiment, traversing edge points in the second road condition image, performing hough transformation on the edge points, and selecting a point with the largest value in the hough space to obtain the maximum value; determining left and right endpoints along a straight line direction based on the maximum value to obtain a detection line segment, and obtaining an intersection point of line segments adjacent to a certain angle threshold value of the detection line segment to obtain a candidate point set; calculating the distance between candidate points in the candidate point set by adopting Euclidean distance, combining the points with the minimum distance value, and carrying out hierarchical clustering to obtain vanishing points; and selecting a target detection area of the high-speed road surface in the second road condition image based on the vanishing point to obtain the preprocessed high-speed road condition image.
Referring to fig. 3, a third embodiment of a machine vision-based high-speed road condition analysis method according to an embodiment of the present invention is shown, and the method includes:
step 301, inputting a standardized road condition image into a road condition analysis model, extracting features through a lightweight backbone network formed by residual units of an attention mechanism of the road condition analysis model, and outputting three scale feature graphs;
in the present embodiment, the output feature map sizes are 52×52, 26×26, 13×13, respectively; in the embodiment, a standardized road condition image is input into a road condition analysis model, the standardized road condition image is H multiplied by W multiplied by C, and the size and the channel number are kept unchanged after 2-layer convolution processing; the processed image generates a 1 multiplied by C feature map through a global average pooling layer, the feature map becomes 1 multiplied by C/r after being input into a 1 st full-connection layer, the feature map is restored to 1 multiplied by C after being input into a 2 nd full-connection layer after nonlinear transformation by using an activation function ReLU, wherein r is a scaling coefficient, and the value is 16; the 1 multiplied by C images processed by the full connection layer generates a weight coefficient matrix through a Sigmoid function, and then the weight coefficient is multiplied by the corresponding channel number to generate an H multiplied by W multiplied by C characteristic diagram; and adding the convolution output image and the H multiplied by W multiplied by C characteristic diagram to obtain a corresponding size characteristic diagram.
Step 302, outputting the three size feature graphs to an ECSK-Residual module, and outputting feature graphs with 13X 13 scale and 24 channel number after the ECSK-Residual module and a plurality of convolution operations;
step 303, performing Concat operation on the deep features after upsampling and the middle layer features, and outputting a feature map with the scale of 26 multiplied by 26 and the channel number of 24 after convolution operation;
step 304, performing Concat operation on the middle layer feature after upsampling and the shallow layer feature, and outputting a feature map with the scale of 52×52 and the channel number of 24 after convolution operation;
and 305, dividing S multiplied by S grids on the feature map obtained after the three feature maps are fused, classifying road conditions through a Softmax classifier, and outputting a road condition analysis result.
Referring to fig. 4, a schematic structural diagram of a machine vision-based high-speed road condition analysis system provided by an embodiment of the present invention includes an image acquisition module, an image preprocessing module, a road condition analysis module and an information sending module, where the image acquisition module 401 is configured to divide a highway into a plurality of sections according to monitoring bayonets as nodes, and acquire real-time road condition images captured by monitoring devices in each section;
the image preprocessing module 402 is configured to preprocess the real-time road condition image to obtain a preprocessed high-speed road condition image, scale the preprocessed high-speed road condition image to a uniform size, and obtain a standardized road condition image;
the road condition analysis module 403 is configured to input the standardized road condition image into a road condition analysis model, analyze the high-speed road condition through the road condition analysis model, and obtain a road condition analysis result;
and the information sending module 404 is configured to send traffic conditions and interval information of the congested road segments to the traffic guidance platform when the road condition analysis result is a congestion type.
In this embodiment, the image preprocessing module includes a distortion correction sub-module, an edge detection sub-module and a target detection sub-module, where the distortion correction sub-module is configured to obtain a real-time road condition image captured by the monitoring device, perform distortion correction processing on the real-time road condition image, and project the point coordinates after distortion removal to a pixel screen to obtain a first road condition image;
the edge detection sub-module is used for sequentially carrying out bilateral filtering and Canny edge detection processing on the first road condition image, extracting edge characteristics of the first road condition image and obtaining a second road condition image;
and the target detection sub-module is used for carrying out target detection on the second road condition image by adopting probability Hough transformation to obtain a preprocessed high-speed road condition image.
In this embodiment, the road condition analysis module includes a feature extraction sub-module, a first convolution sub-module, a second convolution sub-module, a third convolution sub-module, and a feature map sub-module, where the feature extraction sub-module is configured to input a standardized road condition image into the road condition analysis model, perform feature extraction through a lightweight backbone network formed by residual units of an attention mechanism of the road condition analysis model, and output three scale feature maps;
the first convolution submodule is used for outputting three size feature graphs to the ECSK-Residual module, and outputting feature graphs with 13X 13 scale and 24 channel number after the ECSK-Residual module and a plurality of convolution operations;
the second convolution sub-module is used for performing Concat operation on the deep features after upsampling and the middle layer features, and outputting feature images with the dimensions of 26 multiplied by 26 and the channel number of 24 after convolution operation;
the third convolution sub-module is used for performing Concat operation on the middle layer features after up-sampling and the shallow layer features, and outputting feature images with the dimensions of 52 multiplied by 52 and the channel number of 24 after convolution operation;
and the feature map dividing sub-module is used for dividing S multiplied by S grids on the feature map obtained after the three feature maps are fused, classifying road conditions through a Softmax classifier and outputting a road condition analysis result.
Through implementation of the scheme, the system comprises an image acquisition module, an image preprocessing module, a road condition analysis module and an information sending module; the invention collects the high-speed road condition images, analyzes the high-speed road condition, timely knows the high-speed congestion condition, provides support for relieving the congestion, and improves the travel quality of users.
The foregoing has shown and described the basic principles, principal features and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the above-described embodiments, and that the above-described embodiments and descriptions are only preferred embodiments of the present invention, and are not intended to limit the invention, and that various changes and modifications may be made therein without departing from the spirit and scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (10)
1. The high-speed road condition analysis method based on the machine vision is characterized by comprising the following steps of:
dividing the expressway into a plurality of intervals according to the monitoring bayonets as nodes, and acquiring real-time road condition images shot by monitoring equipment in each interval;
preprocessing the real-time road condition image to obtain a preprocessed high-speed road condition image, scaling the preprocessed high-speed road condition image to be uniform in size, and obtaining a standardized road condition image;
inputting the standardized road condition image into a road condition analysis model, and analyzing the high-speed road condition through the road condition analysis model to obtain a road condition analysis result;
and when the road condition analysis result is of a congestion type, transmitting traffic conditions and interval information of the congested road sections to a traffic command platform.
2. The machine vision based high-speed road condition analysis method of claim 1, wherein preprocessing the real-time road condition image to obtain a preprocessed high-speed road condition image comprises:
acquiring a real-time road condition image shot by monitoring equipment, performing distortion correction processing on the real-time road condition image, and projecting the point coordinates after distortion removal to a pixel screen to obtain a first road condition image;
sequentially carrying out bilateral filtering and Canny edge detection on the first road condition image, extracting edge characteristics of the first road condition image, and obtaining a second road condition image;
and performing target detection on the second road condition image by adopting probability Hough transformation to obtain a preprocessed high-speed road condition image.
3. The machine vision based high-speed road condition analysis method according to claim 2, wherein sequentially performing bilateral filtering and Canny edge detection processing on the first road condition image, extracting edge features of the first road condition image, and obtaining a second road condition image, comprises:
adopting a bilateral filter to perform noise reduction treatment on the first road condition image to obtain a noise-reduced first road condition image;
respectively carrying out a Sobel horizontal operator and a Sobel vertical operator on the first road condition image after noise reduction to obtain image gradient information, wherein the image gradient information comprises a horizontal gradient component and a vertical gradient component;
performing non-maximum value inhibition processing and double threshold screening on the first road condition image after noise reduction based on the image gradient information to obtain pixel information, wherein the pixel information comprises strong edge pixels, weak edge pixels and pixel points needing to be inhibited;
and carrying out edge tracking on the first road condition image after noise reduction based on the pixel information, extracting edge characteristics, and obtaining a second road condition image.
4. The machine vision based high-speed road condition analysis method according to claim 2, wherein performing target detection on the second road condition image by using the probability hough transform to obtain a preprocessed high-speed road condition image comprises:
traversing edge points in the second road condition image, carrying out Hough transformation on the edge points, and selecting a point with the largest value in a Hough space to obtain the maximum value;
determining left and right endpoints along a straight line direction based on the maximum value to obtain a detection line segment, and obtaining an intersection point of line segments adjacent to a certain angle threshold value of the detection line segment to obtain a candidate point set;
calculating the distance of the candidate points in the candidate point set by adopting Euclidean distance, combining the points with the minimum distance value, and carrying out hierarchical clustering to obtain vanishing points;
and selecting a target detection area of the high-speed road surface in the second road condition image based on the vanishing point to obtain a preprocessed high-speed road condition image.
5. The machine vision based high-speed road condition analysis method according to claim 1, wherein inputting the standardized road condition image into a road condition analysis model, analyzing the high-speed road condition through the road condition analysis model to obtain a road condition analysis result, comprises:
inputting the standardized road condition image into a road condition analysis model, extracting features through a lightweight backbone network formed by residual units of an attention mechanism of the road condition analysis model, and outputting three scale feature images, wherein the sizes of the output feature images are 52×52, 26×26 and 13×13 respectively;
the three size feature images are output to an ECSK-Residual module, and the feature images with the scale of 13 multiplied by 13 and the channel number of 24 are output after the ECSK-Residual module and a plurality of convolution operations;
performing Concat operation on the deep features after upsampling and the middle layer features, and outputting a feature map with the scale of 26 multiplied by 26 and the channel number of 24 after convolution operation;
performing Concat operation on the middle layer features after upsampling and the shallow layer features, and outputting feature images with the dimensions of 52 multiplied by 52 and the channel number of 24 after convolution operation;
and dividing S multiplied by S grids on the feature map obtained after the three feature maps are fused, classifying road conditions through a Softmax classifier, and outputting a road condition analysis result.
6. The machine vision based high-speed road condition analysis method according to claim 5, wherein the standardized road condition image is input into a road condition analysis model, feature extraction is performed through a lightweight backbone network composed of residual units of an attention mechanism of the road condition analysis model, and three scale feature graphs are output, including:
inputting the standardized road condition image into a road condition analysis model, wherein the standardized road condition image is H multiplied by W multiplied by C, and the size and the channel number are kept unchanged after 2-layer convolution processing;
the processed image generates a 1 multiplied by C feature map through a global average pooling layer, the feature map becomes 1 multiplied by C/r after being input into a 1 st full-connection layer, the feature map is restored to 1 multiplied by C after being input into a 2 nd full-connection layer after nonlinear transformation by using an activation function ReLU, wherein r is a scaling coefficient, and the value is 16;
the 1 multiplied by C images processed by the full connection layer generates a weight coefficient matrix through a Sigmoid function, and then the weight coefficient is multiplied by the corresponding channel number to generate an H multiplied by W multiplied by C characteristic diagram;
and adding the convolution output image and the H multiplied by W multiplied by C characteristic diagram to obtain a corresponding size characteristic diagram.
7. The machine vision based high-speed road condition analysis method according to claim 1, wherein when the road condition analysis result is a congestion type, after sending traffic conditions and section information of a congested road section to a traffic guidance platform, further comprising:
acquiring upstream and downstream road condition images of a congestion zone, and extracting spatial information of traffic data by adopting a graph SAGE (graph SAGE) graph neural network;
taking the output result of the GraphSAGE map neural network and the time information of the upstream and downstream road condition images as the input of an LSTM network, predicting the traffic road conditions in time sequence and space through the LSTM network, and outputting the prediction result;
and sending the prediction result to a traffic command platform.
8. The system is characterized by comprising an image acquisition module, an image preprocessing module, a road condition analysis module and an information sending module, wherein the image acquisition module is used for dividing a highway into a plurality of intervals according to monitoring bayonets as nodes and acquiring real-time road condition images shot by monitoring equipment in each interval;
the image preprocessing module is used for preprocessing the real-time road condition image to obtain a preprocessed high-speed road condition image, scaling the preprocessed high-speed road condition image, and adjusting the preprocessed high-speed road condition image to be of a uniform size to obtain a standardized road condition image;
the road condition analysis module is used for inputting the standardized road condition image into a road condition analysis model, and analyzing the high-speed road condition through the road condition analysis model to obtain a road condition analysis result;
and the information sending module is used for sending the traffic condition and the interval information of the congested road section to the traffic command platform when the road condition analysis result is of the congestion type.
9. The machine vision based high-speed road condition analysis system according to claim 8, wherein the image preprocessing module comprises a distortion correction sub-module, an edge detection sub-module and a target detection sub-module, wherein the distortion correction sub-module is used for acquiring a real-time road condition image shot by monitoring equipment, performing distortion correction processing on the real-time road condition image, and projecting the point coordinates after distortion removal to a pixel screen to obtain a first road condition image;
the edge detection sub-module is used for sequentially carrying out bilateral filtering and Canny edge detection processing on the first road condition image, extracting edge characteristics of the first road condition image and obtaining a second road condition image;
and the target detection sub-module is used for carrying out target detection on the second road condition image by adopting probability Hough transformation to obtain a preprocessed high-speed road condition image.
10. The machine vision based high-speed road condition analysis system according to claim 8, wherein the road condition analysis module comprises a feature extraction sub-module, a first convolution sub-module, a second convolution sub-module, a third convolution sub-module and a feature map sub-module, wherein the feature extraction sub-module is used for inputting the standardized road condition image into a road condition analysis model, extracting features through a lightweight backbone network consisting of residual units of an attention mechanism of the road condition analysis model, and outputting three scale feature maps;
the first convolution submodule is used for outputting three size feature graphs to the ECSK-Residual module, and outputting feature graphs with 13X 13 scale and 24 channel number after the ECSK-Residual module and a plurality of convolution operations;
the second convolution sub-module is used for performing Concat operation on the deep features after upsampling and the middle layer features, and outputting feature images with the dimensions of 26 multiplied by 26 and the channel number of 24 after convolution operation;
the third convolution sub-module is used for performing Concat operation on the middle layer features after up-sampling and the shallow layer features, and outputting feature images with the dimensions of 52 multiplied by 52 and the channel number of 24 after convolution operation;
and the feature map dividing sub-module is used for dividing S multiplied by S grids on the feature map obtained after the three feature maps are fused, classifying road conditions through a Softmax classifier and outputting a road condition analysis result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311589326.8A CN117292552B (en) | 2023-11-27 | 2023-11-27 | High-speed road condition analysis system and method based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311589326.8A CN117292552B (en) | 2023-11-27 | 2023-11-27 | High-speed road condition analysis system and method based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117292552A true CN117292552A (en) | 2023-12-26 |
| CN117292552B CN117292552B (en) | 2024-02-09 |
Family
ID=89257588
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311589326.8A Active CN117292552B (en) | 2023-11-27 | 2023-11-27 | High-speed road condition analysis system and method based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117292552B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118644991A (en) * | 2024-08-16 | 2024-09-13 | 山东高速股份有限公司 | A road condition judgment method, device, equipment, storage medium and product |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100034409A (en) * | 2008-09-24 | 2010-04-01 | 금오공과대학교 산학협력단 | Method for recognizing lane and lane departure with single lane extraction |
| CN110176143A (en) * | 2019-06-10 | 2019-08-27 | 长安大学 | A kind of highway traffic congestion detection method based on deep learning algorithm |
| CN113129585A (en) * | 2021-03-05 | 2021-07-16 | 浙江工业大学 | Road traffic flow prediction method based on graph aggregation mechanism of reconstructed traffic network |
-
2023
- 2023-11-27 CN CN202311589326.8A patent/CN117292552B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100034409A (en) * | 2008-09-24 | 2010-04-01 | 금오공과대학교 산학협력단 | Method for recognizing lane and lane departure with single lane extraction |
| CN110176143A (en) * | 2019-06-10 | 2019-08-27 | 长安大学 | A kind of highway traffic congestion detection method based on deep learning algorithm |
| CN113129585A (en) * | 2021-03-05 | 2021-07-16 | 浙江工业大学 | Road traffic flow prediction method based on graph aggregation mechanism of reconstructed traffic network |
Non-Patent Citations (1)
| Title |
|---|
| 董莹莹: "基于机器视觉的车辆和车道线检测研究", 中国优秀硕士学位论文全文数据库 (工程科技Ⅱ辑), pages 035 - 291 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118644991A (en) * | 2024-08-16 | 2024-09-13 | 山东高速股份有限公司 | A road condition judgment method, device, equipment, storage medium and product |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117292552B (en) | 2024-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110210451B (en) | A zebra crossing detection method | |
| CN107239778B (en) | Efficient and accurate license plate recognition method | |
| CN103985182B (en) | A kind of bus passenger flow automatic counting method and automatic counter system | |
| CN108615358A (en) | A kind of congestion in road detection method and device | |
| Tian et al. | A two-stage character segmentation method for Chinese license plate | |
| CN104700099A (en) | Method and device for recognizing traffic signs | |
| CN108647572A (en) | A kind of lane departure warning method based on Hough transformation | |
| CN112990065B (en) | Vehicle classification detection method based on optimized YOLOv5 model | |
| CN114092917B (en) | MR-SSD-based shielded traffic sign detection method and system | |
| CN101339616A (en) | Roads recognition method and apparatus | |
| CN110705342A (en) | Lane line segmentation detection method and device | |
| CN110298300A (en) | A method of detection vehicle violation crimping | |
| CN105389561B (en) | A kind of bus zone detection method based on video | |
| CN106845444A (en) | A kind of vehicle well cover detection method combined based on acnode | |
| CN117292552B (en) | High-speed road condition analysis system and method based on machine vision | |
| CN101369312B (en) | Method and equipment for detecting intersection in image | |
| CN109543613A (en) | Vehicle Speed and Vehicle License Plate Recognition System and method based on TOF imaging | |
| CN112464914A (en) | Guardrail segmentation method based on convolutional neural network | |
| CN116152758A (en) | Intelligent real-time accident detection and vehicle tracking method | |
| CN114581886A (en) | Visibility discrimination method, device and medium combining semantic segmentation and frequency domain analysis | |
| Zhang et al. | A front vehicle detection algorithm for intelligent vehicle based on improved gabor filter and SVM | |
| Khoshki et al. | Improved Automatic License Plate Recognition (ALPR) system based on single pass Connected Component Labeling (CCL) and reign property function | |
| CN111860113A (en) | Lane line detection method and system | |
| CN109522851B (en) | Vehicle speed and license plate recognition method based on non-visible light imaging | |
| CN113850112B (en) | Road condition recognition method and system based on twin neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |