CN111382704B - Vehicle line pressing violation judging method and device based on deep learning and storage medium - Google Patents
Vehicle line pressing violation judging method and device based on deep learning and storage medium Download PDFInfo
- Publication number
- CN111382704B CN111382704B CN202010163795.3A CN202010163795A CN111382704B CN 111382704 B CN111382704 B CN 111382704B CN 202010163795 A CN202010163795 A CN 202010163795A CN 111382704 B CN111382704 B CN 111382704B
- Authority
- CN
- China
- Prior art keywords
- vehicle
- target vehicle
- line
- region
- license plate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000013135 deep learning Methods 0.000 title claims abstract description 29
- 238000001514 detection method Methods 0.000 claims abstract description 22
- 230000008569 process Effects 0.000 claims abstract description 11
- 230000004927 fusion Effects 0.000 claims abstract description 9
- 238000012545 processing Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 238000003708 edge detection Methods 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 claims description 4
- 230000006399 behavior Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 235000008694 Humulus lupulus Nutrition 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
- G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a vehicle line pressing violation judging method, a device and a storage medium based on deep learning, wherein the method comprises the following steps: acquiring video data shot by a traffic camera; detecting lane lines in the video data by adopting a method based on multi-feature fusion, and defining lane line areas according to the lane lines; detecting a target vehicle in the video data by adopting a target detection and target tracking method based on deep learning; determining a vehicle position of a target vehicle according to the vehicle frame, and tracking the target vehicle according to the vehicle position; in the tracking process, judging whether the target vehicle is in line pressing or not according to the lane line area and the vehicle frame; and if the target vehicle is pressed, license plate recognition is carried out on the target vehicle. The method can automatically judge the line pressing behavior of the vehicle against rules and regulations, and meets the requirements of high efficiency and high accuracy in traffic auditing.
Description
Technical Field
The application relates to the technical field of artificial intelligence judgment of traffic violations, in particular to a vehicle line pressing violation judgment method and device based on deep learning and a storage medium.
Background
With the continuous development of social economy and the continuous improvement of the living standard of people, vehicles are more and more, and the automatic auditing of vehicle violations is more and more urgently required by traffic authorities. The traditional violation auditing method mainly adopts methods such as manual identification and the like, has higher cost and lower efficiency, and has subjectivity to a certain extent, thereby influencing the checking accuracy. How to quickly, accurately and efficiently identify traffic offence, and avoid the defects of high cost, low efficiency and the like of manual identification at the same time is a technical problem which needs to be solved urgently.
Disclosure of Invention
Aiming at the technical defects in the prior art, the embodiment of the application aims to provide a vehicle line-pressing violation judging method, device and storage medium based on deep learning, which can automatically judge the line-pressing behavior of the vehicle against rules so as to meet the requirements of high efficiency and high accuracy in traffic auditing.
In order to achieve the above object, in a first aspect, an embodiment of the present application provides a vehicle wire violation determination method based on deep learning, including:
acquiring video data shot by a traffic camera;
detecting lane lines in the video data by adopting a method based on multi-feature fusion, and defining lane line areas according to the lane lines;
detecting a target vehicle in the video data by adopting a target detection and target tracking method based on deep learning;
determining a vehicle position of a target vehicle according to the vehicle frame, and tracking the target vehicle according to the vehicle position;
in the tracking process, judging whether the target vehicle is in line pressing or not according to the lane line area and the vehicle frame;
and if the target vehicle is pressed, license plate recognition is carried out on the target vehicle.
As a specific embodiment of the present application, a method based on multi-feature fusion is used to detect a lane line in the video data, and a lane line area is defined according to the lane line, which specifically includes:
extracting each frame of image in the video and converting the image into a gray level image;
carrying out Gaussian blur processing and Canny edge detection processing on the gray level image to obtain a line-type image;
extracting a region of interest in the line-type image, and extracting lines by using a Hough change technology on the region of interest;
and fitting a straight line according to the extracted line by using a least square method, and demarcating the lane line area.
Specifically, the method detects a target vehicle and a vehicle frame in each frame image of the video data using YOLO-v3, determines a vehicle position from the vehicle frame, and tracks the detected target vehicle using a Deep-SORT tracking algorithm.
As a specific embodiment of the present application, determining whether the target vehicle is in a line according to the lane line area and the vehicle frame specifically includes:
and if the vehicle frame and the lane line area are overlapped and the overlapping degree is larger than a specified threshold value, determining the line pressing of the target vehicle and storing the current frame image of the target vehicle.
As a specific embodiment of the present application, the license plate recognition for the target vehicle specifically includes:
extracting a vehicle picture corresponding to a target vehicle of the line pressing;
carrying out graying, gaussian blur, binarization and edge detection treatment on the vehicle picture to obtain a picture to be extracted;
carrying out license plate positioning according to the picture to be extracted to obtain a license plate region of the target vehicle;
and identifying the license plate area by adopting a Chinese character network, an alphabetic network and an alphanumeric network, and outputting the identified license plate.
The license plate region comprises a horizontal region and a vertical region, and the license plate positioning according to the picture to be extracted specifically comprises the following steps:
a horizontal area determining step: scanning the picture to be extracted from the bottom to the top, and determining the horizontal region by acquiring a first group of continuous lines, wherein the hopping times of each line in the continuous lines are larger than a hopping threshold value, and simultaneously, the continuous lines are larger than the threshold value;
a vertical area determining step: selecting any row in the horizontal region, moving from left to right by using an L-length window, counting the jump times of adjacent pixels 0,1 in the window, storing the jump times in an array, finding the maximum value in the array, and determining the corresponding region as the vertical region; where L is the difference between the highest row and the lowest row in the horizontal region.
In a second aspect, an embodiment of the present application provides a vehicle wire violation judging device based on deep learning, including:
the acquisition unit is used for acquiring video data shot by the traffic camera;
the detection unit is used for detecting lane lines in the video data by adopting a method based on multi-feature fusion and defining lane line areas according to the lane lines;
the detection unit is also used for detecting a target vehicle in the video data and a vehicle frame of the target vehicle by adopting a target detection and target tracking method based on deep learning;
the vehicle frame tracking unit is used for determining the vehicle position of the target vehicle according to the vehicle frame and tracking the target vehicle according to the vehicle position;
the judging unit is used for judging whether the target vehicle is in line pressing or not according to the lane line area and the vehicle frame in the tracking process;
and the identification unit is used for carrying out license plate identification on the target vehicle if the target vehicle is pressed.
In a third aspect, an embodiment of the present application provides another vehicle wire pressing violation determining apparatus based on deep learning, including a processor, an input device, an output device, and a memory, where the processor, the input device, the output device, and the memory are connected to each other, where the memory is configured to store a computer program, where the computer program includes program instructions, and where the processor is configured to invoke the program instructions to perform the method of the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program, wherein the computer program comprises program instructions which, when executed by a processor, cause the processor to perform the method of the first aspect described above.
According to the embodiment of the application, video data are acquired, processed to obtain a target vehicle, the target vehicle is tracked, whether the target vehicle is pressed or not is judged according to the lane line area and the vehicle frame, and if the target vehicle is pressed, license plate recognition is carried out on the target vehicle; namely, the embodiment can automatically judge the line pressing behavior of the vehicle against rules and regulations, and meets the requirements of high efficiency and high accuracy in traffic auditing.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a schematic flow chart of a vehicle wire pressing violation judging method based on deep learning according to an embodiment of the application;
fig. 2 is another flow chart of a vehicle wire pressing violation judging method based on deep learning according to an embodiment of the application;
FIG. 3 is a schematic diagram of a lane line detection flow;
FIG. 4 is a schematic diagram of a license plate recognition process;
FIG. 5 is a schematic diagram of a character recognition process;
fig. 6 is a schematic structural diagram of a vehicle wire pressing violation judging device based on deep learning according to an embodiment of the present application;
fig. 7 is a schematic diagram of another structure of a vehicle wire violation judging device based on deep learning according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
Referring to fig. 1, the vehicle line pressing violation judging method based on deep learning provided by the embodiment of the application mainly comprises the processes of video data acquisition, lane line detection, vehicle detection, line pressing or not and license plate recognition. The lane line detection mainly comprises the steps of graying, gaussian blur, edge detection, demarcating an area of interest, hough transformation and straight line fitting, and then demarcating a lane line area. SORT tracking is then added to accommodate lane line detection at different days.
Further referring to fig. 1 and fig. 2, the vehicle wire pressing violation judging method based on deep learning provided by the embodiment of the application specifically includes:
s101, acquiring video data shot by a traffic camera.
S102, detecting lane lines in the video data by adopting a method based on multi-feature fusion, and demarcating lane line areas according to the lane lines.
Since the angle of the traffic camera is usually fixed, and the photographed road scene is single. In the process of lane line detection, specific steps are shown in fig. 3, and include:
(1) Extracting each frame of image in video data of video data shot by a traffic camera, and converting the video data into a gray level image;
(2) Smoothing edges using gaussian blur on the resulting gray map to reduce the effects of noise;
(3) The method comprises the steps of identifying the edges of an image by using Canny edge detection for the image subjected to Gaussian blur processing, and removing other data to obtain a line-type image;
(4) For the resulting line-type image, a region of interest is delineated and any lines outside this region are discarded;
(5) Extracting lines from the region of interest by using a Hough transform technique;
(6) For the extracted lines, fitting a straight line by using a least square method, and defining a lane line area;
(7) For the extracted lane line region, a SORT tracking algorithm is used to track the lane line region.
S103, detecting a target vehicle in the video data and a vehicle frame of the target vehicle by adopting a target detection and target tracking method based on deep learning.
S104, determining the vehicle position of the target vehicle according to the vehicle frame, and tracking the target vehicle according to the vehicle position.
After the lane line area is defined, a target vehicle and a vehicle frame in each frame image of the video data are detected by using YOLO-v3, the vehicle position is determined according to the vehicle frame, and the detected target vehicle is tracked by using a Deep-SORT tracking algorithm.
And S105, in the tracking process, judging whether the target vehicle is in line pressing or not according to the lane line area and the vehicle frame.
For each target vehicle, if the border line and the lane line area are overlapped and the overlapping degree is larger than a specified threshold value, the vehicle is considered to be pressed, and a picture is saved. Otherwise, if the overlapping degree is not greater than the specified threshold value, the vehicle is considered to be not pressed.
And S106, if the target vehicle is pressed, license plate recognition is carried out on the target vehicle.
If the vehicle is pressed, the pressed vehicle picture is segmented and extracted, and license plate recognition is carried out, wherein the specific steps are shown in figure 4. For each line-pressing vehicle, graying, gaussian blur and binarization are performed first, similar to the steps in lane line detection. The license plate region of the image after proper binarization processing has the following three basic characteristics:
(1) Columns densely contain a plurality of characters in a small area;
(2) Columns densely contain a plurality of characters in a small area;
(3) Columns densely contain a plurality of characters in a small area;
the license plate area may change frequently from 0 to 1 and 1 to 0 between adjacent pixels in the row. Since the license plate is generally suspended from the lower portion of the vehicle, the image is scanned from top to bottom and from left to right. The character part of the license plate consists of 7 character numbers and two vertical frames, and the jump frequency of any row in the license plate area is at least (7+2) times 2=18 times. And scanning from the bottom to the top of the image, wherein the first group of continuous lines and the hopping times of each line are larger than the hopping threshold value, and simultaneously, the continuous lines are larger than a certain threshold value to determine the horizontal area of the license plate.
In the horizontal area of the license plate, the difference between the highest row and the lowest row is the height H of the license plate in the image. The license plate area of China is rectangular, the aspect ratio is about 3.14, and 3.14 x H is taken as the width of the license plate. Any row is selected in the horizontal area, the window with L length is moved from left to right, the jump times of adjacent pixels 0,1 in the window are counted and stored in an array. If the window moves to the vertical area of the license plate, the number of hops in the window should be the largest. Therefore, the maximum value is found in the array, and the corresponding area is the vertical area of the license plate.
After license plate positioning is completed, character segmentation is carried out on the obtained license plate. According to the license plate standard of China, the first character of the license plate is generally Chinese character, the second English capital letter of the license plate, and the third to seventh English capital letters or numbers. We separately designed three small neural networks: the Chinese character network, the letter network and the letter-number network realize the classification and identification of the characters, as shown in figure 5, and output the identified license plate.
According to the vehicle line pressing violation judging method, video data are obtained, the video data are processed to obtain a target vehicle, the target vehicle is tracked, whether the target vehicle is line pressing or not is judged according to the lane line area and the vehicle frame, and if the target vehicle is line pressing, license plate recognition is carried out on the target vehicle; namely, the embodiment can automatically judge the line pressing behavior of the vehicle against rules and regulations, and meets the requirements of high efficiency and high accuracy in traffic auditing.
Based on the same inventive concept, the embodiment of the application provides a vehicle line pressing violation judging device based on deep learning. As shown in fig. 6, the apparatus includes:
an acquisition unit 10 for acquiring video data captured by a traffic camera;
a detection unit 11, configured to detect a lane line in the video data by using a method based on multi-feature fusion, and define a lane line region according to the lane line;
the detection unit 11 is further configured to detect a target vehicle in the video data and a vehicle frame of the target vehicle by using a target detection and target tracking method based on deep learning;
a determining and tracking unit 12, configured to determine a vehicle position of a target vehicle according to the vehicle frame, and track the target vehicle according to the vehicle position;
a judging unit 13, configured to judge whether the target vehicle is in a line pressing state according to the lane line area and the vehicle frame in the tracking process;
and the identification unit 14 is used for carrying out license plate identification on the target vehicle if the target vehicle is pressed.
Wherein, the detection unit 11 is specifically configured to:
extracting each frame of image in the video data and converting the frame of image into a gray level image;
carrying out Gaussian blur processing and Canny edge detection processing on the gray level image to obtain a line-type image;
extracting a region of interest in the line-type image, and extracting lines by using a Hough change technology on the region of interest;
and fitting a straight line according to the extracted line by using a least square method, and demarcating the lane line area.
Further, the detection unit 11 is also used for causing. And detecting the target vehicle and the vehicle frame in each frame of image of the video data by using YOLO-v 3.
Specifically, the judging unit 13 is specifically configured to:
and if the vehicle frame and the lane line area are overlapped and the overlapping degree is larger than a specified threshold value, determining the line pressing of the target vehicle and storing the current frame image of the target vehicle.
Specifically, the identification unit 14 is specifically configured to:
extracting a vehicle picture corresponding to a target vehicle of the line pressing;
carrying out graying, gaussian blur, binarization and edge detection treatment on the vehicle picture to obtain a picture to be extracted;
carrying out license plate positioning according to the picture to be extracted to obtain a license plate region of the target vehicle;
and identifying the license plate area by adopting a Chinese character network, an alphabetic network and an alphanumeric network, and outputting the identified license plate.
The license plate region comprises a horizontal region and a vertical region, and the license plate positioning according to the picture to be extracted specifically comprises the following steps:
a horizontal area determining step: scanning the picture to be extracted from the bottom to the top, and determining the horizontal region by acquiring a first group of continuous lines, wherein the hopping times of each line in the continuous lines are larger than a hopping threshold value, and simultaneously, the continuous lines are larger than the threshold value;
a vertical area determining step: selecting any row in the horizontal region, moving from left to right by using an L-length window, counting the jump times of adjacent pixels 0,1 in the window, storing the jump times in an array, finding the maximum value in the array, and determining the corresponding region as the vertical region; where L is the difference between the highest row and the lowest row in the horizontal region.
Alternatively, as shown in fig. 7, in another preferred embodiment of the present application, the vehicle wire violation judging device may include: one or more processors 101, one or more input devices 102, one or more output devices 103, and a memory 104, the processors 101, input devices 102, output devices 103, and memory 104 being interconnected by a bus 105. The memory 104 is used for storing a computer program comprising program instructions, the processor 101 being configured to invoke the program instructions to perform the method of the above-described vehicle wire tie violation determination method embodiment part based on deep learning.
It should be appreciated that in embodiments of the present application, the processor 101 may be a central processing unit (Central Processing Unit, CPU), which may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The input device 102 may include a keyboard or the like, and the output device 103 may include a display (LCD or the like), a speaker or the like.
The memory 104 may include read only memory and random access memory and provides instructions and data to the processor 101. A portion of the memory 104 may also include non-volatile random access memory. For example, the memory 104 may also store information of device type.
In a specific implementation, the processor 101, the input device 102, and the output device 103 described in the embodiments of the present application may execute the implementation described in the embodiments of the method for determining the rule violation of the vehicle wire based on deep learning provided in the embodiments of the present application, which is not described herein again.
It should be noted that, for the specific workflow of the violation determination device, reference is made to the foregoing method embodiment, and details are not repeated here.
By implementing the vehicle line regulation violation judging device provided by the embodiment of the application, the vehicle line regulation violation can be rapidly and accurately judged, and the working efficiency of checking the traffic regulation violation is improved.
Further, corresponding to the vehicle wire violation judging method based on deep learning of the first embodiment, the embodiment of the application further provides a readable storage medium storing a computer program, the computer program comprising program instructions which when executed by a processor realize: the vehicle line pressing violation judging method based on deep learning.
The computer readable storage medium may be an internal storage unit of the client according to the foregoing embodiment, for example, a hard disk or a memory of the system. The computer readable storage medium may also be an external storage device of the system, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the system. Further, the computer readable storage medium may also include both internal storage units and external storage devices of the system. The computer readable storage medium is used to store the computer program and other programs and data required by the system. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided by the present application, it should be understood that the disclosed units and methods may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices, or elements, or may be an electrical, mechanical, or other form of connection.
While the application has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (8)
1. The vehicle line pressing violation judging method based on deep learning is characterized by comprising the following steps of:
acquiring video data shot by a traffic camera;
detecting lane lines in the video data by adopting a method based on multi-feature fusion, and defining lane line areas according to the lane lines;
detecting a target vehicle in the video data by adopting a target detection and target tracking method based on deep learning;
determining a vehicle position of a target vehicle according to the vehicle frame, and tracking the target vehicle according to the vehicle position;
in the tracking process, judging whether the target vehicle is in line pressing or not according to the lane line area and the vehicle frame;
if the target vehicle is pressed, license plate recognition is carried out on the target vehicle, specifically:
extracting a vehicle picture corresponding to a target vehicle of the line pressing;
carrying out graying, gaussian blur, binarization and edge detection treatment on the vehicle picture to obtain a picture to be extracted;
carrying out license plate positioning according to the picture to be extracted to obtain a license plate region of the target vehicle;
identifying the license plate area by adopting a Chinese character network, an alphabetic network and an alphanumeric network, and outputting the identified license plate;
the license plate region comprises a horizontal region and a vertical region, and the license plate positioning according to the picture to be extracted specifically comprises the following steps:
a horizontal area determining step: scanning the picture to be extracted from the bottom to the top, and determining the horizontal region by acquiring a first group of continuous lines, wherein the hopping times of each line in the continuous lines are larger than a hopping threshold value, and simultaneously, the continuous lines are larger than the threshold value;
a vertical area determining step: selecting any row in the horizontal region, moving from left to right by using an L-length window, counting the jump times of adjacent pixels 0,1 in the window, storing the jump times in an array, finding the maximum value in the array, and determining the corresponding region as the vertical region; where L is the difference between the highest row and the lowest row in the horizontal region.
2. The vehicle traffic violation judging method according to claim 1, wherein a lane line in the video data is detected by a method based on multi-feature fusion, and a lane line area is defined according to the lane line, and the method specifically comprises:
extracting each frame of image in the video data and converting the frame of image into a gray level image;
carrying out Gaussian blur processing and Canny edge detection processing on the gray level image to obtain a line-type image;
extracting a region of interest in the line-type image, and extracting lines by using a Hough change technology on the region of interest;
and fitting a straight line according to the extracted line by using a least square method, and demarcating the lane line area.
3. The vehicle wire violation judging method of claim 1, wherein the method specifically comprises:
detecting a target vehicle and a vehicle frame in each frame of image of the video data using YOLO-v3, determining a vehicle position according to the vehicle frame, and tracking the detected target vehicle using a Deep-SORT tracking algorithm.
4. The vehicle line pressing violation judging method of claim 1, wherein judging whether the target vehicle is pressed according to the lane line area and a vehicle frame, specifically comprises:
and if the vehicle frame and the lane line area are overlapped and the overlapping degree is larger than a specified threshold value, determining the line pressing of the target vehicle and storing the current frame image of the target vehicle.
5. Vehicle line ball violation judging device based on degree of depth study, its characterized in that includes:
the acquisition unit is used for acquiring video data shot by the traffic camera;
the detection unit is used for detecting lane lines in the video data by adopting a method based on multi-feature fusion and defining lane line areas according to the lane lines;
the detection unit is also used for detecting a target vehicle in the video data and a vehicle frame of the target vehicle by adopting a target detection and target tracking method based on deep learning;
the vehicle frame tracking unit is used for determining the vehicle position of the target vehicle according to the vehicle frame and tracking the target vehicle according to the vehicle position;
the judging unit is used for judging whether the target vehicle is in line pressing or not according to the lane line area and the vehicle frame in the tracking process;
the identification unit is used for carrying out license plate identification on the target vehicle if the target vehicle is pressed, and specifically comprises the following steps:
extracting a vehicle picture corresponding to a target vehicle of the line pressing;
carrying out graying, gaussian blur, binarization and edge detection treatment on the vehicle picture to obtain a picture to be extracted;
carrying out license plate positioning according to the picture to be extracted to obtain a license plate region of the target vehicle;
identifying the license plate area by adopting a Chinese character network, an alphabetic network and an alphanumeric network, and outputting the identified license plate;
the license plate region comprises a horizontal region and a vertical region, and the license plate positioning according to the picture to be extracted specifically comprises the following steps:
a horizontal area determining step: scanning the picture to be extracted from the bottom to the top, and determining the horizontal region by acquiring a first group of continuous lines, wherein the hopping times of each line in the continuous lines are larger than a hopping threshold value, and simultaneously, the continuous lines are larger than the threshold value;
a vertical area determining step: selecting any row in the horizontal region, moving from left to right by using an L-length window, counting the jump times of adjacent pixels 0,1 in the window, storing the jump times in an array, finding the maximum value in the array, and determining the corresponding region as the vertical region; where L is the difference between the highest row and the lowest row in the horizontal region.
6. The vehicle wire pressing violation judging device based on deep learning as claimed in claim 5, wherein the detecting unit is specifically configured to:
extracting each frame of image in the video and converting the image into a gray level image;
carrying out Gaussian blur processing and Canny edge detection processing on the gray level image to obtain a line-type image;
extracting a region of interest in the line-type image, and extracting lines by using a Hough change technology on the region of interest;
and fitting a straight line according to the extracted line by using a least square method, and demarcating the lane line area.
7. A vehicle wire violation determination apparatus based on deep learning, comprising a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory being interconnected, wherein the memory is configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method of claim 4.
8. A computer readable storage medium storing a computer program, characterized in that the computer program comprises program instructions which, when executed by a processor, cause the processor to perform the method of claim 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010163795.3A CN111382704B (en) | 2020-03-10 | 2020-03-10 | Vehicle line pressing violation judging method and device based on deep learning and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010163795.3A CN111382704B (en) | 2020-03-10 | 2020-03-10 | Vehicle line pressing violation judging method and device based on deep learning and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111382704A CN111382704A (en) | 2020-07-07 |
| CN111382704B true CN111382704B (en) | 2023-12-15 |
Family
ID=71222666
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010163795.3A Active CN111382704B (en) | 2020-03-10 | 2020-03-10 | Vehicle line pressing violation judging method and device based on deep learning and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111382704B (en) |
Families Citing this family (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111914669B (en) * | 2020-07-08 | 2024-08-06 | 浙江大华技术股份有限公司 | Tidal lane violation detection method and device, electronic equipment and storage medium |
| CN113989731A (en) * | 2020-07-27 | 2022-01-28 | 阿里巴巴集团控股有限公司 | Information detection method, computing device and storage medium |
| CN114066931A (en) * | 2020-07-31 | 2022-02-18 | 复旦大学 | Image enhancement method using target tracking sequence |
| CN112101156B (en) * | 2020-09-02 | 2024-08-27 | 杭州海康威视数字技术股份有限公司 | Target identification method and device and electronic equipment |
| CN114255597B (en) * | 2020-09-23 | 2023-07-07 | 顺丰科技有限公司 | Emergency lane occupation behavior detection method and device, electronic equipment and storage medium |
| CN112270245B (en) * | 2020-10-23 | 2023-05-26 | 平安科技（深圳）有限公司 | Method, device, equipment and computer medium for detecting driving illegal |
| CN112580516A (en) * | 2020-12-21 | 2021-03-30 | 上海眼控科技股份有限公司 | Road scene recognition method, device, equipment and storage medium |
| CN112949470B (en) * | 2021-02-26 | 2024-12-03 | 上海商汤智能科技有限公司 | Vehicle lane change turn signal recognition method, device, equipment and storage medium |
| CN113076851B (en) * | 2021-03-29 | 2024-05-14 | 开易（北京）科技有限公司 | Method and device for collecting vehicle violation data and computer equipment |
| CN113487878A (en) * | 2021-07-12 | 2021-10-08 | 重庆长安新能源汽车科技有限公司 | Motor vehicle illegal line pressing running detection method and system |
| CN113851001B (en) * | 2021-09-17 | 2023-08-29 | 同济大学 | Multi-lane merging illegal automatic auditing method, system, device and storage medium |
| CN114782904A (en) * | 2022-04-26 | 2022-07-22 | 平安普惠企业管理有限公司 | Vehicle compaction line detection method, device, equipment and storage medium |
| CN117037078A (en) * | 2023-06-02 | 2023-11-10 | 阿里巴巴（中国）有限公司 | Method for determining road area, method and device for detecting illegal behaviors |
| CN117496727A (en) * | 2023-11-13 | 2024-02-02 | 广东高信智能科技有限公司 | Traffic recognition and control method and device based on slot board |
Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1987007057A1 (en) * | 1986-05-05 | 1987-11-19 | Perceptics Corporation | Apparatus for reading a license plate |
| CN101398894A (en) * | 2008-06-17 | 2009-04-01 | 浙江师范大学 | Automobile license plate automatic recognition method and implementing device thereof |
| CN105335743A (en) * | 2015-10-28 | 2016-02-17 | 重庆邮电大学 | Vehicle license plate recognition method |
| CN106845480A (en) * | 2017-01-13 | 2017-06-13 | 河海大学 | A kind of method that car plate is recognized from picture |
| CN107688764A (en) * | 2016-08-03 | 2018-02-13 | 浙江宇视科技有限公司 | Detect the method and device of vehicle peccancy |
| CN108335489A (en) * | 2018-03-11 | 2018-07-27 | 西安电子科技大学 | Vehicle on highway behavior semantic analysis and abnormal behaviour monitoring system and method |
| CN108734189A (en) * | 2017-04-20 | 2018-11-02 | 天津工业大学 | Vehicle License Plate Recognition System based on atmospherical scattering model and deep learning under thick fog weather |
| CN108921956A (en) * | 2018-07-03 | 2018-11-30 | 重庆邮电大学 | A kind of curb parking charge management method based on Video Analysis Technology |
| CN109034247A (en) * | 2018-07-27 | 2018-12-18 | 北京以萨技术股份有限公司 | A kind of recognition of face sample extraction method of the higher degree based on track algorithm |
| WO2018233038A1 (en) * | 2017-06-23 | 2018-12-27 | 平安科技（深圳）有限公司 | Deep learning-based method, apparatus and device for recognizing license plate, and storage medium |
| CN109389589A (en) * | 2018-09-28 | 2019-02-26 | 百度在线网络技术（北京）有限公司 | Method and apparatus for statistical number of person |
| CN109670498A (en) * | 2018-11-10 | 2019-04-23 | 江苏网进科技股份有限公司 | A kind of license plate locating method |
| CN109948416A (en) * | 2018-12-31 | 2019-06-28 | 上海眼控科技股份有限公司 | A kind of illegal occupancy bus zone automatic auditing method based on deep learning |
| CN109949578A (en) * | 2018-12-31 | 2019-06-28 | 上海眼控科技股份有限公司 | A kind of illegal automatic auditing method of vehicle crimping based on deep learning |
| CN110298300A (en) * | 2019-06-27 | 2019-10-01 | 上海工程技术大学 | A method of detection vehicle violation crimping |
| CN110321814A (en) * | 2019-06-18 | 2019-10-11 | 浙江大华技术股份有限公司 | A kind of vehicle violation evidence collecting method, device, electronic equipment and storage medium |
| CN110458868A (en) * | 2019-08-15 | 2019-11-15 | 湖北经济学院 | Multiple target tracking based on SORT identifies display systems |
| CN110516556A (en) * | 2019-07-31 | 2019-11-29 | 平安科技（深圳）有限公司 | Multi-target tracking detection method, device and storage medium based on Darkflow-DeepSort |
| CN110659539A (en) * | 2018-06-28 | 2020-01-07 | 杭州海康威视数字技术股份有限公司 | Information processing method and device |
| CN110675637A (en) * | 2019-10-15 | 2020-01-10 | 上海眼控科技股份有限公司 | Vehicle illegal video processing method and device, computer equipment and storage medium |
-
2020
- 2020-03-10 CN CN202010163795.3A patent/CN111382704B/en active Active
Patent Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1987007057A1 (en) * | 1986-05-05 | 1987-11-19 | Perceptics Corporation | Apparatus for reading a license plate |
| CN101398894A (en) * | 2008-06-17 | 2009-04-01 | 浙江师范大学 | Automobile license plate automatic recognition method and implementing device thereof |
| CN105335743A (en) * | 2015-10-28 | 2016-02-17 | 重庆邮电大学 | Vehicle license plate recognition method |
| CN107688764A (en) * | 2016-08-03 | 2018-02-13 | 浙江宇视科技有限公司 | Detect the method and device of vehicle peccancy |
| CN106845480A (en) * | 2017-01-13 | 2017-06-13 | 河海大学 | A kind of method that car plate is recognized from picture |
| CN108734189A (en) * | 2017-04-20 | 2018-11-02 | 天津工业大学 | Vehicle License Plate Recognition System based on atmospherical scattering model and deep learning under thick fog weather |
| WO2018233038A1 (en) * | 2017-06-23 | 2018-12-27 | 平安科技（深圳）有限公司 | Deep learning-based method, apparatus and device for recognizing license plate, and storage medium |
| CN108335489A (en) * | 2018-03-11 | 2018-07-27 | 西安电子科技大学 | Vehicle on highway behavior semantic analysis and abnormal behaviour monitoring system and method |
| CN110659539A (en) * | 2018-06-28 | 2020-01-07 | 杭州海康威视数字技术股份有限公司 | Information processing method and device |
| CN108921956A (en) * | 2018-07-03 | 2018-11-30 | 重庆邮电大学 | A kind of curb parking charge management method based on Video Analysis Technology |
| CN109034247A (en) * | 2018-07-27 | 2018-12-18 | 北京以萨技术股份有限公司 | A kind of recognition of face sample extraction method of the higher degree based on track algorithm |
| CN109389589A (en) * | 2018-09-28 | 2019-02-26 | 百度在线网络技术（北京）有限公司 | Method and apparatus for statistical number of person |
| CN109670498A (en) * | 2018-11-10 | 2019-04-23 | 江苏网进科技股份有限公司 | A kind of license plate locating method |
| CN109948416A (en) * | 2018-12-31 | 2019-06-28 | 上海眼控科技股份有限公司 | A kind of illegal occupancy bus zone automatic auditing method based on deep learning |
| CN109949578A (en) * | 2018-12-31 | 2019-06-28 | 上海眼控科技股份有限公司 | A kind of illegal automatic auditing method of vehicle crimping based on deep learning |
| CN110321814A (en) * | 2019-06-18 | 2019-10-11 | 浙江大华技术股份有限公司 | A kind of vehicle violation evidence collecting method, device, electronic equipment and storage medium |
| CN110298300A (en) * | 2019-06-27 | 2019-10-01 | 上海工程技术大学 | A method of detection vehicle violation crimping |
| CN110516556A (en) * | 2019-07-31 | 2019-11-29 | 平安科技（深圳）有限公司 | Multi-target tracking detection method, device and storage medium based on Darkflow-DeepSort |
| CN110458868A (en) * | 2019-08-15 | 2019-11-15 | 湖北经济学院 | Multiple target tracking based on SORT identifies display systems |
| CN110675637A (en) * | 2019-10-15 | 2020-01-10 | 上海眼控科技股份有限公司 | Vehicle illegal video processing method and device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111382704A (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111382704B (en) | Vehicle line pressing violation judging method and device based on deep learning and storage medium | |
| CN109993056B (en) | Method, server and storage medium for identifying vehicle illegal behaviors | |
| CN110414507B (en) | License plate recognition method and device, computer equipment and storage medium | |
| EP3806064B1 (en) | Method and apparatus for detecting parking space usage condition, electronic device, and storage medium | |
| Al-Ghaili et al. | Vertical-edge-based car-license-plate detection method | |
| US20190188528A1 (en) | Text detection method and apparatus, and storage medium | |
| CN108268867B (en) | License plate positioning method and device | |
| CN106600977B (en) | Multi-feature recognition-based illegal parking detection method and system | |
| CN101739561B (en) | TV station logo training method and identification method | |
| CN103324930B (en) | A license plate character segmentation method based on gray histogram binarization | |
| CN107895492A (en) | A kind of express highway intelligent analysis method based on conventional video | |
| CN108108734B (en) | License plate recognition method and device | |
| Khalifa et al. | Malaysian Vehicle License Plate Recognition. | |
| CN105303153A (en) | Vehicle license plate identification method and apparatus | |
| Azad et al. | New method for optimization of license plate recognition system with use of edge detection and connected component | |
| CN111178359A (en) | License plate number recognition method, device and device and computer storage medium | |
| CN104463238B (en) | A kind of automobile logo identification method and system | |
| CN110969164A (en) | Low-illumination imaging license plate recognition method and device based on deep learning end-to-end | |
| Paunwala et al. | A novel multiple license plate extraction technique for complex background in Indian traffic conditions | |
| Ganapathy et al. | A Malaysian vehicle license plate localization and recognition system | |
| CN106778675B (en) | A kind of recognition methods of target in video image object and device | |
| CN114724119A (en) | Lane line extraction method, lane line detection apparatus, and storage medium | |
| CN114141022B (en) | Emergency lane occupation behavior detection method and device, electronic equipment and storage medium | |
| CN107506777A (en) | A real-time multi-license plate recognition method and device based on wavelet variation and support vector machine | |
| Khoshki et al. | Improved Automatic License Plate Recognition (ALPR) system based on single pass Connected Component Labeling (CCL) and reign property function |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 266400 No. 77, Lingyan Road, LINGSHANWEI sub district office, Huangdao District, Qingdao City, Shandong Province Applicant after: Qingdao Issa Technology Co.,Ltd. Applicant after: QINGDAO YISA DATA TECHNOLOGY Co.,Ltd. Address before: 100020 room 108, 1 / F, building 17, yard 6, Jingshun East Street, Chaoyang District, Beijing Applicant before: BEIJING YISA TECHNOLOGY Co.,Ltd. Applicant before: QINGDAO YISA DATA TECHNOLOGY Co.,Ltd. Address after: 266400 No. 77, Lingyan Road, LINGSHANWEI sub district office, Huangdao District, Qingdao City, Shandong Province Applicant after: Issa Technology Co.,Ltd. Applicant after: QINGDAO YISA DATA TECHNOLOGY Co.,Ltd. Address before: 266400 No. 77, Lingyan Road, LINGSHANWEI sub district office, Huangdao District, Qingdao City, Shandong Province Applicant before: Qingdao Issa Technology Co.,Ltd. Applicant before: QINGDAO YISA DATA TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |