CN108830192A - Vehicle and detection method of license plate under vehicle environment based on deep learning - Google Patents
Vehicle and detection method of license plate under vehicle environment based on deep learning Download PDFInfo
- Publication number
- CN108830192A CN108830192A CN201810545314.8A CN201810545314A CN108830192A CN 108830192 A CN108830192 A CN 108830192A CN 201810545314 A CN201810545314 A CN 201810545314A CN 108830192 A CN108830192 A CN 108830192A
- Authority
- CN
- China
- Prior art keywords
- detection
- license plate
- vehicle
- network
- branches
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The present invention uses SSD to improve as basic network, and proposes a network end to end, for detecting vehicle and license plate simultaneously in the given image.Network tool is respectively designed with different convolutional layers and carrys out detection for vehicle and license plate there are two independent branch.Finally, our strategies based on SSD improve again and design our network, and use video verification its validity being collected into from real scene.Meanwhile we have proposed a kind of license plate data enhancement methods of not invasion of privacy, this is meaningful to further research.In addition, we improve the recall rate of detection system using attention mechanism, experiments have shown that the performance of target detection can be improved in it.Moreover, we use several strategies specific to task for the detection of vehicle and license plate, to realize the real-time and efficient detection in video.
Description
Technical field
The present invention relates to the object detection methods of computer vision field, mainly propose a kind of mind of depth end to end
It is that basic network improves with SSD, for detecting vehicle and license plate in given image simultaneously through network.
Background technique
Automotive vehicle and car plate detection are the important components of intelligent transportation system.It can be applied to many applications,
Such as traffic monitoring and management.Up to the present, it has caused considerable research concern, and proposes many methods.
Due to machine learning since in recent years（Such as CNN even depth neural network）It rapidly develops, passes through the trial carried out in many fields
And improvement, this method all show good performance.However, the mesh different due to many of natural scene road image
Mark, still will receive very big interference, it is also necessary to take certain measure to reduce erroneous detection, promote our detection performance.And
Real-time vehicle and car plate detection play an important role in intelligent transportation system, so in practical applications, solving this
Many problems that the challenging task of item faces are very important.The present invention is exactly to study how to solve complicated natural field
Challenging vehicle and car plate detection problem under scape.
For detection framework, conventional target detection method is used always in the past few decades, this is general
It is divided into three phases：The region of some candidates is selected on given image first, then to these extracted region features, finally
Classified using trained classifier.But conventional method there is a problem of two it is extremely important：One is based on sliding window
The regional choice strategy of mouth does not have specific aim, and time complexity is high, window redundancy；Second is that the feature of hand-designed is for diversity
Variation there is no good robustness.In recent years, deep neural network is pushed away due to the ability of its character representation and robustness
The progress of target detection is moved, and the method based on DNN has been realized in extraordinary performance.In these methods, it is based on
The detection method in region is widely used.R-CNN（Region-based Convolutional Neural Network）
（Ross B. Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014.
Rich Feature Hierarchies for Accurate Object Detection and Semantic
Segmentation.In CVPR’14 Proceedings of the 2014 IEEE Conference on Computer
Vision and Pattern Recognition. 580–587.）It is a kind of method for detecting area of classics.R-CNN is used
Selective search algorithm extracts candidate frame in the picture, and the candidate region after normalization is input in CNN network,
Carry out the extraction of feature.It is identified for CNN feature, then with svm classifier, bezel locations and big is finely tuned with linear regression
It is small.But R-CNN is relatively time consuming, because these candidate frames require to carry out CNN operation, calculation amount is still very big, wherein having not
It is few to compute repeatedly in fact.The relatively good method occurred later is exactly SPP-Net（Spatial pyramid pond）, FastR-CNN and
FasterR-CNN（Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. 2015.
Faster R-CNN: towards real-time object detection with region proposal
networks. In NIPS’15 Proceedings of the 28th International Conference on
Neural Information Processing Systems - Volume 1, Vol. 2015. 91–99.）.FasterR-
The maximum contribution of CNN is exactly RPN (Region Proposal Networks) network, and the core concept of RPN is using convolution mind
Region proposal candidate frame is directly generated through network, the method used is substantially sliding window.By RPN,
FasterR-CNN becomes a two stage end-to-end detection framework, realizes performance boost by reducing time loss.
MultiBox, YOLO and SSD（Single Shot MultiBox Detector）（Wei Liu, DragomirAnguelov,
DumitruErhan, Christian Szegedy, Scott E. Reed, Cheng-Yang Fu, and Alexander
C. Berg. 2016. SSD: Single Shot MultiBox Detector.european conference on
computer vision (2016), 21–37.）End to end network directly prediction/regressive object bounding box can be used.This
A little method computation complexities are less, simplify entire target detection process, and the promotion of speed is also very big.SSD utilizes multiple dimensioned spy
Sign does not make full use of the information of high-level characteristic figure to be detected, this is very for the detection of Small object in fact
Important.In order to solve this problem, FPN is proposed（Feature Pyramid Networks）And DSSD
（Deconvolutional Single Shot Detector）Method, can be combined semantic powerful low-level feature with
High-level characteristic.In addition, FCN（Full convolutional network）The segmentation template an of high quality can be generated with Mask R-CNN, for real
Example segmentation.Development in the behind of various detection frameworks, basic network CNN plays an important role.From 1998, LeNet,
AlexNet, VGGNet, GoogleNet and ResNet occur successively.These networks（Except LeNet）Constantly at ImageNet points
Extraordinary performance is shown in class challenge.And current main trend is deeper network, bigger parameter and more powerful
Convolution module.Meanwhile the methods of other multiple technologies, such as Adam, PReLU and Focal Loss, it is provided to more preferable
It improves the performance of DNN and proposes in ground.
Negri and Han（Sungji Han, Youngjoon Han, and Hernsoo Hahn. 2009. Vehicle
Detection Method using Haar-like Feature on Real Time System. World Academy
of Science, Engineering and Technology, International Journal of Electrical,
Computer, Energetic, Electronic and Communication Engineering 3, 11 (2009),
1957–1961.）Et al. propose using textural characteristics and carry out vehicle detection.Chabot et al. proposes many vehicle analysis
Task.The framework improves vehicle detection and local positioning based on new target protocol from coarse to fine.Zhou et al. is proposed
A kind of detection method of license plate based on character, they extract local feature using BoW model.Kim et al. proposes one kind
The detection method of license plate extracted using vehicle region, they generate vehicle candidate using R-CNN, then to each vehicle area
The license plate in domain is positioned.However, network is an individual dual stage process, it is necessary to be instructed by two independent stages
Practice.Fu [6] et al. proposes a kind of concatenated convolutional neural network for car plate detection, it generates vehicle using RPN and waits first
Choosing, then detects the license plate in each candidate frame.The frame is a kind of method end to end, but due to two stages reasoning, so
With biggish calculating cost.And this method will lead to the mistake of more license plates when vehicle location fails.Li et al. people
A kind of end-to-end trainable DNN network of unification is proposed, for carrying out text detection and knowledge in natural scene image simultaneously
Not.Liu et al. people proposes a kind of the end-to-end of unification and trains slewing text identification（FOTS）Network, for examining simultaneously
It surveys and identifies, share calculating and visual information in two complementary tasks.
Summary of the invention
（1）The purpose of the present invention（The technical issues of solution）
Key technical problem to be solved by this invention is to solve challenging vehicle and license plate under complicated natural scene to examine
Survey problem.In road scene image in a natural environment, background clutter is very common phenomenon, and detects license plate only
The small part in image is accounted for, this just brings very big difficulty and challenge to the accurate detection of license plate.However, due to
(a) relatively small area, the complex background clutter in (b) real scene and (c) illumination variation, viewpoint variation and block
Problem, car plate detection are still an extremely challenging task.Equally, although vehicle have biggish scale, (b),
(c) and the complex characteristic these problems of its own are still such that the vehicle detection task in real scene becomes relatively difficult, such as
Modern car plate detection task is still faced with many do not solved the problems, such as very well.
（2）Technical solution of the present invention
In recent years, the detection method based on CNN realizes the performance of highly significant, research shows that the low-level feature figure of CNN has very
High resolution ratio and weak semanteme, and high-level characteristic figure has lower resolution ratio and stronger semanteme.Based on these results of study, I
Propose a kind of deep neural network end to end, for detecting vehicle and license plate in given image simultaneously, wherein for vehicle
Detection and car plate detection separately designed two individual branches with different convolutional layers.In view of license plate is small in size, feature
Obviously, and the features such as vehicle volume is big, feature is complicated, therefore the detection of license plate is carried out using low-level feature, uses high-level characteristic
Carry out the positioning of vehicle.In addition, also enhancing recalling for detection system using attention mechanism in conjunction with each detection branches.
We use Single Shot Multibox Detector（SSD）The validity of our end to end network is demonstrated, and
Be collected into multiple videos from the data logger automatically moved, by various experimental verifications it is proposed that method it is high-precision
Degree.Our method is on NVIDIA TITAN 1080Ti GPU with 16 fps efficient operations, it is sufficient to realize real-time video prison
Control.
Core of the invention is described in detail below, that is, designs the master network structure of realization.Our structural design drawing
It can be seen that attached drawing 1.Basic network is inherited from classical VGG-16 structure, wherein remaining the conv1_ in VGG-16
1 arrives the convolutional layer of conv5_3, and then by last two, connecting layer is substituted for two convolutional layers entirely.We also added additional
Layer, from conv6_2 layers to conv8_2 layers, to realize semantic stronger feature extraction.
Firstly, it is 224 × 400 that we, which will input size adjusting according to the video of the 720p and 1080p of mainstream, therefore passing through
After crossing the size of resize input picture, we can reduce the deformation of shape.In addition to this, we are by transformed VGG-16
In the channels of all convolutional layers be customized to original half, be considerably reduced memory and calculation amount in this way.We also calculate
The acceptance region of basic network, further preferably to go distribution detection branches.Simultaneously in order to calculate the receptive field in every layer, I
Need to track the distance between current receptive field size r and two adjacent features j.Furthermore, it is possible to by using Hole algorithm
The method of filling carries out convolutional layer dilation expansive working.When we are carried out using the step-length of the core and s size with k size
Convolution（Chi Hua）When operation, output layer can be calculated by equation below：
The design and implementation of detection branches is introduced in this following part.With reference to attached drawing 2（a）（b）Shown in, it is understood that
All license plates in training set are respectively less than 200 pixels, and less than 600 pixels of most of vehicles in training set.Consider
To the perception domain of network, the combination of low-level feature can contribute to license plate since its is small in size and feature is obvious
Detection.And high-level characteristic is due to its biggish size and more complicated feature, so can be used for the positioning function of vehicle.
After a series of experiments, we are by conv3_3, in the detection branches of conv4_3 and conv5_3 Layer assignment to license plate, and will
Fc7 layers, conv6_2 layers, conv7_2 layers with conv8_2 Layer assignment into the detection branches of vehicle.In addition, license plate usually has
Fixed ratio and space ratio.According to the thought of YOLO2, the license plate number that we acquire according to 1027 is come the space to license plate
Ratio is calculated.By data analysis, we may safely draw the conclusion, and the space proportion of most of license plates is from 1.5 to 3.5.For
For the sake of simplicity, we select space than the anchor for 1/2 and 1/3 to detect license plate.Equally, we are from 1471 real scenes
Their the ratio of width to height is obtained in vehicle.It may be seen that point of a Relatively centralized from 0.5 to 2.0 from experimental data
Cloth, and we select space ratio to be used to carry out the positioning of license plate for 1/1,1/2 and 2/1 anchor.In addition, we use one
The convolution filter of a 1 * 3 carries out car plate detection, this is for being fitted the larger space ratio of license plate and being avoided various aspects
The noise effects of introducing are more preferable.
The two detection branches are all two classification tasks.Each branch operates by individual reasoning, and carries out independent
Back-propagation process.As can be seen that the feature of lower layer network structure, and phase are shared by Liang Ge branch from the structure of whole network
Mutual Enhanced feature extracts.
In addition, we also add attention mechanism, Attention attention mechanism is applied to the two detections by us
In branch, and in target classification and the subset for being added to a similar segmentation before returning.For car plate detection branch and vehicle
Detection branches notice that power module helps to highlight the feature of license plate area or vehicle region, and reduce no license plate
With the characteristic area of vehicle.Meanwhile, it is to be noted that power supervision message can be obtained by the way that true value is added.End, it is noted that power maps quilt
It is input in exponent arithmetic, the dot product of characteristic pattern is then obtained by calculation.
Finally introduce our training objective.Majorized function when training consists of three parts.For classification
With recurrence subset, we use loss function identical with SSD.IfFor matching theA default frame andA true frame
Index.Indicate matching, otherwise.IfFor confidence level,For prediction block,For true frame,It is silent to match
Recognize the quantity of frame.For attention subset, the pixel between the true frame of groundtruth is tried hard in the attention that we calculate generation
Grade difference.Assuming thatIt is the index of all pyramid features used,It is that every grade of attention generated is tried hard to,It is to be retouched in Fig. 3
The true frame stated,WithIt is to balance these weighting parameters, and we are simply to set, loss function
It is defined as follows：
Smooth L1 loss function and two classification Softmax loss functions are respectively adopted to position loss and confidence level damage in we
It loses, and calculates attention loss using the sigmoid cross entropy of Pixel-level.Positioning loss can deviate position and scaling
Revert to default frame.Assuming that a default frameAnd predicted value, default frameIt can be calculated by following formula：
We use two individual majorized functions to car plate detection branch and vehicle detection branch respectively.The two branches carry out
Respective back-propagation process.
（3）Advantage or good effect
Compared to the prior art possessed advantage is the present invention：
Herein, we have proposed a networks end to end, for detecting license plate and vehicle simultaneously in given image,
It is respectively provided with two independent branches for car plate detection and vehicle detection.
Car plate detection has low-level feature, vehicle detection to high-level characteristic.Vehicle detection model and vehicle plate location model are all
It is independently trained, all Analysis On Multi-scale Features of our two detection branches are utilized in each model.Our network
License plate and vehicle can be detected simultaneously on given image, this saves approximately half of runing times.
In addition, the recall rate of detection system is improved using attention mechanism in each detection branches, attention mechanism
The feature of RoI can be highlighted and reduce other characteristic areas, so as to promote detection performance.We pass through various realities
Demonstrate the validity of our methods.
Moreover, a large amount of runing time can be saved by cutting out the size of network and adjustment anchor.Picture size is contracted
224 × 400 are put into, the influence very little along with the convolution filter using 1 × 3 to the speed of service, but since they subtract respectively
The deformation of small input picture and the noise of other various aspects introduce, so improving detection performance, this is in experimental result
Available verifying.
In view of the test problems in video, it is real-time and high to realize that we additionally use several technologies specific to task
The target detection of effect.In addition, we also proposed a kind of license plate Enhancement Method of not invasion of privacy, experiment shows artificial synthesized
License plate can partially substitute outdoor scene license plate.
The deep neural network end to end that we are proposed, and can have for method designed by each problem
The operation of effect ground, and good confirmation has been obtained by experiment.In addition, 3.3 times faster than archetype of our experiment, by dividing
Analysis and verifying, our method achieve the promotion of sizable detection performance.
Detailed description of the invention
Fig. 1 is the whole network structural schematic diagram for detecting network in the present invention end to end.
Fig. 2（a）In be vehicle width and height be distributed.These data come from 1471 label vehicles.X-axis represents big
Small, y-axis represents vehicle fleet size（Do not consider negative）.Bluish violet represents the width of vehicle, and pink represents the height of vehicle.Due to
We consider automobile, the distribution of minibus and truck, vehicle is relatively uniform, it might even be possible to reach 800 pixels.（b）In be vehicle
The width and height of board are distributed.These data are from 1027 license plates.Bluish violet represents the width of license plate, and pink represents vehicle
The height of board.Licence plate distribution is more concentrated, and wherein most is below 150 pixels.（c）In be vehicle and license plate the ratio of width to height.X-axis
Indicate that the ratio of width to height, y-axis indicate license plate or vehicle；
Fig. 3 is attention supervision（License plate is blocked）.Top image is original image.Lower-left image is the prison for vehicle detection
Information is superintended and directed, and bottom-right graph seems the supervision message for car plate detection；
Fig. 4 is manually generated license plate schematic diagram, we replace truthful data with the license plate of synthesis；
Fig. 5 is testing result（Including all identifiable license plates）.Green rectangle indicates the testing result frame of vehicle, red rectangle
Indicate the local boundary frame of license plate.Our detection method robustness is fine, light differential is obvious, visual angle is different and scale
It all detected many vehicles and license plate in the case where difference simultaneously.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with attached drawing to of the invention real
The specific embodiment of the recognition methods and system of applying the vehicle of example is illustrated.It should be appreciated that specific reality described herein
Example is applied only to explain the present invention, is not intended to limit the present invention.
We have proposed one, network is next end to end while detecting vehicle and license plate, wherein distinguishing for vehicle and license plate
Devise two individual branches with different convolutional layers.We carry out the detection of license plate by low-level feature, while using high
The feature of layer positions the position of vehicle.In order to improve the recall rate of detection system, we are added to Attention attention machine
System is to reinforce each detection branches.In addition to this, for the real-time and effective object detection task in video, we
Apply several strategies specific to task.Finally, we have designed and Implemented the network structure based on SSD, wherein multiple dimensioned
Feature is combined the detection and recurrence for carrying out object boundary frame in a single stage.
（1）The data set that the present invention uses
We use three data logger automatically moved, collect video in different places and time respectively, and by its table
It is shown as、With, resolution ratio is respectively 720p, 720p and 1080p.After being extracted frame, the image of 720p is about
Four times of 1080p image.
The pre-training stage：For vehicle data, we used be labelled in KITTI data set Car, Truck and
All pictures of Van.For License Plate data to be used, we have proposed a kind of methods of data enhancing, i.e., at random
10 artificial synthesized license plates are selected to replace groundtruth.In this way, our 354 from collection
The license plate of 3540 synthesis has been obtained in groundtruth.
The fine tuning stage：We are finely adjusted instruction to preparatory trained model using 1027 license plates and 1471 vehicles
Practice.In addition, being comparison model, in addition license plate that we are also synthesized with 10270.
Test phase：It is used to test, we have collected 708 license plate pictures and 742 vehicle pictures again.Equally
Ground, there are also 7080 other license plate pictures for we for further testing use.
（2）Experiment description
In experiment of the invention, we use Caffe frame as training tool.Our model be by pre-training,
Picture is zoomed to the size of 224*400 by the pre-training stage, while using Adam optimization method.Learning rate is fixed as 0.001,
Batchsize is set as 4.Exponential decay rate is set as 0.9 and 0.999.We using KITTI all vehicles and 3540 lifes
At license plate train the network to carry out 120K iteration in advance.Then, we using 1027 license plates and 1471 vehicles to 50,
The model of 000 iteration is finely adjusted.Meanwhile we also use 10270 license plate data generated and identical vehicle data pair
Preparatory trained model is finely adjusted, to obtain contrasting detection model.All training images all pass through mirror image, sampling, mistake
It very and is sized to be enhanced, all follows the strategy process mentioned in SSD.All experiments are all in 4 NVIDIA
It is carried out on the work station of TITAN 1080Ti.
（3）Experimental result
In view of a vehicle can include only a license plate, so working as IoU>When 0, we obtain the license plate of maximum confidence.
In addition, we set 0.6 for the threshold value of IoU, the speed of service is assessed according to the average time of every image.Basic network
For SSD, vehicle detection model and vehicle plate location model are all independently trained, our two inspections are utilized in each model
Survey all Analysis On Multi-scale Features of branch.By experiment it is recognised that our network can be on given image simultaneously
License plate and vehicle are detected, this saves approximately half of runing times.In addition, the size for cutting out network and adjustment anchor all can
A degree of loss is caused to performance, but this can but save a large amount of runing time.Moreover, picture size is zoomed to
224 × 400, the influence along with the convolution filter using 1 × 3 to the speed of service is very little.Since they are respectively reduced
The deformation of input picture and biggish various aspects noise, so improving detection performance.Furthermore, it is noted that power mechanism can protrude
It shows the feature of RoI and reduces other characteristic areas, so as to improving performance.Finally, our method can be effectively
Operation, and 3.3 times faster than archetype.And by being tested on different experiments test set, as a result all show ours
Method detection performance is fine, has good robustness.
In addition, we also test the license plate of generation, detected hardly using outdoor scene or the license plate of generation
It will affect the performance of vehicle detection branch.However, due between the license plate of license plate and generation under true nature scene illumination,
Viewpoint variation and textural characteristics etc. have differences, so the model being trained using the license plate of generation is true in detection
When license plate under scene, performance decline compares larger.But on the other hand, we may safely draw the conclusion thinks, makes
It is very feasible for being enhanced with the license plate data of generation to carry out the detection of license plate.
The invention proposes a networks end to end, for detecting license plate and vehicle simultaneously in given image, point
There are not be used for two independent branches of car plate detection and vehicle detection.Car plate detection has low-level feature, vehicle detection to height
Layer feature.In addition, the recall rate of detection system is improved using attention mechanism in each detection branches.We are based in use
The basic network of SSD, and pass through the above-mentioned various experimental verifications validity of our methods.In view of the detection in video is asked
Topic, we additionally use several technologies specific to task to realize real-time and efficient target detection.In addition, we also propose
A kind of license plate Enhancement Method of not invasion of privacy, experiment show that artificial synthesized license plate can partially substitute outdoor scene license plate.
However for us, the more complicated car plate detection of scene is still a challenging task, we
More complicated and difficult vehicle sample will be also collected to be used to further study.In addition, how to allow the license plate of synthesis closer to very
License plate under real field scape is another a major challenge that we face.
Claims (5)
1. one is detected network end to end, it is characterised in that the vehicle and license plate in given image can be detected simultaneously, wherein
Two individual branches with different convolutional layers are separately designed for vehicle detection and car plate detection, while to improve detection system
Recall rate adds Attention attention mechanism to reinforce each detection branches, in addition to this, in video
Real-time and effective object detection task, using several strategies specific to task.
2. method according to claim 1, which is characterized in that when designing different detection branches, pass through low layer spy
Sign carries out the detection of license plate, while the position of vehicle is positioned using high-rise feature, uses SSD as basic network, vehicle
Detection model and vehicle plate location model are all independently trained, and all multiple dimensioned spies of two detection branches are all utilized
Sign, network can detect license plate and vehicle simultaneously on given image, save approximately half of runing time.
3. according to the method described in claim 1, improving detection system using attention mechanism in each detection branches
Recall rate, attention mechanism can highlight the feature of RoI and reduce other characteristic areas, so as to promote detection
Energy.
4. according to the method described in claim 1, in view of the test problems in video, using several technologies specific to task
It realizes real-time and efficient target detection, in addition, proposing a kind of license plate Enhancement Method of not invasion of privacy, passes through experimental verification
Artificial synthesized license plate can partially substitute outdoor scene license plate.
5. the deep neural network end to end proposed, and can effectively be transported for method designed by each problem
Row, and confirmed well by experiment, while passing through analysis and verifying, it was demonstrated that the detection that the method that the present invention is mentioned obtains
Performance is obviously improved.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810545314.8A CN108830192A (en) | 2018-05-31 | 2018-05-31 | Vehicle and detection method of license plate under vehicle environment based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810545314.8A CN108830192A (en) | 2018-05-31 | 2018-05-31 | Vehicle and detection method of license plate under vehicle environment based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108830192A true CN108830192A (en) | 2018-11-16 |
Family
ID=64145596
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810545314.8A Pending CN108830192A (en) | 2018-05-31 | 2018-05-31 | Vehicle and detection method of license plate under vehicle environment based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108830192A (en) |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109657577A (en) * | 2018-12-07 | 2019-04-19 | 重庆理工大学 | A kind of animal detection method based on entropy and motion excursion amount |
| CN109815919A (en) * | 2019-01-28 | 2019-05-28 | 上海七牛信息技术有限公司 | A kind of people counting method, network, system and electronic equipment |
| CN109948612A (en) * | 2019-03-19 | 2019-06-28 | 苏州怡林城信息科技有限公司 | Detection method of license plate, storage medium and detection device based on convolutional network |
| CN109977956A (en) * | 2019-04-29 | 2019-07-05 | 腾讯科技（深圳）有限公司 | A kind of image processing method, device, electronic equipment and storage medium |
| CN110033002A (en) * | 2019-04-19 | 2019-07-19 | 福州大学 | Detection method of license plate based on multitask concatenated convolutional neural network |
| CN110334709A (en) * | 2019-07-09 | 2019-10-15 | 西北工业大学 | End-to-end multi-task deep learning-based license plate detection method |
| CN110610124A (en) * | 2019-07-30 | 2019-12-24 | 珠海亿智电子科技有限公司 | Image generation method based on generation countermeasure network |
| CN110909742A (en) * | 2019-11-19 | 2020-03-24 | 上海眼控科技股份有限公司 | License plate detection method, system, platform and storage medium |
| CN111126404A (en) * | 2019-12-11 | 2020-05-08 | 杭州电子科技大学 | An ancient text and font recognition method based on improved YOLO v3 |
| CN111222513A (en) * | 2019-12-31 | 2020-06-02 | 深圳云天励飞技术有限公司 | License plate number recognition method, device, electronic device and storage medium |
| CN111325209A (en) * | 2018-12-14 | 2020-06-23 | 顺丰科技有限公司 | License plate recognition method and system |
| CN111489334A (en) * | 2020-04-02 | 2020-08-04 | 暖屋信息科技(苏州)有限公司 | Defect workpiece image identification method based on convolution attention neural network |
| CN111860087A (en) * | 2019-10-16 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Information detection method, information detection device, computer device, and storage medium |
| CN112149661A (en) * | 2020-08-07 | 2020-12-29 | 珠海欧比特宇航科技股份有限公司 | License plate recognition method, device and medium |
| CN112949635A (en) * | 2021-03-12 | 2021-06-11 | 北京理工大学 | Target detection method based on feature enhancement and IoU perception |
| WO2021138893A1 (en) * | 2020-01-10 | 2021-07-15 | 顺丰科技有限公司 | Vehicle license plate recognition method and apparatus, electronic device, and storage medium |
| CN113159016A (en) * | 2019-07-26 | 2021-07-23 | 第四范式（北京）技术有限公司 | Text position positioning method and system and model training method and system |
| CN113222064A (en) * | 2021-05-31 | 2021-08-06 | 苏州晗林信息技术发展有限公司 | Image target object real-time detection method, system, terminal and storage medium |
| CN113657183A (en) * | 2021-07-26 | 2021-11-16 | 西安邮电大学 | Vehicle 24 color identification method under smooth neural network based on multilayer characteristics |
| CN114495022A (en) * | 2021-12-13 | 2022-05-13 | 江苏集萃未来城市应用技术研究所有限公司 | Vehicle license plate detection and localization method based on multi-scale YOLOv5 nesting |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845487A (en) * | 2016-12-30 | 2017-06-13 | 佳都新太科技股份有限公司 | A kind of licence plate recognition method end to end |
| CN107563372A (en) * | 2017-07-20 | 2018-01-09 | 济南中维世纪科技有限公司 | A kind of license plate locating method based on deep learning SSD frameworks |
| CN107833219A (en) * | 2017-11-28 | 2018-03-23 | 腾讯科技（深圳）有限公司 | Image-recognizing method and device |
| CN107845289A (en) * | 2017-12-19 | 2018-03-27 | 智慧互通科技有限公司 | A kind of Roadside Parking management and charging system based on height digital video array |
| CN108053027A (en) * | 2017-12-18 | 2018-05-18 | 中山大学 | A kind of method and device for accelerating deep neural network |
-
2018
- 2018-05-31 CN CN201810545314.8A patent/CN108830192A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106845487A (en) * | 2016-12-30 | 2017-06-13 | 佳都新太科技股份有限公司 | A kind of licence plate recognition method end to end |
| CN107563372A (en) * | 2017-07-20 | 2018-01-09 | 济南中维世纪科技有限公司 | A kind of license plate locating method based on deep learning SSD frameworks |
| CN107833219A (en) * | 2017-11-28 | 2018-03-23 | 腾讯科技（深圳）有限公司 | Image-recognizing method and device |
| CN108053027A (en) * | 2017-12-18 | 2018-05-18 | 中山大学 | A kind of method and device for accelerating deep neural network |
| CN107845289A (en) * | 2017-12-19 | 2018-03-27 | 智慧互通科技有限公司 | A kind of Roadside Parking management and charging system based on height digital video array |
Non-Patent Citations (4)
| Title |
|---|
| A.S.JOHNSON等: "Number-plate matching for automatic vehicle identification", 《IEE COLLOQUIUM ON ELECTRONIC IMAGES AND IMAGE PROCESSING IN SECURITY AND FORENSIC SCIENCE》 * |
| YIQING LIU等: "Vehicle-license-plate recognition based on neural networks", 《2011 IEEE INTERNATIONAL CONFERENCE ON INFORMATION AND AUTOMATION》 * |
| 曹洋: "基于视觉的车辆分类识别算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 谭贞刚: "基于多分辨率卷积网络的交通标志检测与识别", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109657577B (en) * | 2018-12-07 | 2022-06-28 | 重庆理工大学 | An animal detection method based on entropy and motion offset |
| CN109657577A (en) * | 2018-12-07 | 2019-04-19 | 重庆理工大学 | A kind of animal detection method based on entropy and motion excursion amount |
| CN111325209B (en) * | 2018-12-14 | 2024-06-07 | 顺丰科技有限公司 | License plate recognition method and system |
| CN111325209A (en) * | 2018-12-14 | 2020-06-23 | 顺丰科技有限公司 | License plate recognition method and system |
| CN109815919A (en) * | 2019-01-28 | 2019-05-28 | 上海七牛信息技术有限公司 | A kind of people counting method, network, system and electronic equipment |
| CN109948612A (en) * | 2019-03-19 | 2019-06-28 | 苏州怡林城信息科技有限公司 | Detection method of license plate, storage medium and detection device based on convolutional network |
| CN110033002A (en) * | 2019-04-19 | 2019-07-19 | 福州大学 | Detection method of license plate based on multitask concatenated convolutional neural network |
| CN110033002B (en) * | 2019-04-19 | 2022-07-08 | 福州大学 | License plate detection method based on multi-task cascaded convolutional neural network |
| CN109977956B (en) * | 2019-04-29 | 2022-11-18 | 腾讯科技（深圳）有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN109977956A (en) * | 2019-04-29 | 2019-07-05 | 腾讯科技（深圳）有限公司 | A kind of image processing method, device, electronic equipment and storage medium |
| CN110334709A (en) * | 2019-07-09 | 2019-10-15 | 西北工业大学 | End-to-end multi-task deep learning-based license plate detection method |
| CN113159016A (en) * | 2019-07-26 | 2021-07-23 | 第四范式（北京）技术有限公司 | Text position positioning method and system and model training method and system |
| CN110610124A (en) * | 2019-07-30 | 2019-12-24 | 珠海亿智电子科技有限公司 | Image generation method based on generation countermeasure network |
| CN110610124B (en) * | 2019-07-30 | 2021-11-30 | 珠海亿智电子科技有限公司 | Image generation method based on generation countermeasure network |
| CN111860087A (en) * | 2019-10-16 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Information detection method, information detection device, computer device, and storage medium |
| CN110909742A (en) * | 2019-11-19 | 2020-03-24 | 上海眼控科技股份有限公司 | License plate detection method, system, platform and storage medium |
| CN111126404B (en) * | 2019-12-11 | 2023-08-22 | 杭州电子科技大学 | Ancient character and font recognition method based on improved YOLO v3 |
| CN111126404A (en) * | 2019-12-11 | 2020-05-08 | 杭州电子科技大学 | An ancient text and font recognition method based on improved YOLO v3 |
| CN111222513A (en) * | 2019-12-31 | 2020-06-02 | 深圳云天励飞技术有限公司 | License plate number recognition method, device, electronic device and storage medium |
| WO2021138893A1 (en) * | 2020-01-10 | 2021-07-15 | 顺丰科技有限公司 | Vehicle license plate recognition method and apparatus, electronic device, and storage medium |
| CN111489334A (en) * | 2020-04-02 | 2020-08-04 | 暖屋信息科技(苏州)有限公司 | Defect workpiece image identification method based on convolution attention neural network |
| CN111489334B (en) * | 2020-04-02 | 2023-08-08 | 暖屋信息科技(苏州)有限公司 | Defect workpiece image recognition method based on convolution attention neural network |
| CN112149661A (en) * | 2020-08-07 | 2020-12-29 | 珠海欧比特宇航科技股份有限公司 | License plate recognition method, device and medium |
| CN112949635A (en) * | 2021-03-12 | 2021-06-11 | 北京理工大学 | Target detection method based on feature enhancement and IoU perception |
| CN112949635B (en) * | 2021-03-12 | 2022-09-16 | 北京理工大学 | Target detection method based on feature enhancement and IoU perception |
| CN113222064A (en) * | 2021-05-31 | 2021-08-06 | 苏州晗林信息技术发展有限公司 | Image target object real-time detection method, system, terminal and storage medium |
| CN113657183A (en) * | 2021-07-26 | 2021-11-16 | 西安邮电大学 | Vehicle 24 color identification method under smooth neural network based on multilayer characteristics |
| CN114495022A (en) * | 2021-12-13 | 2022-05-13 | 江苏集萃未来城市应用技术研究所有限公司 | Vehicle license plate detection and localization method based on multi-scale YOLOv5 nesting |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108830192A (en) | Vehicle and detection method of license plate under vehicle environment based on deep learning | |
| Luo et al. | Multi-scale traffic vehicle detection based on faster R–CNN with NAS optimization and feature enrichment | |
| Yan et al. | Real-time vehicle detection using histograms of oriented gradients and AdaBoost classification | |
| WO2024037408A1 (en) | Underground coal mine pedestrian detection method based on image fusion and feature enhancement | |
| CN105260749B (en) | Real-time target detection method based on direction gradient binary pattern and soft cascade SVM | |
| CN105844663A (en) | Adaptive ORB object tracking method | |
| Li et al. | Image manipulation localization using multi-scale feature fusion and adaptive edge supervision | |
| Zhai et al. | An improved faster R-CNN pedestrian detection algorithm based on feature fusion and context analysis | |
| Patel et al. | Night vision surveillance: Object detection using thermal and visible images | |
| Xie et al. | An enhanced relation-aware global-local attention network for escaping human detection in indoor smoke scenarios | |
| Yuan et al. | Multi-level object detection by multi-sensor perception of traffic scenes | |
| Kubiak et al. | S3r-net: A single-stage approach to self-supervised shadow removal | |
| Lee et al. | Boundary-aware camouflaged object detection via deformable point sampling | |
| CN110309761A (en) | Continuity gesture identification method based on the Three dimensional convolution neural network with thresholding cycling element | |
| Chen et al. | MFFNet: A lightweight multi-feature fusion network for UAV infrared object detection | |
| Zhang et al. | LL-WSOD: Weakly supervised object detection in low-light | |
| Zhu et al. | Detection of irregular small defects on metal base surface of infrared laser diode based on deep learning | |
| Hezil et al. | Vehicle license plate detection using morphological operations and deep learning | |
| Jiang et al. | MCU-GAN: Colorization method for infrared images based on multi-convolution fusion and generative adversarial network | |
| Jiang et al. | Mask‐guided image person removal with data synthesis | |
| Wang et al. | KCDNet: Multimodal Object Detection in Modal Information Imbalance Scenes | |
| Wang et al. | A detection algorithm of ballast track fastener loss based on shooting direction | |
| Li et al. | A foreground-focused action recognition algorithm for intelligent unmanned systems | |
| Niu et al. | YOLOv8s-DDA: An Improved Small Traffic Sign Detection Algorithm Based on YOLOv8s | |
| Zhao et al. | Small object detection of imbalanced traffic sign samples based on hierarchical feature fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20181116 |
|
| WD01 | Invention patent application deemed withdrawn after publication |