[go: up one dir, main page]

CN109558808A - A kind of road Edge Detection based on deep learning - Google Patents

A kind of road Edge Detection based on deep learning Download PDF

Info

Publication number
CN109558808A
CN109558808A CN201811329308.5A CN201811329308A CN109558808A CN 109558808 A CN109558808 A CN 109558808A CN 201811329308 A CN201811329308 A CN 201811329308A CN 109558808 A CN109558808 A CN 109558808A
Authority
CN
China
Prior art keywords
road
network
road edge
edge detection
layer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811329308.5A
Other languages
Chinese (zh)
Inventor
陈广
卢凡
陈凯
杨谦益
瞿三清
葛艺忻
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tongji University
Original Assignee
Tongji University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tongji University filed Critical Tongji University
Priority to CN201811329308.5A priority Critical patent/CN109558808A/en
Publication of CN109558808A publication Critical patent/CN109558808A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种基于深度学习的路边沿检测方法，包括如下步骤：(1)采集真实道路上包含路边沿的图像数据，并通过人工标注方法标注其中与路边沿检测相关的目标的位置和类别信息，构建路边沿检测的数据集；(2)构建适用于路边沿检测的多任务卷积神经网络及相应的损失函数；(3)将采集到的图像和标注数据输入到步骤(2)构建的卷积神经网络中，根据输出值与目标值之间的损失值对神经网络中的参数值进行更新，最后得到理想的网络参数。本发明对各种可见及非可见、有无明确几何特征和高度差的的路边沿都有着较好的检测能力，相对于3D激光雷达等检测方式具有成本优势，有利于大规模推广应用，促进自动驾驶技术的发展。

A road edge detection method based on deep learning, comprising the following steps: (1) collecting image data containing road edges on real roads, and labeling the location and category information of targets related to road edge detection by manual labeling methods, and constructing The dataset of road edge detection; (2) Construct a multi-task convolutional neural network suitable for road edge detection and the corresponding loss function; (3) Input the collected images and labeled data into the convolution constructed in step (2) In the neural network, the parameter values in the neural network are updated according to the loss value between the output value and the target value, and finally the ideal network parameters are obtained. The invention has better detection ability for various visible and invisible road edges with or without clear geometric features and height difference, and has cost advantages compared with detection methods such as 3D laser radar, which is conducive to large-scale popularization and application, and promotes The development of autonomous driving technology.

Description

A kind of road Edge Detection based on deep learning
Technical field
The invention belongs to intelligent driving technical fields, are related under a kind of special scenes of computer vision combination deep learning Object detection method.
Background technique
Road Edge check is one of automatic Pilot field and the important component in active safety field, it can be helped certainly It is dynamic to drive the current travelable region of vehicle identification and judge routing information.
Due to the significance of road Edge check, for such issues that, lot of domestic and international mechanism has been presented for a part Detection method, and existing road edge sense technology mostly be to be realized based on 3D laser radar, mostly according to roadside along with it is feasible The height change between road is sailed to detect roadside edge, the limitation of this detection method is as follows:
(1) price of 3D laser radar is relatively high, and large-scale application has certain difficulty；
(2) the roadside edge of no clear geometrical characteristic and difference in height can not be identified；
In fact, many efficient algorithm of target detection have been emerged at present with the development of depth learning technology, but Regrettably these algorithm of target detection satisfy the need this occupancy pixel in edge it is few, without clear geometrical characteristic and be continuous linear Target is simultaneously not suitable for.
Summary of the invention
For the limitation of the prior art, the road Edge Detection based on deep learning that the present invention provides a kind of, The image data obtained using vehicle-mounted high-definition camera extracts different piece in image using the convolutional neural networks of multitask Information merges disappearance dot position information, and Preliminary detection goes out road edge portions therein, satisfies the need edge further according to road region information Information further judges to enhance the robustness of algorithm.
To achieve the above object, the technical solution adopted in the present invention is as follows:
(1) acquire real roads on include roadside edge image data, and by artificial mask method mark wherein with road The position of the relevant target of Edge check and classification information construct the data set of road Edge check；
(2) building is suitable for the multitask convolutional neural networks and corresponding loss function of road Edge check；
(3) acquired image and labeled data are input in the convolutional neural networks of step (2) building, according to output Penalty values between value and target value are updated the parameter value in neural network, when penalty values converge to its global minimum When, save final network parameter.
Optionally, in step (1), the acquisition of image data and annotation step are as follows:
(1-1) demarcates the inside and outside parameter of camera；
(1-2) is directed to the roadside being likely to occur under actual condition along type, the roadside edge of acquisition is divided into following several: true Real visible roadside edge, roadside along side there are barrier, non-genuine visible roadside along etc.；
The classification of (1-3) image data mark is broadly divided into following several: background can travel region, and roadside edge is non-feasible Sail region, barrier etc.；Wherein, virtual roadside is marked out along position using manual identified for non-genuine visible roadside edge It sets；
(1-4) is labeled acquired image data using annotation tool, and notation methods include but is not limited to pixel Grade mark, grid mark.
Optionally, in step (2), building suitable for the multitask convolutional neural networks of road Edge check and corresponding Steps are as follows for loss function:
(2-1) constructs roadside along feature extraction network, and the image information for that will input carries out feature extraction and feature is compiled Code, obtain it is multiple dimensioned, can be used for road Edge check, road vanishing Point Detection Method and road area detection sharing feature floor；
(2-2) constructs road vanishing Point Detection Method network, to the further convolution of sharing feature layer obtained in step (2-1), The disappearance dot position information of road can be obtained by its output layer；
(2-3) constructs road edge and detects network, first by sharing feature layer obtained in step (2-1) and step (2- 2) output layer that road end point is detected obtained in, which is connected, obtains the input layer of road edge detection network；To this input layer into Row up-sampling, can be obtained road side information by final output layer；
(2-4) constructs road area and detects network, carries out down-sampling to output layer obtained in step (2-3) first and incites somebody to action This result is connected to obtain the input layer of road area detection network with sharing feature layer obtained in step (2-1), to this input Layer is up-sampled, and road region information can be obtained by final output layer；
(2-5) building is suitable for the target detection loss function of class imbalance, for calculating the detection of road end point Loss, and influence caused by the imbalance of the ratio of end point and background in sample can be inhibited；
(2-6) constructs cross entropy loss function, for calculating the road edge position information of road edge detection network output With the loss of actual position information；
(2-7) constructs cross entropy loss function, for calculating the road area of road area detection network output and true Loss between road area.
Optionally, in step (3), training network the step of it is as follows:
Acquired image is carried out data prediction by (3-1), and key step includes: to turn over image into row stochastic level Turn, cut and uniformly zoom to fixed size, labeled data is also overturn accordingly, cut and scaled, on this basis Obtained image is normalized by channel；
(3-2) carries out pre-training, obtained parameter value to above-mentioned network using SoftMax loss function on ImageNet Initial parameter as network；
Vanishing Point Detection Method network and road Edge check are connected to the network by (3-3), by pretreated picture and labeled data It is input in network, the road end point of network output is calculated using the loss function constructed in step (2-5), step (2-6) And the penalty values of road edge placement and actual position, parameter value is updated, when penalty values converge to its global minimum, Save current network parameter；
Road edge sence network is connected by (3-4) with road area detection network, utilizes the loss constructed in step (2-7) Function calculates the penalty values between network output road area and real road region, and into one on the basis of step (3-3) The parameter that step carries out network updates, and obtains final result.
By adopting the above scheme, the beneficial effects of the present invention are:
The first, sensor of the present invention is monocular camera, the road Edge Detection kind before price is opposite Used 3D laser radar is very cheap, facilitates practical popularization, the application of detection method；
The second, multitask network of the present invention takes full advantage of the information of road end point, so that roadside is along inspection The input layer of survey grid network contains more characteristic informations, enhances the robustness of algorithm；
Third, multitask network of the present invention use the output of road edge sence network in conjunction with shared characteristic layer In detection road area to update again to parameter value, the accuracy of identification is increased；
4th, depth convolutional neural networks of the present invention also have the roadside edge without clear geometrical characteristic Good detection effect.
Detailed description of the invention
Fig. 1 is the overall construction drawing of multitask convolutional neural networks of the present invention.
Fig. 2 is roadside of the present invention along feature extraction network structure.
Fig. 3 is road vanishing Point Detection Method network structure of the present invention.
Fig. 4 is Edge check network structure in road of the present invention.
Fig. 5 is that road area of the present invention detects network structure.
Specific embodiment
The road Edge Detection based on deep learning that the present invention provides a kind of, this method is with depth convolutional neural networks Based on, and merged the accuracy of road Vanishing Point Information and road region information enhancing road Edge check.Detailed network Structure is as shown in Figure 1.Method includes the following steps:
(1) acquire real roads on include roadside edge image data, and by artificial mask method mark wherein with road The position of the relevant target of Edge check and classification information construct the data set of road Edge check；
(2) building is suitable for the multitask convolutional neural networks and corresponding loss function of road Edge check；
(3) acquired image and labeled data are input in the convolutional neural networks of step (2) building, according to output Penalty values between value and target value are updated the parameter value in neural network, finally obtain ideal network ginseng Number；
Optionally, in step (1), the acquisition of image data and annotation step are as follows:
(1-1) demarcates the inside and outside ginseng of camera, and calibration internal reference is demarcated for eliminating pattern distortion caused by camera lens For outer ginseng for determining that the point of the road surface on image is corresponding in position in the real world, calibration process can refer to chessboard method, with to Calibration camera shoots the image that multiple include chessboard calibration plate, then inputs in the tool box Calibration of Matlab, obtains To the inside and outside ginseng of camera；
(1-2) is directed to the roadside being likely to occur under actual condition along type, the roadside edge of acquisition is divided into following several: true , there is the roadside edge of barrier on one side in real visible roadside edge, non-genuine visible roadside along etc..It is equipped with the number of camera Acquisition image data is travelled in real roads according to collecting vehicle, and is acquired under different weather, light conditions to increase The richness of data；
The classification of (1-3) image data mark is broadly divided into following several: background, road area, and roadside edge is non-to can travel Region, barrier etc.；Wherein, background parts are the part such as sky etc. that this method is not concerned with；Road area is the feasible of vehicle Sail region；There is the roadside edge of barrier along true visible roadside edge is divided into roadside, non-genuine visible roadside edge on one side, Wherein for there are the roadside of barrier edges to mark out obstacle information, manual identified is utilized for non-genuine visible roadside edge Mark out virtual road edge placement；Non- travelable region is except road area but on the influential area of judgement road edge placement Domain, such as pavement；
(1-4) is labeled acquired image data using annotation tool, and notation methods include but is not limited to pixel Grade mark, grid mark.Marked content includes each classification being previously mentioned in step (1-3), and annotation tool can be used Computer Vision Annotation Tool (CVAT), the tool can carry out Pixel-level to video and image data Mark, while it can be deployed in page end, facilitate multiple person cooperational；
In step (2), building suitable for road Edge check multitask convolutional neural networks and lose letter accordingly Steps are as follows for number:
(2-1) constructs roadside along feature extraction network, and the image information for that will input carries out feature extraction and feature is compiled Code can be used for the sharing feature floor of road Edge check, road vanishing Point Detection Method and road area detection.The network structure of this part As shown in Fig. 2, extracting the RGB information of each pixel in image first with conventional image procossing library, a spy is formed Levy tensor, the sharing feature layer that the characteristic tensor of input is exported after 3 down-samplings.Wherein single down-sampling is adopted Sample multiple is 2, by a maximum pond layer, convolutional layer and an a kind of ReLu (activation that a convolution kernel size is 3 × 3 Function) active coating constitute.
(2-2) constructs road vanishing Point Detection Method network, to the further convolution of sharing feature layer obtained in step (2-1), The disappearance dot position information of road can be obtained by its output layer.The network structure of this part is as shown in figure 3, input layer is passing through two The convolutional layer that a convolution kernel size is 1 × 1 exports end point hotspot graph；
(2-3) constructs road edge and detects network, as shown in figure 4, first by sharing feature layer obtained in step (2-1) It is connected with the output layer for detecting road end point obtained in step (2-2), i.e., folds two tensors in its third dimension Add to obtain the input layer of road edge detection network.2 up-samplings are carried out to this input layer, it is 2 that single, which up-samples multiple, is above adopted Quadrat method is deconvolution, and score of each pixel in roadside on mark is exported after a logical full articulamentum, to obtain road The location information at edge；
(2-4) constructs road area and detects network, as shown in figure 5, obtaining after up-sampling first to 2 times in step (2-3) Output layer carry out 2 down-samplings, the step of down-sampling and method is with the down-sampling in (2-1), and by this result and step (2- 1) sharing feature layer obtained in is connected, i.e., is superimposed two tensors in its third dimension, obtains road area detection net The input layer of network carries out 2 up-samplings to this input layer, and it is 2 that single, which up-samples multiple, and top sampling method is deconvolution, and is passed through It crosses a full articulamentum and exports score of each pixel on road area mark, to obtain road region information；
(2-5) building is suitable for the target detection loss function of class imbalance, for calculating the detection of road end point Loss, and influence caused by the imbalance of the ratio of end point and background in sample can be inhibited；
(2-6) constructs cross entropy loss function, for calculating the road edge position information of road edge detection network output With the loss of actual position information；
(2-7) constructs cross entropy loss function, for calculating the road area of road area detection network output and true Loss between road area.
In step (3), training network the step of it is as follows:
Acquired image is carried out data prediction by (3-1), and key step includes: to turn over image into row stochastic level Turn, cut and uniformly zoom to fixed size, labeled data is also overturn accordingly, cut and scaled, on this basis Obtained image is normalized by channel, the fixed dimension used in the present embodiment is 2048 × 1024；
(3-2) on ImageNet using SoftMax loss function satisfy the need side information extract network carry out pre-training, obtain Initial parameter of the parameter value arrived as network；
Vanishing Point Detection Method network and road Edge check are connected to the network by (3-3), by pretreated picture and labeled data It is input in network, the road end point of network output is calculated using the loss function constructed in step (2-5), step (2-6) And the penalty values of road edge placement and actual position, it carries out backpropagation and calculates gradient, and update network using Adam optimizer Parameter saves current network parameter when penalty values converge to its global minimum；
Road edge sence network is connected by (3-4) with road area detection network, utilizes the loss constructed in step (2-7) Penalty values between function calculating network output road area and real road region, progress backpropagation calculating gradient, and It is updated on the basis of step (3-3) using the parameter of Adam optimizer further progress network, obtains final result.
In short, the present invention provides a kind of road Edge Detection based on deep learning, to various visible and non-visible Roadside along suffering from preferable detectability.
Person skilled in the art obviously easily can make various modifications to these embodiments, and saying herein Bright General Principle is applied in other embodiments without having to go through creative labor.Therefore, the present invention is not limited to here Embodiment, those skilled in the art's announcement according to the present invention, improvement and modification made without departing from the scope of the present invention are all answered This is within protection scope of the present invention.

Claims (9)

1.一种基于深度学习的路边沿检测方法，其特征在于，包括如下步骤：1. a road edge detection method based on deep learning, is characterized in that, comprises the steps: (1)采集真实道路上包含路边沿的图像数据，并通过人工标注方法标注其中与路边沿检测相关的目标的位置和类别信息，构建路边沿检测的数据集；(1) Collect the image data containing the road edge on the real road, and mark the position and category information of the target related to the road edge detection through the manual annotation method, and construct the data set of the road edge detection; (2)构建适用于路边沿检测的多任务卷积神经网络及相应的损失函数；(2) Construct a multi-task convolutional neural network suitable for road edge detection and the corresponding loss function; (3)将采集到的图像和标注数据输入到步骤(2)构建的卷积神经网络中，根据输出值与目标值之间的损失值对神经网络中的参数值进行更新，当损失值收敛至其全局最小值时，保存最终的网络参数。(3) Input the collected images and labeled data into the convolutional neural network constructed in step (2), and update the parameter values in the neural network according to the loss value between the output value and the target value. When the loss value converges When it reaches its global minimum, save the final network parameters. 2.根据权利要求1所述的基于深度学习的路边沿检测方法，其特征在于，在步骤(1)中，图像数据的采集与标注步骤如下：2. the road edge detection method based on deep learning according to claim 1, is characterized in that, in step (1), the collection of image data and labeling step are as follows: (1-1)对摄像头的内外参数进行标定；(1-1) Calibrate the internal and external parameters of the camera; (1-2)针对实际工况下可能出现的路边沿种类，将采集的路边沿分为以下几种：真实可见的路边沿，路边沿旁存在障碍物，非真实可见的路边沿；(1-2) According to the types of road edges that may appear in actual working conditions, the collected road edges are divided into the following categories: real visible road edges, obstacles next to the road edges, and non-real visible road edges; (1-3)图像数据标注的类别主要分为以下几种：背景，可行驶区域，路边沿，非可行驶区域，障碍物；其中，对于非真实可见的路边沿利用人工识别标注出虚拟的路边沿位置；(1-3) The categories of image data annotation are mainly divided into the following categories: background, drivable area, road edge, non-drivable area, obstacle; among them, artificial recognition is used to mark the virtual road edge for the non-real and visible road edge. road edge location; (1-4)采用标注工具对采集到的图像数据进行标注，标注方式包括但不限于像素级标注，网格标注。(1-4) An annotation tool is used to annotate the collected image data, and the annotation methods include but are not limited to pixel-level annotation and grid annotation. 3.根据权利要求1所述的基于深度学习的路边沿检测方法，其特征在于，在步骤(2)中，构建的适用于路边沿检测的多任务卷积神经网络及相应的损失函数步骤如下：3. the road edge detection method based on deep learning according to claim 1 is characterized in that, in step (2), the multi-task convolutional neural network and corresponding loss function steps that are applicable to road edge detection constructed are as follows : (2-1)构建路边沿特征提取网络，用于将输入的图像信息进行特征提取与特征编码，得到多尺度的，可用于路边沿检测、道路消失点检测与道路区域检测的共享特征层；(2-1) Constructing a road edge feature extraction network for feature extraction and feature encoding of the input image information to obtain a multi-scale shared feature layer that can be used for road edge detection, road vanishing point detection and road area detection; (2-2)构建道路消失点检测网络，对步骤(2-1)中得到的共享特征层进一步卷积，由其输出层可得到道路的消失点位置信息；(2-2) Construct a road vanishing point detection network, further convolve the shared feature layer obtained in step (2-1), and obtain the vanishing point position information of the road from its output layer; (2-3)构建道路边沿检测网络，首先将步骤(2-1)中得到的共享特征层与步骤(2-2)中得到的检测道路消失点的输出层相连得到道路边沿检测网络的输入层；对此输入层进行上采样，通过最终的输出层可得到路边沿信息；(2-3) Construct a road edge detection network. First, connect the shared feature layer obtained in step (2-1) with the output layer for detecting road vanishing points obtained in step (2-2) to obtain the input of the road edge detection network. layer; upsampling this input layer, and the road edge information can be obtained through the final output layer; (2-4)构建道路区域检测网络，首先对步骤(2-3)中得到的输出层进行下采样并将此结果与步骤(2-1)中得到的共享特征层相连得到道路区域检测网络的输入层，对此输入层进行上采样，通过最终的输出层可得到道路区域信息；(2-4) Construct a road area detection network. First, downsample the output layer obtained in step (2-3) and connect this result with the shared feature layer obtained in step (2-1) to obtain a road area detection network. The input layer is upsampled, and the road area information can be obtained through the final output layer; (2-5)构建适用于类别不平衡的目标检测损失函数，用于计算道路消失点的检测损失，并能抑制样本中消失点与背景的比例的不平衡所造成的影响；(2-5) Construct a target detection loss function suitable for class imbalance, which is used to calculate the detection loss of road vanishing points, and can suppress the influence caused by the imbalance of the ratio of vanishing points to background in the sample; (2-6)构建交叉熵损失函数，用于计算道路边沿检测网络输出的路边沿位置信息与真实位置信息的损失；(2-6) Constructing a cross-entropy loss function, which is used to calculate the loss of the road edge position information and the real position information output by the road edge detection network; (2-7)构建交叉熵损失函数，用于计算道路区域检测网络输出的道路区域与真实道路区域之间的损失。(2-7) A cross-entropy loss function is constructed to calculate the loss between the road area output by the road area detection network and the real road area. 4.根据权利要求3所述的基于深度学习的路边沿检测方法，其特征在于：在步骤(2-1)中，首先利用常规的图像处理库提取出图像中每个像素点的RGB信息，形成一个特征张量，将输入的特征张量经过3次下采样后得到输出的共享特征层；其中单次下采样的采样倍数为2，由一个最大池化层，一个卷积核大小为3×3的卷积层以及一个激活函数的激活层构成。4. the road edge detection method based on deep learning according to claim 3, is characterized in that: in step (2-1), first utilize conventional image processing library to extract the RGB information of each pixel in the image, A feature tensor is formed, and the input feature tensor is downsampled 3 times to obtain the output shared feature layer; the sampling multiple of a single downsampling is 2, a maximum pooling layer is used, and a convolution kernel size is 3 The convolutional layer of ×3 and the activation layer of an activation function are composed. 5.根据权利要求4所述的基于深度学习的路边沿检测方法，其特征在于：所述激活函数是ReLu。5 . The road edge detection method based on deep learning according to claim 4 , wherein the activation function is ReLu. 6 . 6.根据权利要求3所述的基于深度学习的路边沿检测方法，其特征在于：在步骤(2-2)中，输入层在经过两个卷积核大小为1×1的卷积层，输出消失点热点。6. The road edge detection method based on deep learning according to claim 3, is characterized in that: in step (2-2), the input layer is 1 × 1 convolution layer after passing through two convolution kernels, Output the vanishing point hotspot. 7.根据权利要求3所述的基于深度学习的路边沿检测方法，其特征在于：在步骤(2-3)中，将两个张量在其第三个维度上叠加得到道路边沿检测网络的输入层；对此输入层进行2次上采样，单次上采样倍数为2，上采样方法为反卷积，并通一个全连接层后输出每个像素点在路边沿标注上的得分，以得到路边沿的位置信息。7. The road edge detection method based on deep learning according to claim 3, is characterized in that: in step (2-3), two tensors are superimposed on its third dimension to obtain the road edge detection network. Input layer: perform 2 upsampling on this input layer, the single upsampling multiple is 2, the upsampling method is deconvolution, and after passing through a fully connected layer, the score of each pixel on the road edge label is output, with Get the location information of the road edge. 8.根据权利要求3所述的基于深度学习的路边沿检测方法，其特征在于：在步骤(2-4)中，下采样的步骤及方法同(2-1)中的下采样，并将此结果与步骤(2-1)中得到的共享特征层相连，即将两个张量在其第三个维度上叠加，得到道路区域检测网络的输入层，对此输入层进行2次上采样，单次上采样倍数为2，上采样方法为反卷积，并经过一个全连接层输出每个像素点在道路区域标注上的得分，以得到道路区域信息。8. The road edge detection method based on deep learning according to claim 3, characterized in that: in step (2-4), the step and method of downsampling are the same as the downsampling in (2-1), and the This result is connected to the shared feature layer obtained in step (2-1), that is, the two tensors are superimposed on its third dimension to obtain the input layer of the road area detection network, and this input layer is upsampled twice, The single upsampling multiple is 2, the upsampling method is deconvolution, and the score of each pixel on the road area label is output through a fully connected layer to obtain road area information. 9.根据权利要求1所述的基于深度学习的路边沿检测方法，其特征在于，在步骤(3)中，训练网络的步骤如下：9. the road edge detection method based on deep learning according to claim 1, is characterized in that, in step (3), the step of training network is as follows: (3-1)将采集到的图像进行数据预处理，包括：将图像进行随机的水平翻转、裁剪并统一缩放到固定的尺寸，标注数据也进行相应的翻转、裁剪和缩放，在此基础上对得到的图像按通道进行归一化处理；(3-1) Perform data preprocessing on the collected images, including: randomly flipping, cropping and uniformly scaling the images to a fixed size horizontally, and flipping, cropping and scaling the labeled data accordingly. Normalize the obtained image by channel; (3-2)在ImageNet上利用SoftMax损失函数对上述网络进行预训练，得到的参数值作为网络的初始参数；(3-2) Use SoftMax loss function to pre-train the above network on ImageNet, and the obtained parameter values are used as the initial parameters of the network; (3-3)将消失点检测网络与路边沿检测网络连接，将预处理后的图片与标注数据输入到网络中，利用步骤(2-5)、步骤(2-6)中构建的损失函数计算网络输出的道路消失点及路边沿位置与真实位置的损失值，对参数值进行更新，当损失值收敛至其全局最小值时，保存当前的网络参数；(3-3) Connect the vanishing point detection network to the road edge detection network, input the preprocessed image and label data into the network, and use the loss function constructed in steps (2-5) and (2-6) Calculate the loss value of the road vanishing point and the road edge position and the real position output by the network, update the parameter value, and save the current network parameters when the loss value converges to its global minimum value; (3-4)将路边沿检测网络与道路区域检测网络相连，利用步骤(2-7)中构建的损失函数计算网络输出道路区域与实际道路区域之间的损失值，并在步骤(3-3)的基础上进一步进行网络的参数更新，得到最终结果。(3-4) Connect the road edge detection network with the road area detection network, use the loss function constructed in step (2-7) to calculate the loss value between the network output road area and the actual road area, and in step (3- On the basis of 3), the parameters of the network are further updated to obtain the final result.
CN201811329308.5A 2018-11-09 2018-11-09 A kind of road Edge Detection based on deep learning Pending CN109558808A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811329308.5A CN109558808A (en) 2018-11-09 2018-11-09 A kind of road Edge Detection based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811329308.5A CN109558808A (en) 2018-11-09 2018-11-09 A kind of road Edge Detection based on deep learning

Publications (1)

Publication Number Publication Date
CN109558808A true CN109558808A (en) 2019-04-02

Family

ID=65866158

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811329308.5A Pending CN109558808A (en) 2018-11-09 2018-11-09 A kind of road Edge Detection based on deep learning

Country Status (1)

Country Link
CN (1) CN109558808A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110263628A (en) * 2019-05-09 2019-09-20 杭州飞步科技有限公司 Obstacle detection method, device, electronic equipment and storage medium
CN111027581A (en) * 2019-08-23 2020-04-17 中国地质大学（武汉） A 3D target detection method and system based on learnable coding
CN111402200A (en) * 2020-02-18 2020-07-10 江苏大学 Fried food detection system based on symbiotic two-stream convolutional network and digital images
CN113887425A (en) * 2021-09-30 2022-01-04 北京工业大学 Low-computational-power-computing-device-oriented lightweight object detection method and system
CN114091506A (en) * 2021-08-24 2022-02-25 广东盈峰智能环卫科技有限公司 A camera-based road edge detection method and device
CN114339049A (en) * 2021-12-31 2022-04-12 深圳市商汤科技有限公司 A video processing method, apparatus, computer equipment and storage medium
CN114821501A (en) * 2022-04-29 2022-07-29 北京智行者科技有限公司 A road boundary classification method and device, computer equipment and mobile tool

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107092862A (en) * 2017-03-16 2017-08-25 浙江零跑科技有限公司 A kind of track edge detection method based on convolutional neural networks
JP2018088151A (en) * 2016-11-29 2018-06-07 株式会社Ｓｏｋｅｎ Boundary line estimation device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018088151A (en) * 2016-11-29 2018-06-07 株式会社Ｓｏｋｅｎ Boundary line estimation device
CN107092862A (en) * 2017-03-16 2017-08-25 浙江零跑科技有限公司 A kind of track edge detection method based on convolutional neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ROMANO LOTTERING: "Estimating the road edge effect on adjacent Eucalyptus grandis forests in KwaZulu-Natal, South Africa, using texture measures and an artificial neural network", 《SPATIAL SCIENCE》 *
刘如意等: "一种自动的高分辨率遥感影像道路提取方法", 《西安电子科技大学学报》 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110263628A (en) * 2019-05-09 2019-09-20 杭州飞步科技有限公司 Obstacle detection method, device, electronic equipment and storage medium
CN110263628B (en) * 2019-05-09 2021-11-23 杭州飞步科技有限公司 Obstacle detection method, obstacle detection device, electronic apparatus, and storage medium
CN111027581A (en) * 2019-08-23 2020-04-17 中国地质大学（武汉） A 3D target detection method and system based on learnable coding
CN111402200A (en) * 2020-02-18 2020-07-10 江苏大学 Fried food detection system based on symbiotic two-stream convolutional network and digital images
CN111402200B (en) * 2020-02-18 2021-12-21 江苏大学 Fried food detection system based on symbiotic two-stream convolutional network and digital images
CN114091506A (en) * 2021-08-24 2022-02-25 广东盈峰智能环卫科技有限公司 A camera-based road edge detection method and device
CN113887425A (en) * 2021-09-30 2022-01-04 北京工业大学 Low-computational-power-computing-device-oriented lightweight object detection method and system
CN113887425B (en) * 2021-09-30 2024-04-12 北京工业大学 Lightweight object detection method and system for low-computation-force computing device
CN114339049A (en) * 2021-12-31 2022-04-12 深圳市商汤科技有限公司 A video processing method, apparatus, computer equipment and storage medium
CN114821501A (en) * 2022-04-29 2022-07-29 北京智行者科技有限公司 A road boundary classification method and device, computer equipment and mobile tool

Similar Documents

Publication Publication Date Title
CN109558808A (en) A kind of road Edge Detection based on deep learning
CN111582083B (en) A Lane Line Detection Method Based on Vanishing Point Estimation and Semantic Segmentation
Shim et al. Road surface damage detection based on hierarchical architecture using lightweight auto-encoder network
CN108846328B (en) Lane Detection Method Based on Geometric Regularization Constraints
WO2020062433A1 (en) Neural network model training method and method for detecting universal grounding wire
CN110008848A (en) A kind of travelable area recognizing method of the road based on binocular stereo vision
CN103383733B (en) A kind of track based on half machine learning video detecting method
CN108460403A (en) The object detection method and system of multi-scale feature fusion in a kind of image
CN102629326A (en) Lane line detection method based on monocular vision
CN112488046B (en) A lane line extraction method based on high-resolution UAV images
CN109583321A (en) The detection method of wisp in a kind of structured road based on deep learning
CN105261020A (en) Method for detecting fast lane line
CN107578091B (en) Pedestrian and vehicle real-time detection method based on lightweight deep network
CN108416292A (en) A kind of unmanned plane image method for extracting roads based on deep learning
CN108830171A (en) A kind of Intelligent logistics warehouse guide line visible detection method based on deep learning
CN111401150A (en) Multi-lane line detection method based on example segmentation and adaptive transformation algorithm
CN110532961A (en) A kind of semantic traffic lights detection method based on multiple dimensioned attention mechanism network model
CN104700072A (en) Lane line historical frame recognition method
Chao et al. Multi-lane detection based on deep convolutional neural network
CN108876805B (en) An End-to-End Unsupervised Scenario Passable Region Cognition and Understanding Approach
CN104915642B (en) Front vehicles distance measuring method and device
CN115238758A (en) A multi-task 3D object detection method based on point cloud feature enhancement
CN106056102A (en) Road vehicle classification method based on video image analysis
KR101224027B1 (en) Method for dectecting front vehicle using scene information of image
CN116503834A (en) Traffic road surface attachment coefficient prediction model and prediction method based on semantic segmentation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20190402

WD01 Invention patent application deemed withdrawn after publication