[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Lightweight lane marking detection CNNs by self soft label attention

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Lane detection models based deep semantic segmentation often fail in challenging scenarios for lacking enough contextual information. In this paper, we first point out that a soft label extracted from a trained semantic segmentation model is capable of encoding rich contextual information. Then, we find that introducing the soft label extracted from the top layer of a well-trained model to low-level features gains substantial improvement. Based on this observation, we propose a novel framework, i.e., Soft Label Attention(SLA), which allows a model to learn contextual attention from the higher layer through a loss function. Significantly, our SLA does not increase the run time in the inference stage because it is only applied in the training stage. We apply the proposed SLA to modified ERFNet and propose an attention deep neural network (DNN) for lane detection (ERFNet-SLA). Tests on the TuSimple dataset show that the ERFNet-SLA outperforms ResNet-18 and ResNet-34 by 3.4% and 3.3% respectively with at least 2.3× faster in computation speed, and is 12.5× faster than SCNN with 0.6% performance loss. Tests on the CULane dataset show that the ERFNet-SLA outperforms ResNet-50 by 5.6% and is 16× faster than ResNet-101 with 0.5% performance loss.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Bottou L (2010) Large-scale machine learning with stochastic gradient descent. COMPSTAT 177–186

  2. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  3. Chiu K-Y, Lin S-F (2005) Lane detection using color-based segmentation. In: IEEE Proceedings. Intelligent vehicles symposium, 2005. pp 706–711

  4. Fan G-F, Yu M, Dong S-Q, Yeh Y-H, Hong W-C (2021) Forecasting short-term electricity load using hybrid support vector regression with grey catastrophe and random forest modeling. Util Policy 73:101294

    Article  Google Scholar 

  5. Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 3146–3154

  6. Ghafoorian M, Nugteren C, Baka N, Booij O, Hofmann M (2018) El-gan: embedding loss driven generative adversarial networks for lane detection. In: Proceedings of the european conference on computer vision (ECCV) Workshops, pp 256–272

  7. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580

  8. Hinton GE, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv:1503.02531

  9. Hou Y, Ma Z, Liu C, Loy CC (2019) Learning lightweight lane detection cnns by self attention distillation. In: 2019 IEEE/CVF International conference on computer vision (ICCV), pp 1013–1021

  10. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 7132–7141

  11. Huang G, Chen D, Li T, Wu F, van der Maaten L, Weinberger KQ (2018) Multi-scale dense networks for resource efficient image classification. In: International conference on learning representations

  12. Hur J, Kang S-N, Seo S-W (2013) Multi-lane detection in urban driving environments using conditional random fields. In: 2013 IEEE Intelligent vehicles symposium (IV), pp 1297–1302

  13. Jung H, Min J, Kim J (2013) An efficient lane detection algorithm for lane departure detection. In: 2013 IEEE Intelligent vehicles symposium (IV), pp 976–981

  14. Li Z-Q, Ma H-M, Liu Z-Y (2016) Road lane detection with gabor filters. In: 2016 International conference on information system and artificial intelligence (ISAI), pp 436–440

  15. Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 510–519

  16. Ma N, Zhang X, Zheng H-T, Sun J (2018) Shufflenet v2: Practical guidelines for efficient cnn architecture design. In: Proceedings of the european conference on computer vision (ECCV), pp 122– 138

  17. Neven D, Brabandere BD, Georgoulis S, Proesmans M, Gool LV (2018) Towards end-to-end lane detection: an instance segmentation approach. In: 2018 IEEE Intelligent vehicles symposium (IV), pp 286–291

  18. Pan X, Shi J, Luo P, Wang X (2018) xiaoou Tang: spatial as deep: spatial cnn for traffic scene understanding. In: AAAI-18 AAAI Conference on artificial intelligence, pp 7276–7283

  19. Paszke A, Chaurasia A, Kim S, Culurciello E (2016) Enet: a deep neural network architecture for real-time semantic segmentation. arXiv:1606.02147

  20. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in pytorch

  21. Romera E, Alvarez JM, Bergasa LM, Arroyo R (2018) Erfnet: efficient residual factorized convnet for real-time semantic segmentation. IEEE Trans Intell Transp Syst 19(1):263–272

    Article  Google Scholar 

  22. Salvaris M, Dean D, Tok WH (2018) Generative adversarial networks. arXiv:1406.2661, 187–208

  23. Srinivasu PN, Bhoi AK, Jhaveri RH, Reddy GT, Bilal M (2021) Probabilistic deep q network for real-time path planning in censorious robotic procedures using force sensors. J Real-Time Image Proc 18(5):1773–1785

    Article  Google Scholar 

  24. Srinivasu PN, SivaSai JG, Ijaz MF, Bhoi AK, Kim W, Kang JJ (2021) Classification of skin disease using deep learning neural networks with mobilenet v2 and lstm. Sensors 21(8):2852

    Article  Google Scholar 

  25. Wang Y, Shen D, Teoh EK (2000) Lane detection using spline model. Pattern Recogn Lett 21(9):677–689

    Article  Google Scholar 

  26. Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the european conference on computer vision (ECCV), pp 3–19

  27. Xiao D, Yang X, Li J, Islam M (2020) Attention deep neural network for lane marking detection. Knowl Based Syst 194:105584

    Article  Google Scholar 

  28. Xue X, Feng J, Gao Y, Liu M, Zhang W, Sun X, Zhao A, Guo S (2019) Convolutional recurrent neural networks with a self-attention mechanism for personnel performance prediction. Entropy 21(12):1227

    Article  Google Scholar 

  29. Youjin T, Wei C, Xingguang L, Lei C (2018) A robust lane detection method based on vanishing point estimation. Procedia Comput Sci 131:354–360

    Article  Google Scholar 

  30. Yu F, Koltun V (2016) Multi-scale context aggregation by dilated convolutions. In: ICLR 2016: International Conference on learning representations 2016

  31. Zagoruyko S, Komodakis N (2017) Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. In: ICLR 2017: International Conference on learning representations 2017

  32. Zhang H, Goodfellow I, Metaxas D, Odena A (2019) Self-attention generative adversarial networks. In: ICML 2019: thirty-Sixth international conference on machine learning, pp 7354–7363

  33. Zhang R, Xiong Z (2019) Recurrent neural network model with self-attention mechanism for fault detection and diagnosis. In: 2019 Chinese Automation Congress (CAC), pp 4706–4711. IEEE

  34. Zhang Z, Zhang X, Peng C, Xue X, Sun J (2018) Exfuse: enhancing feature fusion for semantic segmentation. In: Proceedings of the european conference on computer vision (ECCV), pp 273–288

Download references

Acknowledgments

This work is supported by Zhejiang Dahua Technology Co., Ltd.

Author information

Authors and Affiliations

  1. Zhejiang Dahua Technology Co., Ltd., Bin’an Road, Hangzhou City, Zhejiang Province, China

    Xuefeng Yang, Yanxun Yu, Zhen Zhang, Yu Huang, Zhihui Liu, Zhongbin Niu, Hongwei Chai, Chenglu Wu, Zhijiang Du & Song Li

Authors
  1. Xuefeng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanxun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhihui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhongbin Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Hongwei Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Chenglu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Zhijiang Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Song Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xuefeng Yang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Yu, Y., Zhang, Z. et al. Lightweight lane marking detection CNNs by self soft label attention. Multimed Tools Appl 82, 5607–5626 (2023). https://doi.org/10.1007/s11042-022-13442-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13442-6

Keywords

Search

Navigation