[go: up one dir, main page]
Skip to main content
SpringerLink
Log in

BEACon: a boundary embedded attentional convolution network for point cloud instance segmentation

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Motivated by how humans perceive geometry and color to recognize objects, we propose a boundary embedded attentional convolution (BEACon) network for point cloud instance segmentation. At the core of BEACon, we introduce the attentional weight in the convolution layer to adjust the neighboring features, with the weight being adapted to the relationship between geometry and color changes. As a result, BEACon makes use of both geometry and color information, takes instance boundary as an important feature, and thus learns a more discriminative feature representation in the neighborhood. Experimental results show that BEACon outperforms the state-of-the-art by a large margin. Ablation studies are also provided to prove the large benefit of incorporating both geometry and color into attention weight for instance segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Haoran, L., Fazhi, H., Yilin, C.: Learning dynamic simultaneous clustering and classification via automatic differential evolution and firework algorithm. Appl. Soft Comput. J. 96, 106593 (2020)

    Article  Google Scholar 

  2. Zhang, S., He, F.: DRCDN: learning deep residual convolutional dehazing networks. Vis. Comput. 36, 1797–1808 (2020)

    Article  Google Scholar 

  3. Li, Y., Bu, R., Sun, M., Chen, B.: PointCNN: Convolution On X-Transformed Points. In: Advances in Neural Information Processing Systems (2018)

  4. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems (2017)

  5. Rethage, D., Wald, J., Sturm, J, Navab, N., Tombari, F.: Fully-convolutional point networks for large-scale point clouds. In: ECCV (2018)

  6. Wu, W., Qi, Z., Fuxin, L.: PointConv: Deep convolutional networks on 3D point clouds. In: CVPR (2019)

  7. Xu, Y., Fan, T., Xu, M., Zeng, L., Qiao, Y.: SpiderCNN: Deep learning on point sets with parameterized convolutional filters. In: ECCV (2018)

  8. Li, H., He, F., Liang, Y., Quan, Q.: A dividing-based many-objective evolutionary algorithm for large-scale feature selection. Soft Comput. 24, 6851–6870 (2020)

    Article  Google Scholar 

  9. Liu, Y., Fan, B., Xiang, S., Pan, C.: Relation-shape convolutional neural network for point cloud analysis. In: CVPR (2019)

  10. Wang, L., Huang, Y., Hou, Y., Zhang, S., Shan, J.: Graph attention convolution for point cloud semantic segmentation. In: CVPR (2019)

  11. Zhao, H., Jiang, L., Fu, C.-W., Jia, J.: PointWeb: Enhancing local neighborhood features for point cloud processing. In: CVPR (2019)

  12. Landrieu, L., Obozinski, G.: Cut Pursuit: fast algorithms to learn piecewise constant functions on general weighted graphs. SIAM J. Imaging Sci. Soc. Ind. Appl. Math. 10(4), 1724–1766 (2017)

    Article  MathSciNet  Google Scholar 

  13. Armeni, I., Sener, O., Zamir, A.R., Jiang, H., Brilakis, I., Fischer, M., Savarese, S.: 3D semantic parsing of large-scale indoor spaces. In: CVPR, pp. 1534–1543 (2016)

  14. Mo, K., Zhu, S., Chang, A.X., Yi, L., Tripathi, S., Guibas, L.J., Su, H.: PartNet: A large-scale Benchmark for fine-grained and hierarchical part-level 3d object understanding. In: CVPR (2019)

  15. Maturana, D., Scherer, S.: “VoxNet: A 3D convolutional neural network for real-time object recognition. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2015)

  16. Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3D ShapeNets: A deep representation for volumetric shapes. In: CVPR, vol. 07-12-June, pp. 1912–1920 (2015)

  17. Graham, B., Engelcke, M., Van Der Maaten, L.: 3D semantic segmentation with submanifold sparse convolutional networks. In: CVPR (2018)

  18. Choy, C., Gwak, J., Savarese, S.: 4D Spatio-temporal ConvNets: Minkowski convolutional neural networks. In: CVPR (2019)

  19. Boulch, A., Guerry, J., Le Saux, B., Audebert, N.: SnapNet: 3D point cloud semantic labeling with 2D deep segmentation networks. Comput. Gr. 71, 189–198 (2018)

    Article  Google Scholar 

  20. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: Deep learning on point sets for 3d classification and segmentation. In: CVPR, pp. 601–610 (2017)

  21. Ye, X., Li, J., Huang, H., Du, L., Zhang, X.: 3D Recurrent neural networks with context fusion for point cloud semantic segmentation. In: ECCV, pp. 403–417 (2018)

  22. Liu, S., Xie, S., Chen, Z., Tu, Z.: Attentional ShapeContextNet for point cloud recognition. In: CVPR, pp. 4606–4615 (2018)

  23. Wang, Y., Bronstein, M.M., Solomon, J.M., Sun, Y., Liu, Z., Sarma, S.E.: Dynamic graph CNN for learning on point clouds. In: ACM Trans. Graph. 1, 1, Article, vol. 1, No. 1, p. 13 (2019)

  24. Landrieu, L., Simonovsky, M.: Large-scale point cloud semantic segmentation with superpoint graphs. In: CVPR (2018)

  25. Landrieu, L., Boussaha, M.: Point cloud oversegmentation with graph-structured deep metric learning. In: CVPR (2019)

  26. Sun, Y., Miao, Y., Chen, J., Pajarola, R.: PGCNet: patch graph convolutional network for point cloud segmentation of indoor scenes. Vis. Comput. 36, 2407–2418 (2020)

    Article  Google Scholar 

  27. Wang, C., Samari, B., Siddiqi, K.: Local spectral graph convolution for point set feature learning. In: ECCV (2018)

  28. Li, H., Sun, Z.: A structural-constraint 3D point clouds segmentation adversarial method. Vis. Comput. 37, 325 (2020)

    Article  Google Scholar 

  29. Lei, H., Akhtar, N., Mian, A.: Octree guided CNN with Spherical Kernels for 3D Point Clouds. In: CVPR (2019)

  30. Komarichev, A., Zhong, Z., Hua, J.: A-CNN: Annularly convolutional neural networks on point clouds. In: CVPR (2019)

  31. Thomas, H., Qi, C.R., Deschaud, J.-E., Marcotegui, B., Goulette, F., Guibas, L.J.: “KPConv: flexible and deformable convolution for point clouds. In: ICCV (2019)

  32. Wang, W., Yu, R., Huang, Q., Neumann, U.: SGPN: Similarity group proposal network for 3D point cloud instance segmentation. In: CVPR (2018)

  33. Wang, X., Liu, S., Shen, X., Shen, C., Jia, J.: Associatively segmenting instances and semantics in point clouds. In: CVPR (2019)

  34. Pham, Q.-H., Thanh Nguyen, D., Hua Gemma Roig, B.-S., Yeung, S.-K.: JSIS3D: Joint semantic-instance segmentation of 3D point clouds with multi-task pointwise networks and multi-value conditional random fields. In: CVPR (2019)

  35. Hou, J., Dai, A., Nießner, M.: 3D-SIS: 3D semantic instance segmentation of RGB-D scans. In: CVPR (2019)

  36. Yang, B., Wang, J., Clark, R., Hu, Q., Wang, S., Markham, A., Trigoni, N.: Learning object bounding boxes for 3D instance segmentation on point clouds. In: NeurIPS (2019)

  37. Lahoud, J., Ghanem, B., Pollefeys, M., Zurich, E., Oswald, M.R.: 3D instance segmentation via multi-task metric learning. In: ICCV (2019)

  38. Groh, Fabian, Wieschollek, Patrick, Lensch, Hendrik P.A.: Flex-convolution million-scale point-cloud learning beyond grid-worlds. In: ACCV (2018)

  39. Thomas, H., Deschaud, J.-E., Marcotegui, B., Goulette, F., Le Gall, Y.: Semantic classification of 3D point clouds with multiscale spherical neighborhoods. In: International Conference on 3D Vision (3DV), pp. 390–398 (2018)

  40. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–8(6), 679–698 (1986)

    Article  Google Scholar 

  41. De Brabandere, B., Neven, D., Gool, L.V.: Semantic instance segmentation with a discriminative loss function. In: CVPR Workshop (2017)

  42. Jiang, L., Zhao, H., Liu, S., Shen, X., Fu, C.-W., Jia, J.: Hierarchical point-edge interaction network for point cloud semantic segmentation. In: ICCV (2019)

  43. Li, Y., Zhao, W., Wang, H., Sung, M., Guibas, L.: GSPN: generative shape proposal network for 3D instance segmentation in point cloud. In: CVPR (2019)

Download references

Acknowledgements

This research is supported by the National Research Foundation, Singapore, under its International Research Centres in Singapore Funding Initiative. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore.

Author information

Authors and Affiliations

  1. Institute for Media Innovation, Interdisciplinary Graduate School, Nanyang Technological University, Singapore, Singapore

    Tianrui Liu

  2. School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore, Singapore

    Yiyu Cai

  3. School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore

    Jianmin Zheng & Nadia Magnenat Thalmann

Authors
  1. Tianrui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiyu Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianmin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nadia Magnenat Thalmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tianrui Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, T., Cai, Y., Zheng, J. et al. BEACon: a boundary embedded attentional convolution network for point cloud instance segmentation. Vis Comput 38, 2303–2313 (2022). https://doi.org/10.1007/s00371-021-02112-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02112-7

Keywords

Search

Navigation