[go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Transfer Learning for On-Orbit Ship Segmentation

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 Workshops (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13801))

Included in the following conference series:

Abstract

With the adoption of edge AI processors for space, on-orbit inference on EO data has become a possibility. This enables a range of new applications for space-based EO systems. Since the development of on-orbit AI applications requires rarely available raw data, training of these AI networks remains a challenge. To address this issue, we investigate the effects of varying two key image parameters between training and testing data on a ship segmentation network: Ground Sampling Distance and band misalignment magnitude. Our results show that for both parameters the network exhibits degraded performance if these parameters differ in testing data with respect to training data. We show that this performance drop can be mitigated with appropriate data augmentation. By preparing models at the training stage for the appropriate feature space, the need for additional computational resources on-board for e.g. image scaling or band-alignment of camera data can be mitigated.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Chavez, P.S., et al.: Image-based atmospheric corrections-revisited and improved. Photogramm. Eng. Remote. Sens. 62(9), 1025–1035 (1996)

    Google Scholar 

  2. Cheng, G., Han, J.: A survey on object detection in optical remote sensing images. ISPRS J. Photogramm. Remote. Sens. 117, 11–28 (2016)

    Article  Google Scholar 

  3. Cheng, G., Han, J., Zhou, P., Guo, L.: Multi-class geospatial object detection and geographic image classification based on collection of part detectors. ISPRS J. Photogramm. Remote. Sens. 98, 119–132 (2014)

    Article  Google Scholar 

  4. Cihlar, J.: Land cover mapping of large areas from satellites: status and research priorities. Int. J. Remote Sens. 21(6–7), 1093–1114 (2000)

    Article  Google Scholar 

  5. MMS Contributors: MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark (2020). https://github.com/open-mmlab/mmsegmentation

  6. Dai, W., Jin, O., Xue, G.R., Yang, Q., Yu, Y.: EigenTransfer: a unified framework for transfer learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 193–200 (2009)

    Google Scholar 

  7. Franklin, S.E., Giles, P.T.: Radiometric processing of aerial and satellite remote-sensing imagery. Comput. Geosci. 21(3), 413–423 (1995)

    Article  Google Scholar 

  8. Giuffrida, G., et al.: CloudScout: a deep neural network for on-board cloud detection on hyperspectral images. Remote Sens. 12(14), 2205 (2020)

    Article  Google Scholar 

  9. Giuffrida, G., et al.: The \(\phi \)-sat-1 mission: the first on-board deep neural network demonstrator for satellite earth observation. IEEE Trans. Geosci. Remote Sens. 60, 1–14 (2021)

    Article  Google Scholar 

  10. Gong, P.: Remote sensing of environmental change over china: a review. Chin. Sci. Bull. 57(22), 2793–2801 (2012)

    Article  Google Scholar 

  11. Gupta, R., Hartley, R.I.: Linear pushbroom cameras. IEEE Trans. Pattern Anal. Mach. Intell. 19(9), 963–975 (1997)

    Article  Google Scholar 

  12. Guzmán, R., et al.: A compact multispectral imager for the MANTIS mission 12U CubeSat. In: CubeSats and SmallSats for Remote Sensing IV, vol. 11505, p. 1150507. SPIE (2020)

    Google Scholar 

  13. Hirschmüller, H., Scholten, F., Hirzinger, G.: Stereo vision based reconstruction of huge urban areas from an airborne Pushbroom camera (HRSC). In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 58–66. Springer, Heidelberg (2005). https://doi.org/10.1007/11550518_8

    Chapter  Google Scholar 

  14. Honkavaara, E., Rosnell, T., Oliveira, R., Tommaselli, A.: Band registration of tuneable frame format hyperspectral UAV imagers in complex scenes. ISPRS J. Photogramm. Remote. Sens. 134, 96–109 (2017)

    Article  Google Scholar 

  15. Iwasaki, A.: Detection and estimation satellite attitude jitter using remote sensing imagery. Adv. Spacecraft Technol. 13, 257–272 (2011)

    Google Scholar 

  16. Kang, M., Ji, K., Leng, X., Lin, Z.: Contextual region-based convolutional neural network with multilayer fusion for SAR ship detection. Remote Sens. 9(8), 860 (2017)

    Article  Google Scholar 

  17. Kouw, W.M., Loog, M.: An introduction to domain adaptation and transfer learning. arXiv preprint arXiv:1812.11806 (2018)

  18. Kramer, H.J., et al.: Observation of the Earth and its Environment: Survey of Missions and Sensors, vol. 1982. Springer, Heidelberg (2002). https://doi.org/10.1007/978-3-642-56294-5

  19. Lin, X., Xu, Q., Han, C.: Shoreline data based sea-land segmentation method for on-orbit ship detection from panchromatic images. In: 2018 Fifth International Workshop on Earth Observation and Remote Sensing Applications (EORSA), pp. 1–5. IEEE (2018)

    Google Scholar 

  20. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. CoRR abs/1411.4038 (2014). https://arxiv.org/abs/1411.4038

  21. Manfreda, S., et al.: On the use of unmanned aerial systems for environmental monitoring. Remote Sens. 10(4), 641 (2018)

    Article  Google Scholar 

  22. Mateo-García, G., Laparra, V., López-Puigdollers, D., Gómez-Chova, L.: Transferring deep learning models for cloud detection between Landsat-8 and Proba-V. ISPRS J. Photogramm. Remote. Sens. 160, 1–17 (2020)

    Article  Google Scholar 

  23. Mateo-Garcia, G., et al.: Towards global flood mapping onboard low cost satellites with machine learning. Sci. Rep. 11(1), 1–12 (2021)

    Article  Google Scholar 

  24. Mitchell, T.M.: Machine learning and data mining. Commun. ACM 42(11), 30–36 (1999)

    Article  Google Scholar 

  25. Moran, M.S., Jackson, R.D., Slater, P.N., Teillet, P.M.: Evaluation of simplified procedures for retrieval of land surface reflectance factors from satellite sensor output. Remote Sens. Environ. 41(2–3), 169–184 (1992)

    Article  Google Scholar 

  26. Navalgund, R.R., Jayaraman, V., Roy, P.: Remote sensing applications: an overview. Current Sci. 93, 1747–1766 (2007)

    Google Scholar 

  27. Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)

    Article  Google Scholar 

  28. Peng, D., Zhang, Y., Guan, H.: End-to-end change detection for high resolution satellite images using improved UNet++. Remote Sens. 11(11), 1382 (2019)

    Article  Google Scholar 

  29. Perrier, R., Arnaud, E., Sturm, P., Ortner, M.: Estimation of an observation satellite’s attitude using multimodal Pushbroom cameras. IEEE Trans. Pattern Anal. Mach. Intell. 37(5), 987–1000 (2014)

    Article  Google Scholar 

  30. Phiri, D., Simwanda, M., Salekin, S., Nyirenda, V.R., Murayama, Y., Ranagalage, M.: Sentinel-2 data for land cover/use mapping: a review. Remote Sens. 12(14), 2291 (2020)

    Article  Google Scholar 

  31. Qin, J., Chao, K., Kim, M.S., Lu, R., Burks, T.F.: Hyperspectral and multispectral imaging for evaluating food safety and quality. J. Food Eng. 118(2), 157–171 (2013)

    Article  Google Scholar 

  32. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation (2015). https://doi.org/10.48550/ARXIV.1505.04597. https://arxiv.org/abs/1505.04597

  33. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

    Google Scholar 

  34. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks (2018). https://doi.org/10.48550/ARXIV.1801.04381. https://arxiv.org/abs/1801.04381

  35. Shi, H., He, G., Feng, P., Wang, J.: An on-orbit ship detection and classification algorithm for SAR satellite. In: IGARSS 2019–2019 IEEE International Geoscience and Remote Sensing Symposium, pp. 1284–1287. IEEE (2019)

    Google Scholar 

  36. Shi, W., Zhang, M., Zhang, R., Chen, S., Zhan, Z.: Change detection based on artificial intelligence: state-of-the-art and challenges. Remote Sens. 12(10), 1688 (2020)

    Article  Google Scholar 

  37. Simard, P.Y., Steinkraus, D., Platt, J.C., et al.: Best practices for convolutional neural networks applied to visual document analysis. In: ICDAR, vol. 3. Edinburgh (2003)

    Google Scholar 

  38. Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 240–248. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_28

    Chapter  Google Scholar 

  39. Tang, J., Deng, C., Huang, G.B., Zhao, B.: Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine. IEEE Trans. Geosci. Remote Sens. 53(3), 1174–1185 (2014)

    Article  Google Scholar 

  40. Teke, M., Deveci, H.S., Haliloğlu, O., Gürbüz, S.Z., Sakarya, U.: A short survey of hyperspectral remote sensing applications in agriculture. In: 2013 6th International Conference on Recent Advances in Space Technologies (RAST), pp. 171–176. IEEE (2013)

    Google Scholar 

  41. Teshima, Y., Iwasaki, A.: Correction of attitude fluctuation of terra spacecraft using ASTER/SWIR imagery with parallax observation. IEEE Trans. Geosci. Remote Sens. 46(1), 222–227 (2007)

    Article  Google Scholar 

  42. Tsukiyama, T., Kondo, Y., Kakuse, K., Saba, S., Ozaki, S., Itoh, K.: Method and system for data compression and restoration (Apr 29 1986), uS Patent 4,586,027

    Google Scholar 

  43. Weiss, K., Khoshgoftaar, T.M., Wang, D.D.: A survey of transfer learning. J. Big Data 3(1), 1–40 (2016). https://doi.org/10.1186/s40537-016-0043-6

    Article  Google Scholar 

  44. Wieland, M., Li, Y., Martinis, S.: Multi-sensor cloud and cloud shadow segmentation with a convolutional neural network. Remote Sens. Environ. 230, 111203 (2019)

    Article  Google Scholar 

  45. Yang, X., et al.: Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sens. 10(1), 132 (2018)

    Article  Google Scholar 

  46. Yang, Y., Newsam, S.: Geographic image retrieval using local invariant features. IEEE Trans. Geosci. Remote Sens. 51(2), 818–832 (2012)

    Article  Google Scholar 

  47. Zhang, Z., Zhang, L., Wang, Y., Feng, P., He, R.: ShipRSImageNet: a large-scale fine-grained dataset for ship detection in high-resolution optical remote sensing images. IEEE J. Sel. Top. Appl. Earth Observations Remote Sens. 14, 8458–8472 (2021)

    Article  Google Scholar 

  48. Zhuang, F., et al.: A comprehensive survey on transfer learning. Proc. IEEE 109(1), 43–76 (2020)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ubotica Technologies, Kluyverweg 1, 2629 HS, Delft, The Netherlands

    Vincenzo Fanizza & David Rijlaarsdam

  2. Ubotica Technologies, Camino de Moledores s/n, Incubadora de Empresas, 13005, Ciudad Real, Spain

    Pablo Tomás Toledano González & José Luis Espinosa-Aranda

Authors
  1. Vincenzo Fanizza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Rijlaarsdam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pablo Tomás Toledano González
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José Luis Espinosa-Aranda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vincenzo Fanizza .

Editor information

Editors and Affiliations

  1. IBM Research AI and MIT-IBM Watson AI Lab, Haifa, Israel

    Leonid Karlinsky

  2. Technion – Israel Institute of Technology, Haifa, Israel

    Tomer Michaeli

  3. Kyoto University, Kyoto, Japan

    Ko Nishino

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fanizza, V., Rijlaarsdam, D., González, P.T.T., Espinosa-Aranda, J.L. (2023). Transfer Learning for On-Orbit Ship Segmentation. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13801. Springer, Cham. https://doi.org/10.1007/978-3-031-25056-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25056-9_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25055-2

  • Online ISBN: 978-3-031-25056-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation