Abstract
Although advanced UAVs use self-guided landing with the support of GPS devices, issues such as signal loss, support for indoor environments, accuracy issues, etc., must be considered, which contribute greatly to the overall success of autonomous flight. Therefore, this chapter proposes two different methods that allow for the recognition of a landing platform using Computer Vision techniques in order to assist autonomous landing. The first method is based on an Expert System that allows for the recognition of a patented black and white platform by performing a geometric analysis of the regions based on thresholds that allow for a degree of plane distortion. The second method, based on Cognitive Computing, can be used to solve the limitations to plane distortion inherent to the first approach, and further uses a specific landing platform with six different colors in order to combine color segmentation techniques with pattern recognition algorithms, together with an intelligent geometric analysis based on a decision-making technique. As a result, recognition can be achieved at different ranges and inclination angles between the vision system and the platform. It is not affected by distortions to the image figures due to perspective projection, even making it possible to perform the recognition with only a partial view of the platform, something that has received scant attention in the literature to date. The novelty is therefore the robustness and precision in the recognition from a wide variety of perspectives, different lighting conditions, and even problems that result in only a partial view of the platform, such as those resulting from partial focus or blind spots due to overexposure.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
García-Pulido JA, Pajares G, Dormido S, de la Cruz JM (2017) Recognition of a landing platform for unmanned aerial vehicles by using computer vision-based techniques. Expert Syst Appl 76:152–165
Cruz JM, Sánchez B, Pajares G (2012) System for guiding an unmanned vehicle towards a platform using visual analysis. Patent number 201001592, 2013
SALACOM (2013) Sistema Autónomo para la Localización y Actuación ante Contaminantes en el Mar, DPI2013–46665-C1
García-Pulido JA, Pajares G, Dormido S (2022) UAV landing platform recognition using cognitive computation combining geometric analysis and computer vision techniques. Cogn Comp
Peña JM, Torres-Sanchez J, de Castro AI, Kelly M, López-Granados F (2013) Weed mapping in early-season maize fields using object-based analysis of unmanned aerial vehicle (UAV) images. PLoS One 8(10):e77151
Jiang T, Geller J, Ni D, Collura J (2016) Unmanned aircraft system traffic management: concept of operation and system architecture. Int J Transp Sci Technol 5(3):123–135
Popescu D, Ichim L, Stoican F (2017) Unmanned aerial vehicle systems for remote estimation of flooded areas based on complex image processing. Sensors 17(3):446
Pajares G (2015) Overview and current status of remote sensing applications based on unmanned aerial vehicles (UAVs). Photogramm Eng Remote Sens 81(4):281–329
Menéndez O, Pérez M, Auat Cheein F (2019) Visual-based positioning of aerial maintenance platforms on overhead transmission lines. Appl Sci 9(1)
Amazon Prime Air Available online: https://www.amazon.com/Amazon-Prime-Air/b?ie=UTF8&node=8037720011 Accessed June 2022
Kong W, Hu T, Zhang D, Shen L, Zhang J (2017) Localization framework for real-time UAV autonomous landing: an on-ground deployed visual approach. Sensors 2017(17):1437
Nguyen PH, Kim KW, Lee YW, Park KR (2017) Remote marker-based tracking for UAV landing using visible-light camera sensor. Sensors 2017(17):1987
Anitha G, Kumar RNG (2012) Vision based autonomous landing of an unmanned aerial vehicle. Procedia Eng 2012(38):2250–2256
Xu G, Zhang Y, Ji S, Cheng Y, Tian Y (2009) Research on computer vision–based for UAV autonomous landing on a ship. Pattern Recognit Lett 2009(30):600–605
Xu G, Qi X, Zeng Q, Tian Y, Guo R, Wang B (2013) Use of land’s cooperative object to estimate UAV’s pose for autonomous landing. Chin J Aeronaut 2013(26):1498–1505
Gui Y, Guo P, Zhang H, Lei Z, Zhou X, Du J, Yu Q (2013) Airborne vision-based navigation method for UAV accuracy landing using infrared lamps. J Intell Robot Syst 72(2): 197
Yang T, Li G, Li J, Zhang Y, Zhang X, Zhang Z, Li Z (2016) A ground–based near infrared camera array system for UAV auto–landing in GPS–denied environment. Sensors 2016(16):1–20
Kong W, Zhang D, Wang X, Xian Z, Zhang J (2013) Autonomous landing of an UAV with a ground–based actuated infrared stereo vision system. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems. Tokyo, Japan, pp 2963–2970
Zhou D, Zhong Z, Zhang D, Shen L, Yan C (2015) Autonomous landing of a helicopter UAV with a ground-based multisensory fusion system. In: Seventh international conference on machine vision (ICMV 2014). International Society for Optics and Photonics, 2015, 94451R
Tang D, Hu T, Shen L, Zhang D, Kong W, Low KH (2016) Ground stereo vision-based navigation for autonomous take-off and landing of UAVs: a Chan-Vese model approach. Int J Adv Robot Syst 13(2):67
Martínez C, Campoy P, Mondragón I, Olivares–Méndez MA (2009) Trinocular ground system to control UAVs. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems. St. Louis, MO, USA, pp 3361–3367
Kong W, Zhou D, Zhang Y, Zhang D, Wang X, Zhao B, Yan C, Shen L, Zhang J (2014) A ground-based optical system for autonomous landing of a fixed wing UAV. In: Proceedingsof the IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS 2014) September 14–18. Chicago, IL, USA, pp 1–8
Al-Sharman MK, Emran BJM, Jaradat A, Najjaran H, Al-Husari R, Zweiri Y (2018) Precision landing using an adaptive fuzzy multi-sensor data fusion architecture. Appl Soft Comput 69:149–164
Al-Sharman M, Al-Jarrah MA, Abdel-Hafez M (2018) Auto takeoff and precision terminal-phase landing using an experimental optical flow model for GPS/INS enhancement. ASCE-ASME J Risk Uncertainty Eng Syst Part B Mech Eng
Asadzadeh M, Palaiahnakote S, Idris MYI, Anisi MH, Lu T, Blumenstein M, Noor NM (2019) An automatic zone detection system for safe landing of UAVs. Expert Syst Appl 122:319–333
Patterson T, McClean S, Morrow P, Parr G, Luo C (2014) Timely autonomous identification of UAV safe landing zones. Image Vis Comput 32(9):568–578
Li X (2013) A software scheme for UAV’s safe landing area discovery. AASRI Procedia 2013(4):230–235
Forster C, Faessler M, Fontana F, Werlberger M, Scaramuzza D (2015) Continuous on-board monocular-vision-based elevation mapping applied to autonomous landing of micro aerial vehicles. In: Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA). IEEE, pp 111–118
Johnson A, Montgomery J, Matthies L (2005) Vision guided landing of an autonomous helicopter in hazardous terrain. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation (ICRA)
Bosch S, Lacroix S, Caballero F (2006) Autonomous detection of safe landing areas for an UAV from monocular images. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Desaraju V, Michael N, Humenberger M, Brockers R, Weiss S, Matthies L (2015) Vision-based landing site evaluation and trajectory generation toward rooftop landing. Auton Robots 39(3):445–463
Davide F, Alessio Z, Alessandro S, Jeffrey D, Scaramuzza D (2017) Vision-based autonomous quadrotor landing on a moving platform. J Intell Robot Syst 85(2):369–384
Lee D, Ryan T, Kim HJ (2012) Autonomous landing of a vtol uav on a moving platform using image-based visual servoing. In: 2012 IEEE international conference on robotics and automation. pp 971–976
Lee H, Jung S, Shim DH (2016) Vision–based UAV landing on the moving vehicle. In: Proceedings of the International Conference on Unmanned Aircraft System. Arlington, MA, USA, pp 1–7
Feng Y, Zhang C, Baek S, Rawashdeh S, Mohammadi A (2018) Autonomous landing of a UAV on a moving platform using model predictive control. Drones 2(34)
Line V (2018) Autonomous landing of a multirotor UAVon a platform in motion. Master Thesis, Norwegian University of Science and Technology, Noruega. https://brage.bibsys.no/xmlui/handle/11250/2558185 Accessed June 2022
Rodríguez-Ramos A, Sampedro C, Bavle H, Milosevic Z, García-Vaquero A, Campoy P (2017) Towards fully autonomous landing on moving platforms for rotary unmanned aerial vehicle. In: 2017 International Conference on Unmanned Aircraft Systems (ICUAS). pp 170–178
Polvara R, Sharma S, Wan J, Manning A, Sutton R (2017) Towards autonomous landing on a moving vessel through fiducial markers. In: IEEE European conference on mobile robotics (ECMR). IEEE
Lin S, Garratt MA, Lambert AJ (2017) Monocular vision-based real-time target recognition and tracking for autonomously landing an uav in a cluttered shipboard environment. Auton Robots 41(4):881–901
Chen J, Miao X, Jiang H, Chen J, Liu X (2017) Identification of autonomous landing sign for unmanned aerial vehicle based on faster regions with convolutional neural network. In: IEEE international conference on chinese automation congress (CAC). pp 2019–2114
Polvara R, Patacchiola M, Sharma S, Wan J, Manning A, Sutton R, Cangelosi A (2017) Autonomous quadrotor landing using deep reinforcement learning. arXiv preprint arXiv:1709.03339
Wang L, Yang Q, Guo X (2016) Recognition algorithm of the apron for unmanned aerial vehicle based on image corner points. Laser J 08:71–74
Nyein EE, Tun HM, Naing ZM, Moe WK (2015) Implementation of vision-based landing target detection for VTOL UAV using raspberry Pi. Int J Sci Technol Res 4(8):184–188
Sharp CS, Shakernia O, Sastry SS (2001) A vision system for landing an unmanned aerial vehicle. In: IEEE International Conference on Robotics and Automation. pp 1720–1727
Pajares G, Cruz JM (2007) Visión por computador: imágenes digitales y aplicaciones Fundamentos de Robótica. RA-MA., Segunda Edición, Madrid
Zhao YJ, Pei HL (2013) An improved vision–based algorithm for unmanned aerial vehicles autonomous landing. Appl Mech Mater 273:560–565
Bay H, Ess A, Tuytelaars T, Van Gool L (2008) SURF: speeded up robust features. Comput Vis Image Underst 110(3):346–359
Saavedra-Ruiz M, Pinto-Vargas AM, Romero-Cano V (2018) Detection and tracking of a landing platform for aerial robotics applications. In: IEEE 2nd Colombian Conference on Robotics and Automation (CCRA). Barranquilla, pp 1–6
Saripalli S, Montgomery JF, Sukhatme GS (2002) Vision-based autonomous landing of an unmanned aerial vehicle. IEEE Int Conf Robot Autom 11–15:2799–2804
Saripalli S, Sukhatme G (2006) Landing on a moving target using an autonomous helicopter. Field and service robotics. Springer Tracts Adv Robot 2006:277–286
Cocchioni F, Mancini A, Longhi S (2014) Autonomous navigation, landing and recharge of a quadrotor using artificial vision. In: International conference on unmanned aircraft systems (ICUAS). pp 418–429
Li Y, Wang Y, Luo H, Chen Y, Jiang Y (2012) Landmark recognition for UAV autonomous landing based on vision. Appl Res Comp 07:2780–2783
Vega JA, Dormido-Canto S (2010) Máquinas de Vectores Soporte. Aprendizaje Automático (Pajares G. y de la Cruz J.M., Eds.). RA-MA, Madrid
Guili X, Yong Z, Shengyu J, Yuehua CH, Yupeng T (2009) Research on computer vision-based for UAV autonomous landing on a ship. Pattern Recogn Lett 30(6):600–605
Lange S, Sünderhauf N, Protzel P (2008) Autonomous landing for a multirotor uav using vision. In: Workshop Proceeding of SIMPAR 2008 International conference on simulation, modeling, and programming for autonomous robots. pp 482–491
S Lange N Sunderhauf P Protzel 2009 A vision based onboard approach for landing and position control of an autonomous multirotor uav in gps-denied environments. In: 2009 International conference on advanced robotics. IEEE, pp 1–6
Olson E (2011) AprilTag: a robust and flexible visual fiducial system. In: IEEE international conference on robotics and automation (ICRA)
AprilTag. Available online: https://april.eecs.umich.edu/software/apriltag.html Accessed June 2022
Ling K (2014) Precision landing of a quadrotor uav on a moving target using low–cost sensors. Master Thesis, University of Waterloo, Canada. UWSpace. https://uwspace.uwaterloo.ca/handle/10012/8803 Accessed June 2022
Kyristsis S, Antonopoulos A, Chanialakis T, Stefanakis E, Linardos C, Tripolitsiotis A, Partsinevelos P (2016) Towards autonomous modular UAV missions: the detection, geolocation and landing paradigm. Sensors 16(11):1844
Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn 47(6):2280–2292
Sani F, Karimian G (2017) Automatic navigation and landing of an indoor AR. drone quadrotor using ArUco marker and inertial sensors. In: International Conference on Computer and Drone Applications (IConDA). pp 102–107
Detection of ArUco Markers. http://docs.opencv.org/trunk/d5/dae/tutorial_aruco_detection.html Accessed June 2022
Chaves SM, Wolcott RW, Eustice RM (2015) NEEC research: toward GPS–denied landing of unmanned aerial vehicles on ships at sea. Nav Eng J 127:23–35
Araar O, Aouf N, Vitanov I (2017) Vision based autonomous landing of multirotor uav on moving platform. J Intell Robot Syst 85:369–384
Kalman RE (1960) A new approach to linear filtering and prediction problems. Transactions of the ASME. J Basic Eng 82:35–45
Rabah M, Rohan A, Talha M, Nam K, Kim S (2018) Autonomous vision-based target detection and safe landing for UAV. Int J Control Autom Syst 16:3013–3025
Lôbo-Medeiros FL, Faria-Gomes VC, Campos de Aquino MR, Geraldo D, Lopes-Honorato ME, Moreira-Dias LH (2015) A computer vision system for guidance of vtol uavs autonomous landing. In: Brazilian Conference on Intelligent Systems (BRACIS). pp 333–338
Nguyen PH, Arsalan M, Koo JH, Naqvi RA, Truong NQ, Park KR (2018) LightDenseYOLO: a fast and accurate marker tracker for autonomous UAV landing by visible light camera sensor on drone. Sensors 18(6):1703
Cesetti A, Frontoni E, Mancini A, Zingaretti P, Longhi S (2010) A vision-based guidance system for uav navigation. J Intell Robot Syst 57(1–4):233–257
Zongji C, Lei C, Rui Z, Weiqi L (2008) Unmanned aircraft landing navigation system based on vision. http://www.google.com/patents/CN101109640A Accessed June 2022. Patent CN 101109640
Roy P, Yu J, Linden DS (2008) Methods, apparatus and systems for enhanced synthetic vision and multi-sensor data fusion to improve operational capabilities of unmanned aerial vehicles. http://www.google.sc/patents/US20080215204 Accessed June 2022. Patent US20080215204
Blenkhorn KP, O'Hara SV (2009) Vision-based automated landing system for unmanned aerial vehicles. http://www.google.com/patents/US20090306840 Accessed June 2022. Patent US20090306840
Grzywna J, Da Frota B, Meuse S (2007) System and method for onboard vision processing. http://www.google.nl/patents/US20070093945 Accessed June 2022. Patent US20070093945
Shin-Je C, Jung-Ho M, Seung-Kie C, San K, Sung-Sik S (2008) Automatic recovery method of a UAV using image information to stably recover the UAV by obtaining real-time image information of a landing symbol on a recovery netting. http://patent.ipexl.com/KR/1020070058957.html Accessed June 2022. Patent 1020070058957
Avitzour D (1996) Mobile robot location determination employing error-correcting distributed landmarks. http://www.google.com/patents/US5525883 Accessed June 2022. Patent US5525883
Asaad A, Al-Salih M, Ahson SI (2009) Object detection and features extraction in video frames using direct thresholding. Multimedia, Signal Processing and Communication Technologies. IMPACT ‘09. International, pp 221–224
The MathWorks (2019) The Matlab. https://es.mathworks.com/products/matlab Accessed on June 2022
Otsu N (1979) A threshold selection method from gray-level histogram. IEEE Trans Syst Man Cybern 9:62–66
Kong TY, Rosenfeld A (1996) Topological algorithms for digital image processing. Elsevier Science, Inc.
Dougherty ER, Lotufo RA (2003) Hands-on morphological image processing. SPIE Tutorial Texts in Optical Engineering Vol. TT5. SPIE Publications
Dillencourt MB, Samet H, Tamminen M (1992) A general approach to connected-component labeling for arbitrary image representations. J Assoc Comp Mach 39(2):252–280
Sedgewick R (1998) Algorithms in C, 3rd Ed., Addison-Wesley, pp 11–20
Haralick RM, Shapiro GL (1992) Computer and robot vision, vol 1. Addison-Wesley, pp 28–48
Koschan A, Abido M (2008) Digital color image processing. John Wiley & Sons
Hu MK (1962) Visual Problem recognition by Moment Invariant. IRE Trans Inform Theory, IT-8. pp 179–187
Bohanec M (2009) Decision making: a computer-science and information-technology viewpoint. Interdiscip Descr Complex Syst 7(2):22–37
Duda RO, Hart PE, Stork DG (2006) Pattern classification, 2nd edn. New York; New Delhi: Wiley
The MathWorks (2016) Color-Based Segmentation Using the L*a*b* Color Space. https://es.mathworks.com/help/images/examples/color-based-segmentation-using-the-l-a-b-color-space.html Accessed June 2022
Campos Y, Sossa H, Pajares G (2016) Spatio-temporal analysis for obstacle detection in agricultural videos. Appl Soft Comput 45:86–97
Mercimek M, Gulez K, Mumcu TV (2005) Real object recognition using moment invariants. IEEE Trans Pattern Anal 30:765–775
Viola P, Michael J (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, vol 1:511–518
Fan Z, Lu J, Gong M, Xie H, Goodman ED (2018) Automatic tobacco plant detection in UAV images via deep neural networks. IEEE J Select Top Appl Earth Observ Remote Sens 11(3):876–887
Bicer Y, Moghadam M, Sahin C, Eroglu B, Üre NK (2019) Vision-based UAV guidance for autonomous landing with deep neural networks. In: American Institute of Aeronautics and Astronautics. AIAA Scitech 2019 Forum, AIAA SciTech Forum, (AIAA 2019–0140)
Acknowledgements
The authors would especially thank Dr. Mark Watkins for his proofreading, proof-editing services and grammatical improvement. Without his selfless assistance and dedication this work would undoubtedly not have been possible.
Finally, we would also like to thank Mestrelab, especially Santi Dominguez and Carlos Cobas, for having created a company like this that provides the flexibility and conditions necessary to make research like this possible. Special mention to Agustín Barba from Mestrelab also and Rebeca Cuiñas, for their unconditional support in good and especially in bad moments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
García-Pulido, J.A., Pajares, G. (2023). Analysis of Computer Vision-Based Techniques for the Recognition of Landing Platforms for UAVs. In: Azar, A.T., Kasim Ibraheem, I., Jaleel Humaidi, A. (eds) Mobile Robot: Motion Control and Path Planning. Studies in Computational Intelligence, vol 1090. Springer, Cham. https://doi.org/10.1007/978-3-031-26564-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-26564-8_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26563-1
Online ISBN: 978-3-031-26564-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)