[go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Trajectory prediction and visual localization of snake robot based on BiLSTM neural network

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Snake robot’s low view angle makes them vulnerable to being blocked by obstacles, which causes visual loss and track offset. In this paper, a trajectory prediction method and visual localization system for snake robots based on a Bi-directional Long Short-Term Memory (BiLSTM) neural network are proposed. First, the kinematics model of the snake robot is established by using the Denavit Hartenberg (DH) method, and the robot’s control system is described in detail. Then, the trajectory prediction model based on a BiLSTM neural network is proposed, and the network training platform is introduced. Meanwhile, the BiLSTM network’s model parameters for trajectory prediction are analyzed and optimized. To demonstrate the advantages of the proposed method, three other network prediction methods are compared. Furthermore, in order to solve the visual tracking loss of the snake robot, a novel trajectory prediction and visual localization systems based on a BiLSTM neural network and the Oriented Brief-Simultaneous Localization and Mapping (ORB-SLAM3) systems are designed. Finally, the effectiveness of the proposed method is verified by experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Availability of data and material

The datasets generated during the current study are available if the reasonable request.

Code Availability

The code generated during the current study are available if the reasonable request.

References

  1. Gong D, Wang P, Zhao S, Du L, Duan Y (2017) Bionic quadruped robot dynamic gait control strategy based on twenty degrees of freedom. IEEE/CAA J Autom Sin 5(1):382–388

    Article  MathSciNet  Google Scholar 

  2. Yan Z, Yang H, Zhang W, Lin F, Gong Q, Zhang Y (2022) Bionic fish tail design and trajectory tracking control. Ocean Eng 257:111659

    Article  Google Scholar 

  3. Zhang X, Yan J, Yang K, Zhao J, Tang S (2022) A rapid water sliding robot optimized by bionic motion trajectory. IEEE Robot Autom Lett 7(2):2463–2470

    Article  Google Scholar 

  4. Gao Y, Wei W, Wang X, Li Y, Wang D, Yu Q (2021) Feasibility, planning and control of ground-wall transition for a suctorial hexapod robot. Appl Intell 51(8):5506–5524

    Article  Google Scholar 

  5. Wang D, Wei W, Wang X, Gao Y, Li Y, Yu Q, Fan Z (2022) Formation control of multiple mecanum-wheeled mobile robots with physical constraints and uncertainties. Appl Intell 52(3):2510–2529

    Article  Google Scholar 

  6. Han S, Chon S, Kim J, Seo J, Shin DG, Park S, Kim JT, Kim J, Jin M, Cho J (2022) Snake robot gripper module for search and rescue in narrow spaces. IEEE Robot Autom Lett 7(2):1667–1673

    Article  Google Scholar 

  7. Luo M, Yan R, Wan Z, Qin Y, Santoso J, Skorina EH, Onal CD (2018) Orisnake: design, fabrication, and experimental analysis of a 3-d origami snake robot. IEEE Robot Autom Lett 3(3):1993–1999

    Article  Google Scholar 

  8. Jang H, Kim TY, Lee YC, Kim YS, Kim J, Lee HY, Choi HR (2022) A review: technological trends and development direction of pipeline robot systems. J Intell Robot Syst 105(3):1–20

    Article  Google Scholar 

  9. Jeddisaravi K, Alitappeh RJ, Pimenta A, LC, Guimarães, FG, (2016) Multi-objective approach for robot motion planning in search tasks. Appl Intell 45(2):305–321

  10. Takemori T, Tanaka M, Matsuno F (2023) Adaptive helical rolling of a snake robot to a straight pipe with irregular cross-sectional shape. IEEE Trans Robot 39(1):437–451

    Article  Google Scholar 

  11. Qin G, Wu H, Cheng Y, Pan H, Zhao W, Shi S, Song Y, Ji A (2022) Adaptive trajectory control of an under-actuated snake robot. Appl Math Model 106:756–769

    Article  MathSciNet  Google Scholar 

  12. Bae J, Kim M, Song B, Yang J, Kim D, Jin M, Yun D (2022) Review of the latest research on snake robots focusing on the structure, motion and control method. Int J Control Autom Syst 20(10):3393–3409

    Article  Google Scholar 

  13. Cao Z, Zhang D, Zhou M (2021) Direction control and adaptive path following of 3-d snake-like robot motion. IEEE Trans Cybern 52(10):10980–10987

    Article  Google Scholar 

  14. Ariizumi R, Takahashi R, Tanaka M, Asai T (2018) Head-trajectory-tracking control of a snake robot and its robustness under actuator failure. IEEE Trans Control Syst Technol 27(6):2589–2597

    Article  Google Scholar 

  15. Li D, Pan Z, Deng H, Peng T (2020) 2d underwater obstacle avoidance control algorithm based on ib-lbm and apf method for a multi-joint snake-like robot. J Intell Robot Syst 98(3):771–790

    Article  Google Scholar 

  16. Gao Y, Wei W, Wang X, Wang D, Li Y, Yu Q (2022) Trajectory tracking of multi-legged robot based on model predictive and sliding mode control. Inf Sci 606:489–511

    Article  Google Scholar 

  17. Zhang K, Chen J, Yu G, Zhang X, Li Z (2020) Visual trajectory tracking of wheeled mobile robots with uncalibrated camera extrinsic parameters. IEEE Trans Syst Man Cybern: Syst 51(11):7191–7200

    Article  Google Scholar 

  18. Qin W, Tang J, Lao S (2022) Deepfr: a trajectory prediction model based on deep feature representation. Inf Sci 604:226–248

    Article  Google Scholar 

  19. Xue H, Huynh DQ, Reynolds M (2017) Bi-prediction: pedestrian trajectory prediction based on bidirectional lstm classification. In: 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8. IEEE

  20. Lu Q, Chen J, Wang Q, Zhang D, Sun M, Su CY (2022) Practical fixed-time trajectory tracking control of constrained wheeled mobile robots with kinematic disturbances. ISA Trans 129:273–286

    Article  Google Scholar 

  21. Jin H, Zhu L, Chen Z (2016) Trajectory control of snake-like robots in natural oscillation. Asian J Control 18(5):1908–1913

    Article  MathSciNet  MATH  Google Scholar 

  22. Mukherjee J, Roy S, Kar IN, Mukherjee S (2021) Maneuvering control of planar snake robot: An adaptive robust approach with artificial time delay. Int J Robust Nonlinear Control 31(9):3982–3999

    Article  MathSciNet  Google Scholar 

  23. Li H, Zhao T, Dian S (2022) Prioritized planning algorithm for multi-robot collision avoidance based on artificial untraversable vertex. Appl Intell 52(1):429–451

    Article  Google Scholar 

  24. Ariizumi R, Tanaka M (2020) Manipulability analysis of a snake robot without lateral constraint for head position control. Asian J Control 22(6):2282–2300

    Article  MathSciNet  Google Scholar 

  25. Liu X, Lin G, Wei W (2022) Adaptive transition gait planning of snake robot based on polynomial interpolation method. In: Actuators, vol. 11, p. 222

  26. Miao MD, Gao XS, Zhao J, Zhao P (2023) Rehabilitation robot following motion control algorithm based on human behavior intention. Appl Intell 53:6324–6343

    Article  Google Scholar 

  27. Cao Z, Xiao Q, Huang R, Zhou M (2017) Robust neuro-optimal control of underactuated snake robots with experience replay. IEEE Trans Neural Netw Learn Syst 29(1):208–217

    Article  MathSciNet  Google Scholar 

  28. Shi J, Dear T, Kelly SD (2020) Deep reinforcement learning for snake robot locomotion. IFAC-Pap 53(2):9688–9695

    Google Scholar 

  29. Fukushima H, Yanagiya T, Ota Y, Katsumoto M, Matsuno F (2020) Model predictive path-following control of snake robots using an averaged model. IEEE Trans Control Syst Technol 29(6):2444–2456

    Article  Google Scholar 

  30. Bando Y, Suhara H, Tanaka M, Kamegawa T, Itoyama K, Yoshii K, Matsuno F, Okuno HG (2016) Sound-based online localization for an in-pipe snake robot. In: 2016 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pp. 207–213. IEEE

  31. Zhao X, Liu C, Dou L, Qiu J, Su Z (2019) 3d visual sensing technique based on focal stack for snake robotic applications. Results Phys 12:1520–1528

    Article  Google Scholar 

  32. Gilani C, Chen X, Pretty C, Koike C (2016) Visualisation of the motion trajectory for rolling motion of snake robots using virtual chassis and simplified kinematics motion model. IFAC-Pap 49(32):228–233

    Google Scholar 

  33. Yue H, Chen W (2014) Comments on automatic visual bag-of-words for online robot navigation and mapping. IEEE Trans Robot 31(1):223–224

    Article  Google Scholar 

  34. Han J, Kim J, Shim DH (2018) Precise localization and mapping in indoor parking structures via parameterized slam. IEEE Trans Intell Transp Syst 20(12):4415–4426

    Article  Google Scholar 

  35. Qin T, Li P, Shen S (2018) Vins-mono: a robust and versatile monocular visual-inertial state estimator. IEEE Trans Robot 34(4):1004–1020

    Article  Google Scholar 

  36. Li X, Ren C, Ma S (2022) Optimal path following control with efficient computation for snake robots subject to multiple constraints and unknown frictions. IEEE Robot Autom Lett 7(4):9151–9158

    Article  Google Scholar 

  37. Bloesch M, Burri M, Omari S, Hutter M, Siegwart R (2017) Iterated extended kalman filter based visual-inertial odometry using direct photometric feedback. Int J Robot Res 36(10):1053–1072

    Article  Google Scholar 

  38. Yin H, Li S, Tao Y, Guo J, Huang B (2023) Dynam-slam: an accurate, robust stereo visual-inertial slam method in dynamic environments. IEEE Trans Robot 39(1):289–308

    Article  Google Scholar 

  39. Jiang J, Yuan J, Zhang X, Zhang X (2020) Dvio: An optimization-based tightly coupled direct visual-inertial odometry. IEEE Trans Ind Electron 68(11):11212–11222

    Article  Google Scholar 

  40. Campos C, Elvira R, Rodríguez JJG, Montiel JM, Tardós JD (2021) Orb-slam3: An accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Trans Robot 37(6):1874–1890

  41. Mur-Artal R, Montiel JMM, Tardos JD (2015) Orb-slam: a versatile and accurate monocular slam system. IEEE Trans Robot 31(5):1147–1163

    Article  Google Scholar 

  42. Mur-Artal R, Tardós JD (2017) Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Trans Robot 33(5):1255–1262

    Article  Google Scholar 

  43. Belkin I, Abramenko A, Yudin D (2021) Real-time lidar-based localization of mobile ground robot. Procedia Comput Sci 186:440–448

    Article  Google Scholar 

  44. Eder M, Reip M, Steinbauer G (2022) Creating a robot localization monitor using particle filter and machine learning approaches. Appl Intell 52(6):6955–6969

    Article  Google Scholar 

  45. Li D, Zeng L, Xiu Y, Pan Z, Zhang D, Deng H (2023) Sideslip elimination and coefficient approximation-based trajectory tracking control for snake robots. IEEE Trans Ind Inform 19(8):8754–8764

  46. Xiu Y, Deng H, Li D, Zhang M, Law R, Huang Y, Wu EQ, Xu X (2023) Finite-time sideslip differentiator-based los guidance for robust path following of snake robots. IEEE/CAA J Autom Sin 10(1):239–253

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (grant numbers 61573148, 61603358), and the Science and Technology Planning Project of Guangdong Province, China (grant number 2015B010919007).

Funding

The National Natural Science Foundation of China (grant numbers 61573148, 61603358), and the Science and Technology Planning Project of Guangdong Province, China (grant number 2015B010919007).

Author information

Authors and Affiliations

  1. School of Automation Science and Engineering, South China University of Technology, 510641, Guangzhou, Guangdong, China

    Xiongding Liu, Wu Wei, Yong Gao, Zhendong Xiao & Guangjie Lin

  2. The Key Laboratory of Autonomous Systems and Networked Control, Ministry of Education, Unmanned Aerial Vehicle Systems Engineering Technology Research Center of Guangdong, South China University of Technology, 510641, Guangzhou, Guangdong, China

    Wu Wei

  3. School of Automation, Jiangsu University of Science and Technology, 212003, Zhenjiang, Jiangsu, China

    Yanjie Li

Authors
  1. Xiongding Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wu Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanjie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yong Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhendong Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guangjie Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to this work. Conceptualization, X.L. and G.L.; methodology, X.L. W.W. and Y.L.; software, X.L. and Z.X.; validation, Y.G. and W.W.; writing-original draft, X.L. and G.L.; writing-review and editing, Y.L., Y.G. and Z.X. All authors have read and agreed to the published version of the manuscript.

Corresponding authors

Correspondence to Xiongding Liu or Wu Wei.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethics approval

All authors agree the Ethics approval.

Consent to participate

All authors consent to participate.

Consent for publication

All authors consent to publication.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Wei, W., Li, Y. et al. Trajectory prediction and visual localization of snake robot based on BiLSTM neural network. Appl Intell 53, 27790–27807 (2023). https://doi.org/10.1007/s10489-023-04897-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04897-7

Keywords

Search

Navigation