[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Explainable AI for Suicide Risk Assessment Using Eye Activities and Head Gestures

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13336))

Included in the following conference series:

  • 3180 Accesses

Abstract

The prevalence of suicide has been on the rise since the 20th century, causing severe emotional damage to individuals, families, and communities alike. Despite the severity of this suicide epidemic, there is so far no reliable and systematic way to assess suicide intent of a given individual. Through efforts to automate and systematize diagnosis of mental illnesses over the past few years, verbal and acoustic behaviors have received increasing attention as biomarkers, but little has been done to study eyelids, gaze, and head pose in evaluating suicide risk. This study explores statistical analysis, feature selection, and machine learning classification as means of suicide risk evaluation and nonverbal behavioral interpretation. Applying these methods to the eye and head signals extracted from our unique dataset, this study finds that high-risk suicidal individuals experience psycho-motor retardation and symptoms of anxiety and depression, characterized by eye contact avoidance, slower blinks and a downward eye gaze. By comparing results from different methods of classification, we determined that these features are highly capable of automatically classifying different levels of suicide risk consistently and with high accuracy, above 98%. Our conclusion corroborates psychological studies, and shows great potential of a systematic approach in suicide risk evaluation that is adoptable by both healthcare providers and naïve observers.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www6.nhk.or.jp/heart-net/mukiau.

  2. 2.

    Detailed MANOVA results are presented here: https://bit.ly/32h6CRa.

  3. 3.

    Full classification results are presented here: https://bit.ly/3tnqcY1.

References

  1. Abaei, N., Osman, H.A.: A hybrid model for bipolar disorder classification from visual information. In: 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2020, pp. 4107–4111 (2020). https://doi.org/10.1109/ICASSP40776.2020.9054648

  2. Alghowinem, S., Goecke, R., Wagner, M., Parker, G., Breakspear, M.: Eye movement analysis for depression detection. In: 2013 IEEE International Conference on Image Processing, pp. 4220–4224 (2013). https://doi.org/10.1109/ICIP.2013.6738869

  3. Alghowinem, S., Goecke, R., Wagner, M., Parkerx, G., Breakspear, M.: Head pose and movement analysis as an indicator of depression. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, pp. 283–288. IEEE (2013)

    Google Scholar 

  4. Alghowinem, S.M., Gedeon, T., Goecke, R., Cohn, J., Parker, G.: Interpretation of depression detection models via feature selection methods. IEEE Trans. Affect. Comput. 1(1), 1 (2020). https://doi.org/10.1109/TAFFC.2020.3035535

  5. Ambady, N., Rosenthal, R.: Thin slices of expressive behavior as predictors of interpersonal consequences: a meta-analysis. Psychol. Bull. 111(2), 256 (1992)

    Article  Google Scholar 

  6. Bernert, R.A., Hilberg, A.M., Melia, R., Kim, J.P., Shah, N.H., Abnousi, F.: Artificial intelligence and suicide prevention: a systematic review of machine learning investigations. Int. J. Environ. Res. Public Health 17(16), 5929 (2020). https://doi.org/10.3390/ijerph17165929

    Article  Google Scholar 

  7. Cech, J., Soukupova, T.: Real-time eye blink detection using facial landmarks, pp. 1–8 (2016)

    Google Scholar 

  8. Chicco, D., Jurman, G.: The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21(1) (2020). https://doi.org/10.1186/s12864-019-6413-7

  9. Chowdhury, G.: TREC: Experiment and Evaluation in Information Retrieval (2007). Online information review

    Google Scholar 

  10. Cohn, J.F., et al.: Detecting depression from facial actions and vocal prosody. In: 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. pp. 1–7 (2009). https://doi.org/10.1109/ACII.2009.5349358

  11. Eigbe, N., Baltrusaitis, T., Morency, L., Pestian, J.: Toward visual behavior markers of suicidal ideation. In: 2018 13th IEEE International Conference on Automatic Face Gesture Recognition, FG 2018, pp. 530–534 (2018). https://doi.org/10.1109/FG.2018.00085

  12. Eisenberg, N., Spinrad, T.L.: Emotion-related regulation: sharpening the definition. Child Dev. 75(2), 334–339 (2004)

    Article  Google Scholar 

  13. Fonseka, T.M., Bhat, V., Kennedy, S.H.: The utility of artificial intelligence in suicide risk prediction and the management of suicidal behaviors. Aust. NZ J. Psychiatry 53(10), 954–964 (2019)

    Article  Google Scholar 

  14. Fossi, L., Faravelli, C., Paoli, M.: The ethological approach to the assessment of depressive disorders. J. Nerv. Ment. Dis. 172(6), 332–341 (1984). https://doi.org/10.1097/00005053-198406000-00004

  15. Gómez, J.M.: Language technologies for suicide prevention in social media. In: Proceedings of the Workshop on Natural Language Processing in the 5th Information Systems Research Working Days (JISIC), pp. 21–29 (2014)

    Google Scholar 

  16. Gorodkin, J.: Comparing two k-category assignments by a k-category correlation coefficient. Comput. Biol. Chem. 28(5–6), 367–374 (2004). https://doi.org/10.1016/j.compbiolchem.2004.09.006

    Article  MATH  Google Scholar 

  17. Grandin, T.: How people with autism think. In: Schopler, E., Mesibov, G.B. (eds.) Learning and Cognition in Autism, pp. 137–156. Springer, Boston, MA (1995). https://doi.org/10.1007/978-1-4899-1286-2_8

    Chapter  Google Scholar 

  18. Guan, K., Fox, K.R., Prinstein, M.J.: Nonsuicidal self-injury as a time-invariant predictor of adolescent suicide ideation and attempts in a diverse community sample. J. Consult. Clin. Psychol. 80(5), 842 (2012)

    Article  Google Scholar 

  19. Horley, K., Williams, L.M., Gonsalvez, C., Gordon, E.: Social phobics do not see eye to eye: a visual scanpath study of emotional expression processing. J. Anxiety Disord. 17(1), 33–44 (2003). https://doi.org/10.1016/S0887-6185(02)00180-9. https://www.sciencedirect.com/science/article/pii/S0887618502001809

  20. Huang, Y.P., Goh, T., Liew, C.L.: Hunting suicide notes in web 2.0-preliminary findings. In: 9th IEEE International Symposium on Multimedia Workshops, ISMW 2007, pp. 517–521. IEEE (2007)

    Google Scholar 

  21. Jamison, K.R., et al.: Suicide and bipolar disorder. J. Clin. Psychiatry 61, 47–51 (2000)

    Article  Google Scholar 

  22. Jun, Y.Y., Mareschal, I., Clifford, C.W., Dadds, M.R.: Cone of direct gaze as a marker of social anxiety in males. Psychiatry Res. 210(1), 193–198 (2013). https://doi.org/10.1016/j.psychres.2013.05.020. https://www.sciencedirect.com/science/article/pii/S0165178113002795

  23. Laksana, E., Baltrušaitis, T., Morency, L., Pestian, J.P.: Investigating facial behavior indicators of suicidal ideation. In: 2017 12th IEEE International Conference on Automatic Face Gesture Recognition, FG 2017, pp. 770–777 (2017). https://doi.org/10.1109/FG.2017.96

  24. Lecrubier, Y., et al.: The mini international neuropsychiatric interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI. Eur. Psychiatry 12(5), 224–231 (1997)

    Google Scholar 

  25. Low, D.M., Bentley, K.H., Ghosh, S.S.: Automated assessment of psychiatric disorders using speech: a systematic review. Laryngoscope Invest. Otolaryngol. 5(1), 96–116 (2020). https://doi.org/10.1002/lio2.354

    Article  Google Scholar 

  26. Matsubayashi, T., Sekijima, K., Ueda, M.: Government spending, recession, and suicide: evidence from Japan. BMC Pub. Health 20(1), 243 (2020). https://doi.org/10.1186/s12889-020-8264-1

  27. Moukheiber, A., et al.: Gaze avoidance in social phobia: objective measure and correlates. Behav. Res. Ther. 48(2), 147–151 (2010). https://doi.org/10.1016/j.brat.2009.09.012. https://www.sciencedirect.com/science/article/pii/S0005796709002265

  28. NIH: Suicide prevention (2021). https://www.nimh.nih.gov/health/topics/suicide-prevention/index.shtml. Accessed 03 Apr 2021

  29. Perez, J.E., Riggio, R.E.: Nonverbal social skills and psychopathology. In: Philippot, P., Feldman, R.S., Coats, E.J. (eds.) Nonverbal Behavior in Clinical Settings. Series in Affective Science, pp. 17–44. Oxford University Press (2003)

    Google Scholar 

  30. Plans, L.: Association between completed suicide and bipolar disorder: a systematic review of the literature. J. Affect. Disord. 242, 111–122 (2019)

    Article  Google Scholar 

  31. Runeson, B., Odeberg, J., Pettersson, A., Edbom, T., Jildevik Adamsson, I., Waern, M.: Instruments for the assessment of suicide risk: a systematic review evaluating the certainty of the evidence. PLoS ONE 12(7), e0180292 (2017)

    Article  Google Scholar 

  32. Schelde, J.T.M.: Major depression: behavioral markers of depression and recovery. J. Nerv. Ment. Dis. 186(3), 133–140 (1998)

    Article  Google Scholar 

  33. Shah, A.P., Vaibhav, V., Sharma, V., Al Ismail, M., Girard, J., Morency, L.P.: Multimodal behavioral markers exploring suicidal intent in social media videos. In: 2019 International Conference on Multimodal Interaction, pp. 409–413 (2019)

    Google Scholar 

  34. Sugano, Y., Matsushita, Y., Sato, Y.: Learning-by-synthesis for appearance-based 3D gaze estimation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2014, pp. 1821–1828 (2014). https://doi.org/10.1109/CVPR.2014.235

  35. Waern, M., Kaiser, N., Renberg, E.S.: Psychiatrists’ experiences of suicide assessment. BMC Psychiatry 16(1), 440 (2016)

    Article  Google Scholar 

  36. Waxer, P.H.: Nonverbal cues for anxiety: an examination of emotional leakage. J. Abnorm. Psychol. 86(3), 306 (1977)

    Article  Google Scholar 

  37. WHO: Suicide. https://www.who.int/news-room/fact-sheets/detail/suicide

  38. Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., Li, S.Z.: S\(^3\)FD: single shot scale-invariant face detector. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 192–201 (2017)

    Google Scholar 

  39. Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: MPIIGaze: real-world dataset and deep appearance-based gaze estimation. IEEE Trans. Pattern Anal. Mach. Intell. 41(1), 162–175 (2019). https://doi.org/10.1109/TPAMI.2017.2778103

    Article  Google Scholar 

  40. Zhang, X., Sugano, Y., Fritz, M., Bulling, A.: Appearance-based gaze estimation in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015, pp. 4511–4520 (2015)

    Google Scholar 

Download references

Acknowledgement

We thank and acknowledge the effort made by NHK, Nippon Hoso Kyokai (Japan Broadcasting Corporation) for conducting, recording and providing the interview dataset used in this work.

Author information

Authors and Affiliations

  1. Massachusetts Institute of Technology, Cambridge, MA, 02139, USA

    Siyu Liu & Catherine Lu

  2. MIT Media Lab, Personal Robotics Group, Cambridge, MA, 02139, USA

    Sharifa Alghowinem, Cynthia Breazeal & Hae Won Park

  3. Japan Broadcasting Corporation, Tokyo, Japan

    Lea Gotoh

Authors
  1. Siyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Catherine Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sharifa Alghowinem
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lea Gotoh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Cynthia Breazeal
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hae Won Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Siyu Liu .

Editor information

Editors and Affiliations

  1. Siemens (United States), Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, S., Lu, C., Alghowinem, S., Gotoh, L., Breazeal, C., Park, H.W. (2022). Explainable AI for Suicide Risk Assessment Using Eye Activities and Head Gestures. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2022. Lecture Notes in Computer Science(), vol 13336. Springer, Cham. https://doi.org/10.1007/978-3-031-05643-7_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05643-7_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05642-0

  • Online ISBN: 978-3-031-05643-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation