research-article

Open access

On Specifying for Trustworthiness

Authors:

Dhaminda B. Abeywickrama,

Amel Bennaceur,

Yiannis Demiris,

Anastasia Kordoni,

Mohammad Reza Mousavi,

Bashar Nuseibeh,

Subramanian Ramamoorthy,

Jan Oliver Ringert,

Kerstin EderAuthors Info & Claims

Communications of the ACM, Volume 67, Issue 1

Pages 98 - 109

https://doi.org/10.1145/3624699

Published: 21 December 2023 Publication History

All formats PDF

Abstract

As autonomous systems increasingly become part of our lives, it is crucial to foster trust between humans and these systems, to ensure positive outcomes and mitigate harmful ones.

References

[1]

Amershi, S. et al. Software engineering for machine learning: A case study. In Proc. of the 41^st Intern. Conf. on Software Engineering: Software Engineering in Practice. IEEE/ACM (2019), 291--300; 10.1109.

Digital Library

[2]

Criado, N. Resource-bounded norm monitoring in multi-agent systems. J. Artificial Intelligence Res. 62, 1 (May 2018), 153--192

Digital Library

[3]

D'Amour, A. et al. Underspecification presents challenges for credibility in modern machine learning. CoRR (2020); https://arxiv.org/abs/2011.03395

[4]

DeGrave, A.J., Janizek, J.D., and Lee, S. AI for radiographic COVID-19 detection selects shortcuts over signal. Nature Machine Intelligence (2021), 1--10.

[5]

Demiris, Y. Prediction of intent in robotics and multi-agent systems. Cognitive Processing 8, 3 (2007), 151--158

[6]

Department for Transport. The highway code--Using the road (159 to 203), 2022; https://bit.ly/49nAFbh.

[7]

Drury, J. The role of social identity processes in mass emergency behaviour: An integrative review. European Rev. of Social Psychology 29, 1 (2018), 38--81.

[8]

Escalante, M.A.L. et al. Is IT ethical? Board game: Playing with speculative ethics of IT innovation in disaster and risk management. In Proc. of the IX Latin American Conf. on Human Computer Interaction. ACM (2019), 8; Article 9

Digital Library

[9]

Famelis, M., Salay, R., and Chechik, M. Partial models: Towards modeling and reasoning with uncertainty. In 34^th Intern. Conf. on Software Engineering. IEEE Computer Society (2012), 573--583

[10]

Fischbein, D. et al. Weak alphabet merging of partial behavior models. ACM Trans. Software Engineering Methodology 21, 2 (2012), 9:1--9:47

Digital Library

[11]

Fisher, M. et al. Towards a framework for certification of reliable autonomous systems. Autonomous Agents and Multi-Agent Systems 35, 8 (2021), 65

Digital Library

[12]

Fletcher, L., Clarke, R., Richardson, T., and Hansen, M. Reinforcement learning for a perched landing in the presence of wind. In AIAA SciTech 2021 Forum. American Institution of Aeronautics and Astronautics (2021)

[13]

International Organization for Standardization. ISO/IEC/IEEE 24765:2017 systems and software engineering---Vocabulary; https://www.iso.org/standard/71952.html

[14]

Freeman, K. et al. Use of artificial intelligence for image analysis in breast cancer screening programmes: Systematic review of test accuracy. BMJ 374 (2021)

[15]

Gao, Y., Chang, H.J., and Demiris, Y. User modelling using multimodal information for personalised dressing assistance. IEEE Access 8 (2020), 45700--45714

[16]

Georgiou, T. and Demiris, Y. Adaptive user modelling in car racing games using behavioural and physiological data. User Modelling and User-Adapted Interaction 27 (2017), 267--311

Digital Library

[17]

Grigore, E.C. et al. Joint action understanding improves robot-to-human object handover. In Proc. of the 2013 IEEE/RSJ Intern. Conf. on Intelligent Robots and Systems (2013), 4622--4629

[18]

Hamon, R., Junklewitz, H., and Sanchez, I. Robustness and explainability of artificial intelligence. Technical Report. Publications Office of the European Union (2020)

[19]

Harper, C. et al. Safety validation of autonomous vehicles using assertion-based oracles. arXiv (2021)

[20]

Harrison, M.D., Masci, P., Campos, J.C., and Curzon, P. Verification of user interface software: The example of use-related safety requirements and programmable medical devices. IEEE Trans. Hum. Mach. Systems 47, 6 (2017), 834--846

[21]

Heimlich, J.E. and Ardoin, N.M. Understanding behavior to understand behavior change: A literature review. Environmental Education Research 14, 3 (2008), 215--237.

[22]

Huynh, T.D. et al. Addressing regulatory requirements on explanations for automated decisions with provenance: A case study. Digital Government: Research and Practice 2, 2 (Jan. 2021)

Digital Library

[23]

Ishikawa, F. and Matsuno, Y. Evidence-driven requirements engineering for uncertainty of machine learning-based Systems. In Proceedings of the 28^th IEEE Intern. Requirements Engineering Conf. (2020), 346--351.

[24]

Jackson, D. Alloy: A language and tool for exploring software designs. Commun. ACM 62, 9 (2019), 66--76

Digital Library

[25]

James, K. The organizational science of disaster/terrorism prevention and response: Theory-building toward the future of the field. J. of Organizational Behavior 32, 7 (2011), 1013--1032.

[26]

Jia, Y., McDermid, J., Lawton, T., and Habli, I. The role of explainability in assuring safety of machine learning in healthcare. arXiv (2021)

[27]

Koopman, P. et al. Certification of Highly Automated Vehicles for Use on UK Roads: Creating an IndustryWide Framework for Safety. Five AI Ltd. (2019).

[28]

Kramer, J. Is abstraction the key to computing? Communications of the ACM 50, 4 (2007), 36--42

Digital Library

[29]

Kress-Gazit, H. et al. Formalizing and guaranteeing human-robot interaction. Communications of the ACM 64, 9 (Aug. 2021), 78--84

Digital Library

[30]

Lee, Y., Lee, J., and Lee, Z. The effect of self identity and social identity on technology acceptance. In Proc. of the Intern. Conf. on Information Systems 59, (2001); https://aisel.aisnet.org/icis2001/59/.

[31]

Maoz, S. and Ringert, J.O. A framework for relating syntactic and semantic model differences. Software and Systems Modeling 17, 3 (2018), 753--777

Digital Library

[32]

Maoz, S. and Ringert, J.O. On the software engineering challenges of applying reactive synthesis to robotics. In Proc. of the 1^st Intern. Workshop on Robotics Software Eng., ACM (2018), 17--22

Digital Library

[33]

Mens, T. Introduction and roadmap: History and challenges of software evolution. In Software Evolution, T. Mens and S. Demeyer (Eds.), Springer (2008), 1--11

[34]

Murukannaiah, P.K., Ajmeri, N., Jonker, C.M., and Singh, M.P. New foundations of ethical multiagent systems. In Proc. of the 19^th Intern. Conf. on Autonomous Agents and MultiAgent Systems, Intern. Found. for Autonomous Agents and Multiagent Systems (2020), 1706--1710.

[35]

Naiseh, M., Bentley, C.M., and Ramchurn, S. Trustworthy autonomous systems (TAS): Engaging TAS experts in curriculum design. In Proc. of the 2022 IEEE Global Eng. Education Conf.

[36]

Narayanan, A. 21 fairness definitions and their politics. In Proceedings of the Conf. on Fairness, Accountability, and Transparency, ACM (2021).

[37]

Şahin, E. Swarm robotics: From sources of inspiration to domains of application. In Swarm Robotics, E. Şahin and W. Spears (Eds.). Springer Berlin Heidelberg, (2005), 10--20.

Digital Library

[38]

Schneiders, E. et al. Non-dyadic interaction: A literature review of 15 years of human-robot interaction conference publications. J. Human-Robot Interactions 11, 2, Article 13, (Feb. 2022), 32

Digital Library

[39]

Sebanz, N., Bekkering, H., and Knoblich, G. Joint action: Bodies and minds moving together. Trends in Cognitive Sciences 10, 2 (2006), 70--76.

[40]

Soh, H. and Demiris, Y. Learning assistance by demonstration: Smart mobility with shared control and paired haptic controllers. J. Human-Robot Interactions 4, 3 (Dec. 2015), 76--100

Digital Library

[41]

Spears, R. Social influence and group identity. Annual Rev. of Psychology 72 (2021), 367--390.

[42]

Topol, E.J. High-performance medicine: The convergence of human and artificial intelligence. Nature Medicine 25, 1 (2019), 44--56.

[43]

Tsakalakis, N. et al. The dual function of explanations: Why it is useful to compute explanations. Computer Law and Security Rev. 41 (Mar. 18, 2021)

[44]

Wei, O., Gurfinkel, A., and Chechik, M. On the consistency, expressiveness, and precision of partial modeling formalisms. Inf. Comput. 209, 1 (2011), 20--47

Digital Library

[45]

Winfield, A.F.T. and Nembrini, J. Safety in numbers: Fault-tolerance in robot swarms. Intern. J. on Modelling Identification and Control 1, 1 (2006), 30--37

Cited By

Gavidia-Calderon CKordoni ABennaceur ALevine MNuseibeh B(2024)The IDEA of Us: An Identity-Aware Architecture for Autonomous SystemsACM Transactions on Software Engineering and Methodology10.1145/365443933:6(1-38)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3654439
Abeywickrama DLe NChance GWinter PManzini APartridge AIves JDowner JDeacon GRossiter JEder KWindsor S(2024)Soft Gripping: Specifying for Trustworthiness2024 IEEE 7th International Conference on Soft Robotics (RoboSoft)10.1109/RoboSoft60065.2024.10522042(565-571)Online publication date: 14-Apr-2024
https://doi.org/10.1109/RoboSoft60065.2024.10522042
Attaoui ADjemal AAtitallah BJaoui AKanoun O(2024)Biomimetic Robotic Hand with Highly Sensitive Integrated Nanocomposite Force Sensors for Adaptive Grasping2024 IEEE International Symposium on Robotic and Sensors Environments (ROSE)10.1109/ROSE62198.2024.10591203(01-06)Online publication date: 20-Jun-2024
https://doi.org/10.1109/ROSE62198.2024.10591203

Index Terms

On Specifying for Trustworthiness

Recommendations

Perceived Trustworthiness Within the Organization: The Moderating Impact of Communication Frequency on Trustor and Trustee Effects

This paper investigates the antecedents of intraorganizational trust and, more specifically, how the frequency of communication between trustor and trustee moderates the impact of these factors on perceived trustworthiness. Data on 157 dyadic ...
Trust and trustworthiness: a framework for successful design of telemedicine
Toward Helping Users in Assessing the Trustworthiness of User-Generated Reviews
BCS-HCI '14: Proceedings of the 28th International BCS Human Computer Interaction Conference on HCI 2014 - Sand, Sea and Sky - Holiday HCI

With the growing number of systems that provide user-generated reviews come new forms of trust relationships. User trust in vendors can be mediated by trust in either reviews or reviewers, or possibly both. These new forms of trust relationships might ...

Comments

Information & Contributors

Information

Published In

cover image Communications of the ACM

Communications of the ACM Volume 67, Issue 1

January 2024

122 pages

EISSN:1557-7317

DOI:10.1145/3638509

Editor:
James Larus
Association for Computing Machinery, New York, NY

Issue’s Table of Contents

Copyright © 2023 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 December 2023

Published in CACM Volume 67, Issue 1

Check for updates

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
7,303
Total Downloads

Downloads (Last 12 months)7,303
Downloads (Last 6 weeks)125

Reflects downloads up to 04 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gavidia-Calderon CKordoni ABennaceur ALevine MNuseibeh B(2024)The IDEA of Us: An Identity-Aware Architecture for Autonomous SystemsACM Transactions on Software Engineering and Methodology10.1145/365443933:6(1-38)Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3654439
Abeywickrama DLe NChance GWinter PManzini APartridge AIves JDowner JDeacon GRossiter JEder KWindsor S(2024)Soft Gripping: Specifying for Trustworthiness2024 IEEE 7th International Conference on Soft Robotics (RoboSoft)10.1109/RoboSoft60065.2024.10522042(565-571)Online publication date: 14-Apr-2024
https://doi.org/10.1109/RoboSoft60065.2024.10522042
Attaoui ADjemal AAtitallah BJaoui AKanoun O(2024)Biomimetic Robotic Hand with Highly Sensitive Integrated Nanocomposite Force Sensors for Adaptive Grasping2024 IEEE International Symposium on Robotic and Sensors Environments (ROSE)10.1109/ROSE62198.2024.10591203(01-06)Online publication date: 20-Jun-2024
https://doi.org/10.1109/ROSE62198.2024.10591203

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Digital Edition

View this article in digital edition.

Digital Edition

Magazine Site

View this article on the magazine site (external)

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents