Unveiling the Use of Networked Ontologies to Develop a Supporting Tool for UX Evaluation in an Immersive Context
DOI:
https://doi.org/10.5753/jbcs.2025.4351Keywords:
User Experience, UX Evaluation, Immersive Experience, Ontology, Ontology NetworkAbstract
Immersive technologies have emerged as a new type of interactive system that aims to provide users with immersive experiences. They have been adopted in various fields and are gradually becoming part of our lives. UX is a key quality attribute to evaluate or model such experiences. However, when it comes to immersive experiences, evaluating UX is particularly challenging because the user should not be interrupted to provide feedback. Aiming at giving a step to address this issue, we have explored using ontologies from an ontology network to support evaluating immersive experiences. In this work, we adopted the Human-Computer Interaction Ontology Network (HCI-ON) and used an extract containing concepts from some of its networked ontologies to develop the User eXperience evaluation based on Ontology Network (UXON), an ontology-based tool that supports UX experts evaluating immersive experiences based on data recorded in interaction logs. HCI-ON is a framework for organizing knowledge of the HCI domain, offering a general understanding of the field, regardless of specific solutions. UXON was used to evaluate the UX of Compomus, an immersive application that supports collaborative music composition. UXON extracts data from the application interaction logs, calculates UX metrics, and provides consolidated data and information in graphs and tables. We conducted a study and collected feedback from the UXON developer and three UX experts who used the tool. Results showed that using networked ontologies to develop a tool to support UX evaluation is feasible and valuable. In summary, the ontologies helped at the conceptual level by offering a basis to define the system's structural model and at the implementation level by assigning semantics to data to make inferences about UX. Based on the UX experts' perceptions, UXON was considered a promising system, beneficial, helpful, and easy to use. The conceptualization used to develop UXON was evaluated by HCI experts and it was considered adequate and understandable, having the potential to be used by other people to solve HCI evaluation problems.
Downloads
References
Albert, W. and Tullis, T. (2013). Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Interactive Technologies. Morgan Kaufmann, Boston, USA, 2th edition. DOI: 10.1016/C2011-0-00016-9.
Amazonas, M., Castro, T., De Freitas, R., and Gadelha, B. (2019). Composing through Interaction: a framework for collaborative music composition based on human interaction on public spaces. In Proceedings of Brazilian Symposium on Musical Computing (SBCM 2019), pages 42-49, Porto Alegre, RS, Brasil. Sociedade Brasileira de Computaç ao - SBC. DOI: 10.5753/sbcm.2019.10421.
Arango-López, J., Vela, F. L. G., Collazos, C. A., and Moreira, F. (2018). Modeling and defining the pervasive games and its components from a perspective of the player experience. In Rocha, Á., Adeli, H., Reis, L. P., and Costanzo, S., editors, Proceedings of the Trends and Advances in Information Systems and Technologies. WorldCIST'18 2018. Advances in Intelligent Systems and Computing, vol 746, pages 625-635, Cham. Springer International Publishing. DOI: 10.1007/978-3-319-77712-2_58.
Bakaev, M., Heil, S., Perminov, N., and Gaedke, M. (2019). Integration platform for metric-based analysis of web user interfaces. In Proceedings of the International Conference on Web Engineering. ICWE 2019. Lecture Notes in Computer Science, vol 11496, pages 525-529, Cham. Springer International Publishing. DOI: 10.1007/978-3-030-19274-7_39.
Barbosa, S. D. J., da Silva, B. S., Silveira, M. S., Gasparini, I., Darin, T., and Barbosa, G. D. J. (2021). Interação Humano-Computador e Experiência do Usuário. Autopublicação. Book.
Barcellos, M., Falbo, R. d. A., and Frauches, V. G. V. (2014). Towards a measurement ontology pattern language. Proceedings of the 1st Joint Workshop ONTO.COM / ODISE on Ontologies in Conceptual Modeling and Information Systems Engineering, 1301:14. available at: [link].
Barcellos, M. P., Falbo, R. d. A., and Rocha, A. R. (2010). Establishing a well-founded conceptualization about software measurement in high maturity levels. In Proceedings of 2010 Seventh International Conference on the Quality of Information and Communications Technology, pages 467-472, Porto, Portugal. IEEE. DOI: 10.1109/QUATIC.2010.84.
Basili, V. R., Caldiera, G., and Rombach, H. D. (1994). The Goal Question Metric Approach. available at: [link].
Calero, C., Ruiz, F., and Piattini, M. (2006). Ontologies for Software Engineering and Software Technology. Springer-Verlag, Berlin, Heidelberg. Book.
Carraretto, R. and Almeida, J. P. A. (2012). Separating Ontological and Informational Concerns: Towards a Two-Level Model-Driven Approach. In Proceedings of 2012 IEEE 16th International Enterprise Distributed Object Computing Conference Workshops, pages 29-37, Beijing, China. IEEE. DOI: 10.1109/EDOCW.2012.14.
Carroll, J. M. (2014). Human Computer Interaction (HCI). In Soegaard, M. and Dam, R. F., editors, The Encyclopedia of Human-Computer Interaction, chapter 2, pages 21-61. The Interaction Design Foundation, Aarhus, Denmark, 2nd edition. Book.
Castro, M. V. H. B. C. (2021). An Ontology to support Knowledge Management Solutions for Human-Computer Interaction Design. DOI: 10.1145/3571473.3571502.
Costa, S. and Barcellos, M. (2024a). HCI-ON's Conceptualization behind UXON - Suplemmentary Material. DOI: 10.6084/m9.figshare.27080482.v1.
Costa, S. D. (2022). An Ontology Network to support Knowledge Representation and Semantic Interoperability in the HCI Domain. page 205. Available at: [link].
Costa, S. D. and Barcellos, M. P. (2024b). UXON Evaluation (Users' Perspective) - Suplemmentary Material. DOI: 10.6084/m9.figshare.25452535.v2.
Costa, S. D., Barcellos, M. P., and Falbo, R. d. A. (2021). Ontologies in human–computer interaction: A systematic literature review. Applied Ontology, 16(4):1-32. DOI: 10.3233/AO-210255.
Costa, S. D., Barcellos, M. P., Falbo, R. d. A., and Castro, M. V. H. B. (2020). Towards an Ontology Network on Human-Computer Interaction. In Dobbie, G., Frank, U., Kappel, G., Liddle, S. W., and Mayr, H. C., editors, Proceedings of the 39th International Conference on Conceptual Modeling, pages 331-341, Cham. Springer International Publishing. DOI: 10.1007/978-3-030-62522-1_24.
Costa, S. D., Barcellos, M. P., Falbo, R. d. A., Conte, T., and de Oliveira, K. M. (2022). A core ontology on the Human–Computer Interaction phenomenon. Data & Knowledge Engineering, 138:101977. DOI: 10.1016/j.datak.2021.101977.
Costa, S. D., Manso, C. D. F., Marques, L. C., Gadelha, B. F., Conte, T. U., and Barcellos, M. P. (2024). Using networked ontologies to support ux evaluation in immersive context. In Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems, IHC '23, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3638067.3638080.
Elyusufi, Y., Seghiouer, H., and Alimam, M. A. (2014). Building profiles based on ontology for recommendation custom interfaces. In 2014 International Conference on Multimedia Computing and Systems (ICMCS), pages 558-562, Marrakech, Morocco. IEEE. DOI: 10.1109/ICMCS.2014.6911166.
Feilmayr, C. and Wöß, W. (2016). An analysis of ontologies and their success factors for application to business. Data & Knowledge Engineering, 101:1-23. DOI: 10.1016/j.datak.2015.11.003.
Guarino, N. (1998). Formal Ontology in Information Systems: Proceedings of the 1st International Conference June 6-8, 1998, Trento, Italy. IOS Press, Amsterdam, The Netherlands, The Netherlands, 1st edition. Book.
Guarino, N., Oberle, D., and Staab, S. (2009). What Is an Ontology? In Handbook on Ontologies, pages 1-17. Springer Berlin Heidelberg, Berlin, Heidelberg. DOI: 10.1007/978-3-540-92673-3_0.
Guizzardi, G. (2005). Ontological foundations for structural conceptual models. PhD thesis, University of Twente. Available at: [link].
Guizzardi, G. (2007). On Ontology, Ontologies, Conceptualizations, Modeling Languages, and (Meta)Models. In Vasilecas, O., Edler, J., and Caplinskas, A., editors, Proceedings of the Frontiers in Artificial Intelligence and Applications, Databases and Information Systems IV, pages 18-39, Amsterdam, The Netherlands, The Netherlands. IOS Press. Available at: [link].
Guizzardi, G., Falbo, R., and Guizzardi, R. S. S. (2008). Grounding software domain ontologies in the unified foundational ontology (ufo): The case of the ode software process ontology. In Proceedings of the 1th Iberoamerican Workshop on Requirements Engineering and Software Environments (IDEAS'2008), pages 127-140. Available at: [link].
Guizzardi, G., Wagner, G., Falbo, R. d. A., Guizzardi, R. S. S., and Almeida, J. P. A. (2013). Towards Ontological Foundations for the Conceptual Modeling of Events. In Ng, W., Storey, V. C., and Trujillo, J. C., editors, Proceedings of the Conceptual Modeling, pages 327-341, Berlin, Heidelberg. Springer Berlin Heidelberg. DOI: 10.1007/978-3-642-41924-9_27.
Happel, H.-J. and Seedorf, S. (2006). Applications of Ontologies in Software Engineering. Proceedings of the 2nd International Workshop on Semantic Web Enabled Software Engineering (SWESE 2006), held at the 5th International Semantic Web Conference (ISWC 2006), page 14. Available at: [link].
Hassan, H. M. and Galal-Edeen, G. H. (2017). From usability to user experience. In 2017 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), pages 216-222, Okinawa, Japan. IEEE. DOI: 10.1109/ICIIBMS.2017.8279761.
Hassenzahl, M. (2008). User experience (ux) towards an experiential perspective on product quality. In Proceedings of the 20th Conference on l'Interaction Homme-Machine, pages 11-15, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/1512714.1512717.
Hassenzahl, M. and Tractinsky, N. (2006). User experience - a research agenda. Behaviour & Information Technology, 25(2):91-97. DOI: 10.1080/01449290500330331.
ISO (2018). Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts. Standard ISO 9241-11:2018, International Organization for Standardization. Available at: [link].
ISO (2019). Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems. Standard ISO 9241-210:2019, International Organization for Standardization. Available at: [link].
ISO/IEC (2007). Software engineering — Software product Quality Requirements and Evaluation (SQuaRE) — Measurement reference model and guide. Standard ISO 25020:2007, International Organization for Standardization. Available at: [link].
ISO/IEC/IEEE (2017). Systems and software engineering — Measurement process. Standard ISO 15939:2017, International Organization for Standardization. Available at: [link].
Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P., and Kort, J. (2009). Understanding, scoping and defining user experience: a survey approach. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 719-728, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/1518701.1518813.
Marques, L., Amazonas, M., Castro, T., Assuncao, W., Zaina, L., Gadelha, B., and Conte, T. (2020). UX-Trek: A post-interaction journey from immersive experience logs. In Proceedings of the XIX Brazilian Symposium on Human Factors in Computing Systems (IHC'20), page 6, Diamantina, Brazil. ACM. newblock doi:hrefhttp://doi.org/10.1145/3424953.342654710.1145/3424953.3426547. DOI: https://doi.org/10.1145/3424953.3426547.
McGarry, J., Card, D., Jones, C., Layman, B., Ckark, E., Dean, J., and Hall, F. (2002). Practical Software Measurement: Objective Information for Decision Makers. Addison-Wesley. Available at: [link].
Mezhoudi, N. and Vanderdonckt, J. (2015). A user's feedback ontology for context-aware interaction. In Proceedings of the 2015 2nd World Symposium on Web Applications and Networking (WSWAN), pages 1-7, Sousse, Tunisia. IEEE. DOI: 10.1109/WSWAN.2015.7210331.
Nagel, L., Karras, O., and Klunder, J. (2021). Ontology-Based Software Graphs for Supporting Code Comprehension During Onboarding. In Proceedings of 47th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pages 158-165. IEEE. DOI: 10.1109/SEAA53835.2021.00028.
Nakamura, W. T., de Oliveira, E. C., de Oliveira, E. H., Redmiles, D., and Conte, T. (2022). What factors affect the ux in mobile apps? a systematic mapping study on the analysis of app store reviews. Journal of Systems and Software, 193(C):111462. DOI: 10.1016/j.jss.2022.111462.
Negru, S. and Buraga, S. (2013). A Knowledge-Based Approach to the User-Centered Design Process. In Fred, A., Dietz, J. L. G., Liu, K., and Filipe, J., editors, Proceedings of the Knowledge Discovery, Knowledge Engineering and Knowledge Management. IC3K 2012. Communications in Computer and Information Science, vol 415., pages 165-178. Springer Berlin Heidelberg, Berlin, Heidelberg. DOI: 10.1007/978-3-642-54105-6_11.
Negru, S. and Buraga, S. C. (2012). Persona Modeling Process - From Microdata-based Templates to Specific Web Ontologies. In Filipe, J. and Dietz, J., editors, Proceedings of the International Conference on Knowledge Engineering and Ontology Development, pages 34-42, Barcelona, Spain. SciTePress - Science and and Technology Publications. DOI: 10.5220/0004127000340042.
Nielsen, J. (1992). Finding usability problems through heuristic evaluation. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 373-380. DOI: 10.1145/142750.142834.
O’Brien, H. L., Cairns, P., and Hall, M. (2018). A practical approach to measuring user engagement with the refined user engagement scale (ues) and new ues short form. International Journal of Human-Computer Studies, 112:28-39. DOI: 10.1016/j.ijhcs.2018.01.004.
Petrie, H. and Bevan, N. (2009). The Evaluation of Accessibility, Usability, and User Experience. The universal access handbook, 1:1-16. DOI: 10.1201/9781420064995-c20.
Pettersson, I., Lachner, F., Frison, A.-K., Riener, A., and Butz, A. (2018). A bermuda triangle? a review of method application and triangulation in user experience evaluation. In Proceedings of the 2018 CHI conference on human factors in computing systems, CHI '18, page 1–16, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3173574.3174035.
Preece, J., Sharp, H., and Rogers, Y. (2019). Interaction Design: Beyond Human-Computer Interaction. John Wiley & Sons, USA, 5th edition. Book.
Rivero, L. and Conte, T. (2017). A systematic mapping study on research contributions on ux evaluation technologies. In Proceedings of the XVI Brazilian symposium on human factors in computing systems, pages 1-10, New York, NY, USA. Association for Computing Machinery. DOI: 10.1145/3160504.3160512.
Robson, C. and McCartan, K. (2016). Real World Research. Wiley, 4th edition. Book.
Runeson, P., Host, M., Rainer, A., and Regnell, B. (2012). Case study research in software engineering: Guidelines and examples. John Wiley & Sons, 1st edition. Book.
Ruy, F. B., Falbo, R. d. A., Barcellos, M. P., Costa, S. D., and Guizzardi, G. (2016). SEON: A Software Engineering Ontology Network. In Blomqvist, E., Ciancarini, P., Poggi, F., and Vitali, F., editors, Proceedings of the Knowledge Engineering and Knowledge Management, pages 527-542, Cham. Springer International Publishing. DOI: 10.1007/978-3-319-49004-5_34.
Santos Júnior, P. S., Barcellos, M. P., Falbo, R. d. A., and Almeida, J. P. A. (2021). From a Scrum Reference Ontology to the Integration of Applications for Data-Driven Software Development. Information and Software Technology, 136:106570. DOI: 10.1016/j.infsof.2021.106570.
Scherp, A., Saathoff, C., Franz, T., and Staab, S. (2011). Designing core ontologies. Applied Ontology, 6(3):177-221. DOI: 10.3233/AO-2011-0096.
Staab, S. and Studer, R. (2004). Handbook on Ontologies. Springer Berlin Heidelberg, Berlin, Heidelberg, 1 edition. DOI: 10.1007/978-3-540-24750-0.
Studer, R., Benjamins, V., and Fensel, D. (1998). Knowledge engineering: Principles and methods. Data & Knowledge Engineering, 25(1-2):161-197. DOI: 10.1016/S0169-023X(97)00056-6.
Suárez-Figueroa, M. C., Gómez-Pérez, A., Motta, E., and Gangemi, A. (2012). Ontology Engineering in a Networked World. Springer Berlin Heidelberg, Berlin, Heidelberg. DOI: 10.1007/978-3-642-24794-1.
Suh, A. and Prophet, J. (2018). The state of immersive technology research: A literature analysis. Computers in Human Behavior, 86:77-90. DOI: https://doi.org/10.1016/j.chb.2018.04.019.
W3C (2015). World wide web consortium (w3c), semantic web. Available at: [link]. Last accessed on 2023-02-03.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Simone Dornelas Costa, Carolina de Freitas Manso, Leonardo Carneiro Marques, Bruno Freitas Gadelha, Tayana Uchôa Conte, Monalessa Perini Barcellos
This work is licensed under a Creative Commons Attribution 4.0 International License.
