[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Framework for User Experience Evaluation in MOOC Platforms

  • Conference paper
  • First Online:
Artificial Intelligence in HCI (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13336))

Included in the following conference series:

Abstract

User Experience (UX) from the simplest perspective is defined in how people feel when using a product or service, which is fundamental to the success or failure of any product in the market. On the other hand, Massive Open Online Courses (MOOCs) have become one of the most popular trends in the field of education, reaching great popularity among several universities, which offer MOOCs through prestigious platforms, however, most of them do not meet the expectations and satisfaction of users, and mechanisms have not yet been designed to comprehensively measure the UX in these platforms. Therefore, the objective of this paper is to develop a comprehensive framework for the evaluation of UX in MOOC platforms from a technological point of view, after a systematic review of the literature to identify the most frequently applied and/or important evaluation approaches, which are analyzed and organized according to the following components: technological criteria and MOOC indicators, type of users, UX dimensions and UX factors. Through this approach it is possible to evaluate the UX in individual components, compare it between similar products and measure it over time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Soegaard, M.: The Basics of User Experience Design: A UX Design Book by the Interaction Design Foundation (2002)

    Google Scholar 

  2. Morville, P.: The 7 Factors that Influence User Experience, Interaction Design Foundation (2018). https://www.interaction-design.org/literature/article/the-7-factors-that-influence-user-experience

  3. Rabahallah, K., Mahdaoui, L., Azouaou, F.: MOOCs recommender system using ontology and memory-based collaborative filtering. In: ICEIS 2018 - 20th International Conference on Enterprise Information Systems, vol. 1, ICEIS, pp. 635–641 (2018)

    Google Scholar 

  4. Foon, K., Sum, W.: Students’ and Instructors’ use of Massive Open Online Courses (MOOCs): motivations and challenges. Educ. Res. Rev. 12, 45–58 (2014). https://doi.org/10.1016/j.edurev.2014.05.001

    Article  Google Scholar 

  5. Jordan, K.: Massive open online course completion rates revisited: assessment, length and attrition. Int. Rev. Res. Open Distrib. Learn. 16(3), 341–358 (2015)

    Google Scholar 

  6. Luján, S.: “¿Qué son los MOOCs?” (2012). http://desarrolloweb.dlsi.ua.es/cursos/2012/que-son-los-moocs/preguntas-respuestas#que-es-un-mooc. Accessed 30 Jan 2020

  7. Cruz, J., Borras, O., Garcia, F., Blanco, A., Theron, R.: Learning communities in social networks and their relationship with the MOOCs. Rev. Iberoam. Tecnol. del Aprendiz. 12(1), 24–36 (2017). https://doi.org/10.1109/RITA.2017.2655218

    Article  Google Scholar 

  8. Frolov, I., Johansson, S.: An adaptable usability checklist for MOOCs A usability evaluation instrument for Massive Open Online Courses (2014)

    Google Scholar 

  9. Hakami, N., White, S., Chakaveh, S.: Motivational factors that influence the use of MOOCs: learners’ perspectives a systematic literature review. In: CSEDU 2017 – Proceedings of 9th International Conference on Computer Supporting Education, vol. 2, CSEDU, pp. 323–331 (2017). https://doi.org/10.5220/0006259503230331

  10. Pascual, J., Castillo, C., García, V., González, R.: Method for analysing the user experience in MOOC platforms. In: 2014 International Symposium Computing Education, SIIE 2014, pp. 157–162 (2014). https://doi.org/10.1109/SIIE.2014.7017722

  11. Wautelet, Y., Heng, S., Kolp, M., Penserini, L., Poelmans, S.: Designing an MOOC as an agent-platform aggregating heterogeneous virtual learning environments. Behav. Inf. Technol. 35(11), 980–997 (2016). https://doi.org/10.1080/0144929X.2016.1212095

    Article  Google Scholar 

  12. Korableva, O., Durand, T., Kalimullina, O., Stepanova, I.: Studying user satisfaction with the MOOC platform interfaces using the example of coursera and open education platforms. In: ACM International Conference Proceeding Series, pp. 26–30 (2019). https://doi.org/10.1145/3322134.3322139

  13. Korableva, O., Durand, T., Kalimullina, O., Stepanova, I.: Usability testing of MOOC: Identifying user interface problems. In: ICEIS 2019 – 21st International Conference on Enterprise Information Systems, vol. 2, ICEIS, pp. 468–475 (2019). https://doi.org/10.5220/0007800004680475

  14. Xiao, J., Jiang, B., Xu, Z., Wang, M.: The usability research of learning resource design for MOOCs. In: 2014 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), pp. 277–282 (2014). https://doi.org/10.1109/TALE.2014.7062640

  15. Zaharias, P., Poylymenakou, A.: Developing a usability evaluation method for e-learning applications: beyond functional usability. Int. J. Hum. Comput. Interact. 25(1), 75–98 (2009). https://doi.org/10.1080/10447310802546716

    Article  Google Scholar 

  16. Azhar, T., Kasiyah, H.S.: Evaluation of instructional and user interface design for MOOC: short and free futurelearn courses. In: 2019 International Conference on Advanced Computer Science Information System, ICACSIS 2019, pp. 425–434 (2019). https://doi.org/10.1109/ICACSIS47736.2019.8979754

  17. Veletsianos, G., Collier, A., Schneider, E.: Digging deeper into learners’ experiences in MOOCs: Participation in social networks outside of MOOCs, notetaking and contexts surrounding content consumption. Br. J. Educ. Technol. 46(3), 570–587 (2015). https://doi.org/10.1111/bjet.12297

    Article  Google Scholar 

  18. Poma, A., Rodríguez, G., Torres, P.: User experience evaluation in MOOC platforms: a hybrid approach human-computer interaction. In: Ruiz, P.H., Agredo-Delgado, V., Kawamoto, A.L.S. (eds.) Human-Computer Interaction: 7th Iberoamerican Workshop, HCI-COLLAB 2021, pp. 208–224. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-92325-9_16

  19. Jacoby, J.: The disruptive potential of the Massive Open Online Course: a literature review. J. Open Flexible Distance Learn. 18, 73–85 (2018). http://journals.akoaotearoa.ac.nz/index.php/JOFDL/article/viewFile/214/168%5Cnhttps://ezproxy.royalroads.ca/login?url=http://search.proquest.com/docview/1749263827?accountid=8056%5Cnhttp://wp6eu6tz5x.search.serialssolutions.com/?ctx_ver=Z39.88-2004&ctx_en

    Google Scholar 

  20. McAvinia, C.: Lessons for the Future – The VLE and the MOOC. In: Online Learning and its Users, pp. 207–228 (2016)

    Google Scholar 

  21. Peco, P.P., Mora, S.L.: Los MOOC: orígenes, historia y tipos, Centro de Comunicación y Pedagogía Facebook Twitter Pinterest (2013). http://www.centrocp.com/los-mooc-origenes-historia-y-tipos/. Accessed 27 Jan 2020

  22. Casanova, A., Espinoza, A.: Experiencia De Usuario En Entornos Virtuales De Aprendizaje (2015)

    Google Scholar 

  23. Kopp, M., Lackner, E.: Do MooCs need a special instructional design? In: EDULEARN 2014 Proceedings, July 2014, pp. 7138–7147 (2014). http://library.iated.org/view/KOPP2014DOM

  24. Fahmy, Y., Chatti, M., Schroeder, U., Wosnitza, M.: What drives a successful MOOC? An empirical examination of criteria to assure design quality of MOOCs. In: 2014 IEEE 14th International Conference on Advanced Learning Technologies, ICALT 2014, 2014, pp. 44–48, July 2014. https://doi.org/10.1109/ICALT.2014.23

  25. Norman, D.A., Henderson, A., Nonnan, D., Miller, J., Henderson, A.: What you see, some of what’s in the future, and how we go about doing it: HI at Apple computer. In: Conference Companion Human Factors Computing Systems - CHI 1995, January 1995, pp. 4–5 (1995). https://doi.org/10.1145/223355.223477

  26. Hassenzahl, M., Tractinsky, N.: User experience - a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006). https://doi.org/10.1080/01449290500330331

    Article  Google Scholar 

  27. Hassan, Y., Martín, F.: La Experiencia del Usuario (2005). http://www.nosolousabilidad.com/articulos/experiencia_del_usuario.htm. Accessed 11 Mar 2020

  28. Kuniavsky, M.: User Experience and HCI. In: The Human-Computer Interaction Handbook Fundamentals, Evolving Technologies, and Emerging Applications, 2nd edn., pp. 897–916 (2009)

    Google Scholar 

  29. Garrett, J.J.: The Elements of User Experience: User Centered Design for the Web and Beyond, Second, Berkeley (2011)

    Google Scholar 

  30. Norman, D., Nielsen, J.: The Definition of User Experience (UX) (2003). https://www.nngroup.com/articles/definition-user-experience/. Accessed 11 Mar 2020

  31. Arhippainen, L., Tähti, M.: Empirical evaluation of user experience in two adaptive mobile application prototypes. In: Proceedings of 2nd International Conference on Mobile and Ubiquitous Multimedia, pp. 27–34 (2003). http://www.ep.liu.se/ecp/011/007/ecp011007.pdf

  32. Hassenzahl, M.: User experience (UX): towards an experiential perspective on product quality. In: ACM International Conference Proceeding Series, September 2008, pp. 11–15 (2008). https://doi.org/10.1145/1512714.1512717

  33. Roto, V., Law, E., Vermeeren, A., Hoonhout, J.: Abstracts collection demarcating user experience. In: Dagstuhl Semin. Proceedings, pp. 1–26 (2011)

    Google Scholar 

  34. ISO 9241-11: 2018(en), Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts (2018). https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en. Accessed 25 July 2020

  35. Morville, P.: User Experience Design (2004). http://semanticstudios.com/user_experience_design/. Accessed 29 July 2020

  36. Tullis, T., Albert, B.: Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics, 2nd edn., London, UK (2013)

    Google Scholar 

  37. Roto, V., et al.: All UX evaluation methods, ALL ABOUT UX (2011). https://www.allaboutux.org/all-methods. Accessed 14 June 2020

  38. Sánchez, L., Urquiza, J., Mendoza, S.: Measuring anticipated and episodic UX of tasks in social networks. Appl. Sci. 10(22), 1–17 (2020). https://doi.org/10.3390/app10228199

    Article  Google Scholar 

  39. Dix, A., Finlay, J., Abowd, G.D., Beale, R.: Human-Computer Interaction Chapter 9: Evaluation Techniques (2004)

    Google Scholar 

  40. Granollers, T.: MPIu+a. Una metodología que integra la Ingeniería del Software, la Interacción Persona-Ordenador y la Accesibilidad en el contexto de equipos de desarrollo multidisciplinares, Universitat de Lleida (2004)

    Google Scholar 

  41. Nurhudatiana, A., Anggraeni, A., Putra, S.: An exploratory study of MOOC adoption in Indonesia. In: ACM International Conference Proceeding Series, pp. 97–101 (2019). https://doi.org/10.1145/3337682.3337690

  42. Lu, Y., Wang, B., Lu, Y.: Understanding key drivers of MOOC satisfaction and continuance intention to use. J. Electron. Commer. Res. 20(2), 105–117 (2019)

    MathSciNet  Google Scholar 

  43. Moshagen, M., Thielsch, M.T.: A short version of the visual aesthetics of websites inventory. Behav. Inf. Technol. 32(12), 1305–1311 (2013). https://doi.org/10.1080/0144929X.2012.694910

    Article  Google Scholar 

  44. Liu, S., Liang, T., Shao, S., Kong, J.: Evaluating localized MOOCs: the role of culture on interface design and user experience. IEEE Access 8, 107927–107940 (2020). https://doi.org/10.1109/ACCESS.2020.2986036

    Article  Google Scholar 

  45. Hanifa, M.R., Santoso, H.B., Kasiyah: Evaluation and recommendations for the instructional design and user interface design of Coursera MOOC platform. In: 2019 International Conference on Advanced Computer Science and information Systems (ICACSIS) ICACSIS 2019, vol. 2014, pp. 417–424 (2019). https://doi.org/10.1109/ICACSIS47736.2019.8979689

  46. Jiménez, S., et al.: Heuristic approach to evaluate basic types of interactions-communications in MOOCs (2016)

    Google Scholar 

  47. Acosta, T., Zambrano, J., Luján, S.: Analysis of the accessibility of educational videos in Massive Open Online Courses. In: EDULEARN 2019 Proceedings, vol. 1, pp. 8321–8331, July 2019. https://doi.org/10.21125/edulearn.2019.2076

  48. Iniesto, F., Covadonga, R.: Accessibility assessment of MOOC platforms in Spanish: UNED COMA, COLMENIA and Miriada X, pp. 169–172 (2014). https://doi.org/10.1109/SIIE.2014.7017724

  49. Calle, T., Sánchez, S., Luján, S.: Web accessibility evaluation of massive open online courses on Geographical Information Systems. In: IEEE Global Engineering Education Conference EDUCON, pp. 680–686, April 2014. https://doi.org/10.1109/EDUCON.2014.6826167

  50. Tsironis, A., Katsanos, C., Xenos, M.: Comparative usability evaluation of three popular MOOC platforms. In: IEEE Global Engineering Education Conference EDUCON, pp. 608–612 (2016). https://doi.org/10.1109/EDUCON.2016.7474613

  51. Chen, C., Lee, C., Hsiao, K., Chen, C., Lee, C., Hsiao, K.: Comparing the determinants of non-MOOC and MOOC continuance intention in Taiwan effects of interactivity and openness (2018). https://doi.org/10.1108/LHT-11-2016-0129

  52. Nurhudatiana, A., Caesarion, A.S.: Exploring user experience of massive open online courses (MOOCs), pp. 44–49 (2020). https://doi.org/10.1145/3383923.3383968

  53. Rabin, E., Kalman, Y.M., Kalz, M.: An empirical investigation of the antecedents of learner-centered outcome measures in MOOCs. Int. J. Educ. Technol. High. Educ. 16(1), 1–20 (2019). https://doi.org/10.1186/s41239-019-0144-3

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Electronic Department, Universidad Técnica Particular de Loja, Loja, Ecuador

    Ana Poma Gallegos, Germania Rodríguez Morales, Pablo V. Torres-Carrión & Samanta Cueva Carrión

Authors
  1. Ana Poma Gallegos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Germania Rodríguez Morales
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pablo V. Torres-Carrión
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Samanta Cueva Carrión
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Germania Rodríguez Morales .

Editor information

Editors and Affiliations

  1. Siemens (United States), Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Appendix

Appendix

Appendix 1.

Components of the proposed framework for UX evaluation in MOOC platforms (FUXE-MOOC).

figure a

Appendix 2.

Description of the UX evaluation approaches that make up FUXE-MOOC.

Table 1. Description of anticipated UX evaluation approaches.
Full size table
Table 2. Description of task-based UX evaluation approaches.
Full size table
Table 3. Description of general EUX or UX evaluation approaches.
Full size table
Table 4. Description of cumulative UX evaluation approaches.
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Poma Gallegos, A., Rodríguez Morales, G., Torres-Carrión, P.V., Cueva Carrión, S. (2022). Framework for User Experience Evaluation in MOOC Platforms. In: Degen, H., Ntoa, S. (eds) Artificial Intelligence in HCI. HCII 2022. Lecture Notes in Computer Science(), vol 13336. Springer, Cham. https://doi.org/10.1007/978-3-031-05643-7_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05643-7_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05642-0

  • Online ISBN: 978-3-031-05643-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation