are not made or distributed for profit or commercial advantage and that copies bear the full cita... more are not made or distributed for profit or commercial advantage and that copies bear the full citation on the first page. Copyrights for components of this work owned by others than IFETS must be honoured. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from the editors at kinshuk@ieee.org. Effects of the ISIS Recommender System for navigation support in self-
Practical Assessment, Research and Evaluation, 2021
Recommended Citation Ackermans, Kevin; Rusman, Ellen; Nadolski, Rob; Brand-Gruwel, Saskia; and Sp... more Recommended Citation Ackermans, Kevin; Rusman, Ellen; Nadolski, Rob; Brand-Gruwel, Saskia; and Specht, Marcus (2021) "Feedback is a gift: Do Video-enhanced rubrics result in providing better peer feedback than textual rubrics?," Practical Assessment, Research, and Evaluation: Vol. 26 , Article 17. DOI: https://doi.org/10.7275/hk9e-8d82 Available at: https://scholarworks.umass.edu/pare/vol26/iss1/17
Educational Technology Research and Development, 2021
Acquiring complex oral presentation skills is cognitively demanding for students and demands inte... more Acquiring complex oral presentation skills is cognitively demanding for students and demands intensive teacher guidance. The aim of this study was twofold: (a) to identify and apply design guidelines in developing an effective formative assessment method for oral presentation skills during classroom practice, and (b) to develop and compare two analytic rubric formats as part of that assessment method. Participants were first-year secondary school students in the Netherlands (n = 158) that acquired oral presentation skills with the support of either a formative assessment method with analytic rubrics offered through a dedicated online tool (experimental groups), or a method using more conventional (rating scales) rubrics (control group). One experimental group was provided text-based and the other was provided video-enhanced rubrics. No prior research is known about analytic video-enhanced rubrics, but, based on research on complex skill development and multimedia learning, we expect...
Psychology, Pedagogy, and Assessment in Serious Games
The objective of this chapter is to provide an overview of the different methods that can be used... more The objective of this chapter is to provide an overview of the different methods that can be used to evaluate the learning outcomes of serious games. These include Randomised Control Trials (RCT), quasi-experimental designs, and surveys. Case studies of a selection of serious games developed for use in higher education are then presented along with evaluations of these games. The evaluations illustrate the different evaluation methods, along with an assessment of how well the evaluation method performed. Finally, the chapter discusses the lessons learned and compares the experiences with the evaluation methods and their transferability to other games.
In this paper we explore existing log files of the VIBOA environmental policy game. Our aim is to... more In this paper we explore existing log files of the VIBOA environmental policy game. Our aim is to identify relevant player behaviours and performance patterns. The VIBOA game is a 50 hours master level serious game that supports inquiry-based learning: students adopt the role of an environmental consultant in the (fictitious) consultancy agency VIBOA, and have to deal with complex, multi-faceted environmental problems in an academic and methodologically sound way. A sample of 118 master students played the game. We used learning analytics to extract relevant data from the logging and find meaningful patterns and relationships. We observed substantial behavioural variability across students. Correlation analysis suggest a behavioural trade that reflects the rate of “switching” between different game objects or activities. We were able to establish a model that uses switching indicators as predictors for the efficiency of learning. Also we found slight evidence that students who displ...
Societal changes demand educators to apply new pedagogical approaches. Many educational stakehold... more Societal changes demand educators to apply new pedagogical approaches. Many educational stakeholders feel that serious games could play a key role in fulfilling this demand, and they lick their chops when looking at the booming industry of leisure games. However, current toolkits for developing leisure games show severe shortcomings when applied to serious games. Developing effective serious games in an efficient way requires a specific approach and tool set. This article describes the EMERGO methodology and generic toolkit for developing and delivering scenario-based serious games that are aimed at the acquisition of complex cognitive skills in higher education. Preliminary evaluation results with case developers using the EMERGO methodology and toolkit and with learners using EMERGO cases are presented.
are not made or distributed for profit or commercial advantage and that copies bear the full cita... more are not made or distributed for profit or commercial advantage and that copies bear the full citation on the first page. Copyrights for components of this work owned by others than IFETS must be honoured. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from the editors at kinshuk@ieee.org. Effects of the ISIS Recommender System for navigation support in self-
Practical Assessment, Research and Evaluation, 2021
Recommended Citation Ackermans, Kevin; Rusman, Ellen; Nadolski, Rob; Brand-Gruwel, Saskia; and Sp... more Recommended Citation Ackermans, Kevin; Rusman, Ellen; Nadolski, Rob; Brand-Gruwel, Saskia; and Specht, Marcus (2021) "Feedback is a gift: Do Video-enhanced rubrics result in providing better peer feedback than textual rubrics?," Practical Assessment, Research, and Evaluation: Vol. 26 , Article 17. DOI: https://doi.org/10.7275/hk9e-8d82 Available at: https://scholarworks.umass.edu/pare/vol26/iss1/17
Educational Technology Research and Development, 2021
Acquiring complex oral presentation skills is cognitively demanding for students and demands inte... more Acquiring complex oral presentation skills is cognitively demanding for students and demands intensive teacher guidance. The aim of this study was twofold: (a) to identify and apply design guidelines in developing an effective formative assessment method for oral presentation skills during classroom practice, and (b) to develop and compare two analytic rubric formats as part of that assessment method. Participants were first-year secondary school students in the Netherlands (n = 158) that acquired oral presentation skills with the support of either a formative assessment method with analytic rubrics offered through a dedicated online tool (experimental groups), or a method using more conventional (rating scales) rubrics (control group). One experimental group was provided text-based and the other was provided video-enhanced rubrics. No prior research is known about analytic video-enhanced rubrics, but, based on research on complex skill development and multimedia learning, we expect...
Psychology, Pedagogy, and Assessment in Serious Games
The objective of this chapter is to provide an overview of the different methods that can be used... more The objective of this chapter is to provide an overview of the different methods that can be used to evaluate the learning outcomes of serious games. These include Randomised Control Trials (RCT), quasi-experimental designs, and surveys. Case studies of a selection of serious games developed for use in higher education are then presented along with evaluations of these games. The evaluations illustrate the different evaluation methods, along with an assessment of how well the evaluation method performed. Finally, the chapter discusses the lessons learned and compares the experiences with the evaluation methods and their transferability to other games.
In this paper we explore existing log files of the VIBOA environmental policy game. Our aim is to... more In this paper we explore existing log files of the VIBOA environmental policy game. Our aim is to identify relevant player behaviours and performance patterns. The VIBOA game is a 50 hours master level serious game that supports inquiry-based learning: students adopt the role of an environmental consultant in the (fictitious) consultancy agency VIBOA, and have to deal with complex, multi-faceted environmental problems in an academic and methodologically sound way. A sample of 118 master students played the game. We used learning analytics to extract relevant data from the logging and find meaningful patterns and relationships. We observed substantial behavioural variability across students. Correlation analysis suggest a behavioural trade that reflects the rate of “switching” between different game objects or activities. We were able to establish a model that uses switching indicators as predictors for the efficiency of learning. Also we found slight evidence that students who displ...
Societal changes demand educators to apply new pedagogical approaches. Many educational stakehold... more Societal changes demand educators to apply new pedagogical approaches. Many educational stakeholders feel that serious games could play a key role in fulfilling this demand, and they lick their chops when looking at the booming industry of leisure games. However, current toolkits for developing leisure games show severe shortcomings when applied to serious games. Developing effective serious games in an efficient way requires a specific approach and tool set. This article describes the EMERGO methodology and generic toolkit for developing and delivering scenario-based serious games that are aimed at the acquisition of complex cognitive skills in higher education. Preliminary evaluation results with case developers using the EMERGO methodology and toolkit and with learners using EMERGO cases are presented.
Uploads
Papers