Automated Assessment

Automated assessment tools (AATs) are software systems used in teaching environments to automate the evaluation of computer programs implemented by students. These tools can be used to stimulate the interest of computer science students in programming courses by providing quick feedback on their work and highlighting their mistakes. Despite the abundance of such tools, most of them are developed for a specific course and are not production-ready. Others lack advanced features that are required for certain pedagogical goals (e.g. Git integration) and/or are not flexible enough to be used with students having different computer literacy levels, such as first year and second year students. In this paper we present Drop Project (DP), an automated assessment tool built on top of the Maven build automation software. We have been using DP in our teaching activity since 2018, having received more than fifty thousand submissions between projects, classroom exercises, tests and homework assignments. The tool's automated feedback has allowed us to raise the difficulty level of the course's projects, while the grading process has become more efficient and consistent between different teachers. DP is an extensively tested, production-ready tool. The software's code and documentation are available in GitHub under an open-source software license.

Online journal submission systems help lecturers with time management concerns and student evaluations processes. The aim of this paper is to explore available tools, design and develop an interactive web based application which can be used by computer science students to submit programming journals and get real-time feedback. The online application which has been designed and developed within the scope of this paper consists of an authorization, submission and analysis component.

Facilitating and empowering learners and teachers through technology has primarily concentrated on content and management aspects rather than on assessments aspects of the education enterprise. Recent developments in virtual world technology have sparked a raft of projects exploring the possibilities to enhance learning outcomes. In this article, automated support for educational assessment is focussed on via three case studies implemented in Second Life, the 3D virtual world implemented by Linden Labs. This paper discusses the assessment tasks, the methods used and provides an analysis of our results. The University of Hamburg case study involves scenario-based software development projects. The Curtin University case study focuses on simulating the business processes of the Automated Assessment Lab, which provides automated assessment opportunities to staff of the Curtin Business School, and in which a virtual world model of the entire assessment life cycle is created. A third case study from the University of New England, VirtualPREX (Virtual Professional Experience), explores self, peer and academic formative and summative assessment in virtual world role-play scenarios. Each case study comprises a description of intent and implementation, followed by evaluative comments from the role players, providing insight into the benefits that may accrue from the use of virtual world technology for students, teachers and educational administrators.

Inform plays a very important role in life and nowadays, the world largely depends on the World Wide Web to obtain any information. Web comprises of a lot of websites of every discipline, whereas websites consists of web pages which are interlinked with each other with the help of hyperlinks. The success of a website largely depends on the design aspects of the web pages. Researchers have done a lot of work to appraise the web pages quantitatively. Keeping in mind the importance of the design aspects of a web page, this paper aims at the design of an automated evaluation tool which evaluate the aspects for any web page. The tool takes the HTML code of the web page as input, and then it extracts and checks the HTML tags for the uniformity. The tool comprises of normalized modules which quantify the measures of design aspects. For realization, the tool has been applied on four web pages of distinct sites and design aspects have been reported for comparison. The tool will have various advantages for web developers who can predict the design quality of web pages and enhance it before and after implementation of website without user interaction.

Despite the rapid emergence of game-based learning as a method for conveying educational content, constructing pedagogies which effectively combine elements of entertainment gaming with methods of instruction remains a demanding task. Through the notion of 'intuitive guided' learning, this paper presents an approach which seeks to facilitate a structured learning experience whilst allowing learners to explore a non-linear environment. To do so, a framework is presented which externalizes the assessment process in a serious game, whilst also providing a means for game content to be adapted dynamically to translate the outcomes of the assessment process to effective feedback. A developed prototype is implemented to examine the theory in-practice through the case of a game for civil defence training in schools. Using this prototype, a range of methods in which achievements might be related to learner actions are introduced, and their subsequent implications for intuitive learning discussed. Furthermore, the prototype illustrates how assessment rules can be defined as external to the game and subsequently used to generate feedback for a virtual companion who assumes the role of a more-able partner. The long-term potential of such methods as a source of data on player behaviour is discussed, suggesting further benefits the technique might offer to educators seeking to introduce game-based learning within the curriculum in a blended fashion.

Neonatal endotracheal intubation (ETI) is a resuscitation skill and therefore, requires an effective training regimen with acceptable success rates. However, current training regimen faces some challenges , such as the lack of visualization inside the manikin and quantification of performance, resulting in inaccurate guidance and highly variable manual assessment. We present a Cross Reality (XR) ETI simulation system which registers ETI training tools to their virtual counterparts. Thus, our system can capture all aspects of motions and visualize the entire procedure, offering instructors with sufficient information for assessment. A machine learning approach was developed to automatically evaluate the ETI performance for standardizing assessment protocols by using the performance parameters extracted from the motions and the scores from an expert rater. The classification accuracy of the machine learning algorithm is 83.5%.

Recently, a new age of information has appeared where information and communication technology plays a major role in education and learning society. Consequently, modern learning settings represented by emerging learning strategies have been used in higher education. A primary goal for higher education is to support students and lifelong learners to be independent, self-motivated, and self-regulated. As a result, students should be provided with enhanced approaches of learning and assessment that support them to set their learning goals effectively, to plan and use effective strategies in order to achieve their goals, to manage resources, to monitor their understanding, and to assess their progress towards their goals. This paper proposes an enhanced approach of e-assessment where automatically created formative assessments are provided to support self-regulated learners.

This paper addresses two MOOCs in Estonian about programming where different kinds of assessment were used. We have used two kinds of automated assessment: quizzes in Moodle and programming exercises with automated feedback provided by Moodle plug-in VPL. We also used two kinds of self-assessment: (1) self-assessment questions with feedback and explanations for every answer and (2) so-called “troubleshooters” for every programming exercise, which contain answers to the questions that can arise during the solution of a given exercise. This paper describes our experience in the creation of quizzes, programming exercises, and tests for automated feedback, self-assessment questions, and troubleshooters. The paper discusses the problems and questions that arose during this process and presents learners’ opinions about self- and automated assessment. The paper concludes with a discussion of the impact of self- and automated assessment in MOOCs, describes the work of MOOC organizers and the behaviour of learners in MOOCs.

This paper addresses two MOOCs in Estonian about programming where different kinds of assessment were used. We have used two kinds of automated assessment: quizzes in Moodle and programming exercises with automated feedback provided by Moodle plug-in VPL. We also used two kinds of self-assessment: (1) self-assessment questions with feedback and explanations for every answer and (2) so-called “troubleshooters” for every programming exercise, which contain answers to the questions that can arise during the solution of a given exercise. This paper describes our experience in the creation of quizzes, programming exercises, and tests for automated feedback, self-assessment questions, and troubleshooters. The paper discusses the problems and questions that arose during this process and presents learners’ opinions about self- and automated assessment. The paper concludes with a discussion of the impact of self- and automated assessment in MOOCs, describes the work of MOOC organizers and th...