Usability Evaluation of Imikode Virtual Reality Game to Facilitate Learning of Object-Oriented Programming

Nowadays, playing a digital game is one of the main ways of getting acquainted with computers, and many people spend a lot of time playing games. The use of games in education is not new. Many empirical studies have shown that educational games impact education and increase learning effectiveness, students’ motivation and engagement (Bile, 2022; Brown et al., 2009; Erhel & Jamet, 2019; Oyelere et al., 2019; Peters et al., 2021). The evolution of new technologies has not only paved the way for novel ways to design and develop digital games but also for how users experience the games. For example, immersion, the perception of being physically present in a virtual world in which a person loses awareness of the physical environment, is created by virtual reality (VR) technology. VR games which are increasingly popular among children and adults develop a sense of presence that fundamentally enables them to feel connected, engage and interact within the virtual environment. Virtual reality games have gained relevance in formal, non-formal, and informal settings and across a wide range of learning topics, learning objectives, and other pedagogical attributes (Checa et al., 2019; Sunday et al., 2022).

Educational virtual reality games offer good learning opportunities, but only if the game is well designed and rigorously evaluated to ensure that it meets the overall learning goal. The aim of this study is to evaluate the usability of Imikode, a VR game that was developed to teach the concepts of object-oriented programming to novices (Bouali et al., 2019). Usability evaluation is used primarily to assess how well users can use and learn a particular system to achieve their desired goals. Therefore, this study will address the evaluation of virtual reality game in two ways: first, to measure the usefulness, ease of use, ease of learning and learner satisfaction of educational virtual reality games for learning object-oriented programming using the USE (Usefulness, Satisfaction, Ease of Use and Ease of Learning) questionnaire. Second, to examine the correlations among the USE questionnaire attributes to understand the extent to which the variables are related in the context of usage of Imikode. Hence, the following research questions were investigated to determine the research intentions:

1. (i)

   How do the students find the Imikode educational virtual reality game usable for learning?

2. (ii)

   How does ease of use, ease of learning and usefulness attributes contribute to user satisfaction with Imikode?

The following section elaborates on the review of related literature, including factors affecting student performance in programming in Nigeria. VR-based learning programming solutions, the usability evaluation of VR tools and how Imikode was used were included in this study. In the subsequent sections, the description and analysis of the evaluation results were provided. In addition to evaluating the Imikode artifact regarding its usability for learning and understanding the extent of the correlation of the USE questionnaire attributes, this study provides valuable findings on improving the future version of the Imikode system. Similarly, learned lessons can specify design guidelines for educational VR games for particular contextual applications.

Virtual reality is a simulated environment that resembles the real world. It involves an interactive and multimodal technology through which users interact with the simulated world in the best natural way possible (Cruz-Neira et al., 2018). With VR, the real world can be simulated or augmented cost-effectively and risk-free. In this simulated environment, a user typically makes use of a special goggle and fibre optic gloves to move around and interact with the object in a way that looks as if they are immersed in it. A small VR device can act like a whole science laboratory offering students practical experience with their subjects and learners can then explore topics and see how things are combined (Babich, 2019).

One of the prominent features of VR technology is the concept of immersion and presence. During immersion, users remain in the VR zone and are fully involved in the learning process to engage the VR tool for better representation of reality which involves a far-reaching view by the users (Slater & Wilbur, 1997). This concept also captures the degree to which computers are used to present an inclusive and elaborate environment that provides an illusion of reality to the human senses (Slater & Wilbur, 1997). VR presence, however, is concerned with the conscious awareness of existing in the virtual world. This perceptual illusion recognizes the objects and events coupled with a spontaneous reaction by the brain-body system to change which results in a slow response by the cognitive system. Furthermore, factors such as emotions, societal and internal factors influence a user’s response and experiences of the technology which are mainly psychological in nature (Berkman & Akan, 2018).

Gimeno et al., (2007) developed a reconfigurable immersive workbench and wall system for designing and training in 3D environments. They introduced a new concept of workbench designed for virtual and augmented reality purposes and specifically oriented to the fields of tele-education and engineering. This new approach allows easy utilization and configuration as well as a cost-effective immersive visualization system based on off-the-shelf elements. Similar studies include Pan et al. (2006), Gavish et al. (2015), Lau and Lee (2015); entertainment (Hock et al., 2017; Vargas et al., 2017); flight and driving simulation (Casas et al., 2016); exergames for health and well-being (Barathi et al., 2018; Ijaz et al., 2019; Balasaravanan & Andrew, 2022); sketching in 3D (Machuca et al., 2019) and cultural heritage (Portales et al., 2017).

Many students are slow learners hence, they need to explore things on their own to understand concepts, especially courses that are science inclined. This is because science-based classes often place great psychological and cognitive demand on learners. Furthermore, the students involved in this learning process tend to avoid further exploration of complicated problems involving science-based courses due to their poor foundation (Tekkumru-Kisa et al., 2021; Yager, 2000; Zheng et al., 2015). A study on game based learning by Bile (2022) has shown the effectiveness of using technology in enhancing the teaching and learning of science based concepts. Specifically, Bile (2022) used the Minecraft game application to teach the basic principles of some science based courses such as geometry, mathematics and computer science to 186 students. The results of the evaluation study revealed that students’ learning increased when they are involved in the actual game play. In addition, learning through this method fostered the ability for these categories of students to connect, disseminate and extend the acquired knowledge to others.

2.1 Factors Affecting Student Performance in Programming in Nigeria

Programming is the act of writing computer programs to solve a particular task. There is no gainsaying that computer programs have taken over the way things are done in today’s world. Hence, it is essential to leverage on this technology to students in tertiary institutions. For students at various universities to acquire the basic programming knowledge and skills, teaching and learning introductory programming becomes indispensable (Divna et al., 2015). According to Divna and colleagues, despite this lofty idea of catching them young by instilling programming skills early enough at the university level, students often find this subject very challenging. In addition, these courses titled introductory programming are often concealed under the title “computer science” in the literature (Radenski, 2006).

According to Jens et al. (2008), students' programming abilities are essentially a function of their mathematical prowess; hence, teachers leverage this method to develop a curriculum that will benefit students undertaking mathematical studies, while other areas such as problem solving skills are ignored in contrast to extant literature on the enormous benefits of programming in developing problem solving skills (Bile, 2022; Brown et al., 2009; De Freitas, 2018; Erhel & Jamet, 2019; Papastergiou, 2008; Peters et al., 2021; Pierre et al., 2020). A study by Divna et al. (2015) conducted a correlation analysis to unravel the relationship between mathematical abilities and students' programming prowess. However, the research found a significant relationship between the success of students in both mathematical and programming courses. Thus, if a student fails a mathematics subject, that student has a high probability of failing programming courses. They concluded by advocating a close examination of the programming curriculum to determine if the presented concepts and assignments favor a mathematical way of thinking. Ten factors affecting student performance in programming courses were emphasized by Hawi (2010).

According to Hawi (2010), some of the factors which became evident as a result of the interviews and observation based on their performance in a programming course included: “cheating”, “teaching method”, “lack of study”, “exam anxiety”, “lack of practice” and “learning strategy”. Some of these factors were also highlighted by Jens et al. (2008) in their research to predict students' success in a programming class. A study conducted by Sunday et al. (2020) used a classification method, such as the decision tree algorithm to analyze the performance of students in a programming class. Specifically, they used the ID3 & J48 classifier to analyze 239 instances in a programming course at Usmanu Danfodiyo University, Sokoto, Nigeria. The results of their study showed that students' massive failure rate in introductory programming courses was due to the low rate of class attendance by the students and the lack of infrastructure to support a large number of students in a class. Similarly, Dasuki and Quaye (2016) examined the factors responsible for students' massive failure in introductory programming in a Nigerian institution. According to Dasuki and colleagues, poor lecturer skills and behavior, peer influences, anxiety, lack of future expectations, and lack of intrinsic motivation are the primary factors contributing to the failure rate in programming.

2.2 Existing VR-based Tools to Learn Programming

To support and ease learning of programming, several VR tools/systems have been proposed/ developed to help students learn effectively. Table 1 shows a summary of the software that support learning programming.

2.3 Usability Evaluation of Educational Technology Tools

According to usability.gov (2020), usability measures a user’s experience when interacting with products, systems, software, devices, etc. It measures the effectiveness or efficiency of the product and the overall user satisfaction. It also measures the intuitiveness of the error frequency, severity, design, efficiency of use, satisfaction, and ease of learning design. Therefore, this research aims to evaluate a VR game for learning programming to measure students' usability and satisfaction. Studies such as Mikropoulos and Natsis (2011) have concluded that 3D spaces (including VR) bring some benefits to the increasingly complex problem such as the ability to learn by experience, fostering collaborative learning and the willingness to tackle problems. This has enabled virtual reality applications to be well grounded in education, including teaching programming courses. 3D applications are often designed to support VR in education and enable the user to navigate the environment seamlessly (Segura et al., 2019). However, measuring this technology's effect, particularly for teaching programming, would significantly improve the learning process. Sutcliffe and Kaur (2000) evaluated the usability of the Virtual Reality user interfaces using the theory of interaction. However, this study does not consider the perspective of the users themselves. Another study by De Pace et al. (2019) measured the VR game interface's usability using a hybrid approach in which the first player interacts with AR while the second player interacts with VR. The manner in which users navigate through the process determines the usability. According to the authors, it is challenging to measure usability in this manner. Nonetheless, it is a unique approach that has been invented. In this work, another dimension had been taken to evaluate the usability by involving users. As the users are the system's ultimate users, their evaluation is better for determining the VR/AR system's usability.

This study demonstrates VR games (Imikode) in a programming class to teach and learn object-oriented programming (OOP) concepts. As already established, 3D games have been known to improve students’ performance and results compared to 2D games (De Boer et al., 2015; Shengcai et al., 2013). For example, the quality of the 3D environment in the Imikode VR tool makes it very easy for students to locate their current position in the virtual world. Furthermore, the appealing nature of the 3D game enhances the identification and creation of objects without spending too much time gallivanting around the space. Imikode, a 3D game, was intentionally developed to support students’ understanding of OOP concepts within an immersive virtual environment. The choice of Imikode over other VR tools is its user interface design. Imikode makes it possible for users to play games with little or no supervision. This is because of the integration of an intelligent agent (Bob) that virtually guides users at each step of the game play. Furthermore, the game incorporated many OOP features that were tailored to the needs of the students. According to a survey by Sunday et al. (2020), the students preferred a simple approach to learning OOP for better understanding such as learning the basics of OOP before delving into classes, constructors, method overloading/ overriding, and garbage collection.

Furthermore, the game depicts a typical classroom setting where students write on the board, cleans the board when errors are made and many more. Imikode was implemented with Google Cardboard SDK (Software Development Kits), although Imikode can be played by using more advanced headsets, minimum requirements in terms of devices that support the use of Imikode were considered in the development. This approach provides users with low income the opportunity to learn the OOP programming concept with Imikode. Therefore, the game can be helpful within and outside the class and is easy to play. Aside from the Google Cardboard SDK, other technologies used in implementing Imikode include the Unity 3D game Engine and Android SDK. Basic OOP concepts covered in the Imikode VR game include objects, methods (behaviors), setters and getters methods, method arguments, overloading/overriding, inheritance garbage collection for destroying unused objects and lots more. A detailed overview of these OOP concepts implemented in Imikode is presented in (Bouali et al., 2019). The first version of the Imikode VR game was implemented to allow teachers to introduce basic OOP concepts by building a virtual world of creating objects such as trees, houses, and animals through a storytelling approach to provide true immersion and engagement in the virtual space. The second version tackled the drawbacks of the first version by providing real time feedback when errors are made. In addition, it featured an improved artificial intelligence component that allows learners to receive more interpretive and intelligent feedback in terms of errors.

3.1 Operations of Imikode VR Game and the Teaching Method

At the Imikode application's instantiation, the virtual character “Bob” introduces the game environment and its objective to the player. This character guides the player throughout the game play in a conversational manner through a generated text and sound. A talking agent (Bob) teaches the players on what they will be learning and practicing within a particular time frame. For instance, the talking agent could inform the players on the need to learn classes and objects after which the players append the bubbled instructions on the whiteboard to show their level of assimilation. Bob guides the player to create objects in the virtual environment by using the knowledge of variable declaration and assignment statements. Furthermore, the virtual character guides the player in the area of method invocation using the setters and getters functions including method overriding, overloading and the artificial intelligence component that allows learners to receive intelligent and more interpretative feedback in terms of errors. If a player was able to append the write syntax of a portion of the code correctly, the virtual character responds with a smile pointing to the direction of the created object. For a function however, Bob responds by pointing the user to the created task. If the player, however, fails to append the right code on the whiteboard, Bob frowns at such a player and informs the player to repeat the task. For example, two trees and two houses can be built by assigning a value to a variable (tree = 2; house = 2) similar to the conventional programming languages coding style.

In the VR environment however, this is achieved by clicking and dragging each unit of the variable or value (tree = 2; house = 2) which is available on the bubbled list of the Imikode VR game to the whiteboard. To create an instance of an object such as a fox, the bubble command that a player will select and append to the whiteboard is fox = new Fox(). Each time a player selects the instruction correctly, an immediate execution is visible to the player to understand how this OOP concept behaves. If an incorrect instruction is selected, Bob informs the player to retry. The method calls to implement object behaviors are modeled in the Imikode OOP concept. For instance, to cause the fox object to walk, a method called fox.walk() must be selected and appended by the player. When this is done correctly, the player then sees the object fox walking in the simulated environment. If the player selects and appends the method fox.die() in the virtual space, the object fox immediately drops dead and after some time it leaves the space through garbage collection methods. It is worthy to note here that a method call precedes object instantiation and this must be taken into consideration when invoking methods. The detailed procedure of the research is given in Sect. 4.3.

Figure 1 presents a screenshot of a player creating trees, houses, and the instantiation of fox objects as explained above. The left part of Fig. 1 shows how the player creates these objects, while the right screen shows the output of executing each line of instruction.

Screenshots of a player creating objects in the Imikode VR environment

Full size image

More advanced OOP concepts and their applications are shown in Fig. 2. For instance, how to use the getter and setter methods to create variables and assign them values through a function with arguments is depicted in Fig. 2.

Imikode environment of virtual objects being created through OOP concepts

Full size image

3.2 Improved Version of the Imikode VR Game

Diagnostic messages emanating from compilers and interpreters of programming languages help learners understand errors, warnings and other types of messages. Unfortunately, research has shown that compilers' error messages can be unclear to learners, especially novices (Becker et al., 2019). This problem motivated the Imikode VR game's improvement to include an artificial intelligence component that allows learners to receive more interpretive and intelligent feedback in terms of errors (see Fig. 3). Aside from the AI component in the improved Imikode, multiple game play levels can be dynamically added to give learners more opportunities to explore the VR environment for immersive learning experiences. In this improved Imikode VR game, users learn more as a result of more inbuilt functions and OOP concepts such as constructors, instant display of scores and instantaneous error presentation. If an error is committed by the player when instantiating objects or during method invocation, immediate feedback is provided to such players through the artificial intelligence component of the game.

Improved version of Imikode with dynamic levels and artificial intelligence component to better understand error messages generated from the OOP compilers

Full size image

Figure 3 shows a talking agent telling the players what they will learn.

Table 2 presents the objectives and the expected outcome of the Imikode VR game.

The usability of Imikode VR game was evaluated in promoting the learning of introductory programming courses among participants from a Nigerian University (RQ1) and how the USE instrument contributes in ensuring users’ satisfaction (RQ2). The study design comprises mixed method design which was used in order to answer the research questions similar to the methods used by Hariyanto et al. (2020) and Sunday et al. (2022).

In subsequent subsections, the instrument and material were discussed; followed by the study participants and then the research procedure and finally, how the data analysis was conducted.

4.1 Instrumentation and Material

In this research, the USE questionnaire containing 30 items was utilized to collect information about students’ experiences of playing the VR game and its impact in enhancing the learning process of object introductory programming. The USE questionnaire administered in this research is a seven-point Likert scale consisting of four (4) variables: usefulness, ease of use, ease of learning, and satisfaction, as presented in Table 3.

There is also an option “NA” (not applicable) in this questionnaire for items that the students feel is not appropriate. The outline of the questionnaire is illustrated in Appendix I. Aside from the 30 questions; there is an open-ended option for the participants to provide comments regarding their VR game experience. Their comments in this part were used to canvass the collected quantitative data.

4.1.1 Construct Definition and Research Model

LUND (2001) initially proposed a three (3) dimensional construct of the USE questionnaire comprising Usefulness, Satisfaction, and Ease of Use. Later, the study discovered a substantial influence of the Ease of Use scale on the Usefulness scale and these two scales were also found to influence the satisfaction scale. In this subsection, the construct used were defined as follows:

Usefulness This construct shows how useful the Imikode VR game is. In addition, it signifies the extent of trust users have on the Imikiode VR game in enhancing their learning.

Ease of Use The ease of use refers to the effortlessness of playing the Imikode VR game by the students. In order words, it defines how easy it is to play the game.

Ease of Learning The ease of learning is concerned with the level of effort that is being channeled for learning. In order for students to benefit maximally in this learning process from the technology, the amount of effort required should be minimal.

Satisfaction This construct emphasizes how satisfied the students are with the Imikode VR game. Usually, after playing a game or using a product, there is a level of enjoyment one derives.

Dependent Variable This is a variable that depends on other variables’ data. In this study, the satisfaction construct is the dependent variable since it depends on other construct data in order to determine the level of satisfaction the students derive in playing the Imikode VR game.

Independent Variable This variable does not depend on any other variable for result with regards to playing the game. Hence, the Usefulness, Ease of use and Ease of learning constructs serve as the independent variable.

Figure 4 presents a model of the interrelationships among the variables in the USE Questionnaire.

Abstract Model of the interrelationships among the variables in the USE Questionnaire (Hariyanto et al., 2020)

Full size image

4.2 Reliability and Validity of the Instrument

To examine the factor structure of our scale and the structural validity of the scale, both exploratory factor analysis (EFA) were performed (see Table 4) and confirmatory factor analysis (see Figs. 5 and 6). Furthermore and in line with Fornell and Larcker (1981) and Hair et al. (2006), composite reliability (CR) and average variance extracted (AVE) were conducted to further examine the validity of the scale (see Table 4).

Initial factor loading of individual questions and variables

Full size image

Final factor loading of individual questions and variables

Full size image

From this table, we can observe that the KMO is greater than 0.5, which shows that there is no sample problem. Bartlett’s test shows that the test is significant, which implies that the variables are correlated. The initial factor loading is shown in Fig. 5.

In Fig. 5, the questions in the USE questionnaire were utilized to determine whether they would load in their respective factors. The figure depicts the initial factor loading of the individual questions and variables, and it was discovered that some factors were less than the threshold value of 0.7. These factors were dropped because the threshold for every item loading has been reached which is 0.7 (see Table 5) which is the reliability test of Cronbach’s alpha which explains the relationship between latent variables and manifest indicators (Hair et al., 2011; Witjaksono & Saputra, 2019). Table 5 represents the factor loading of constructs and their item measurements.

In this table, all the items loaded under their latent variables conformed to the threshold of 0.7. The final factor loading is shown in Fig. 6.

In Fig. 6, some factors were dropped because of their threshold values. The model quality metrics comprising cronbach’s alpha, effect size, f, composite reliability and average variance extracted were computed and presented in Table 6.

Table 6 shows the quality criteria for latent variables. It can be seen from the table that the Cronbach’s alpha values reached the threshold of 0.7, hence providing an acceptable level of reliability for the research instrument. Furthermore, the rho_A and composite reliability also satisfies the threshold of 0.7, signifying a valid scale for the instrument while the average variance extracted exceeds the benchmark of 0.5 which further validates the scale. The effect size was computed, f, to determine the level of intervention a variable has over another variable (Cohen, 1988). According to Cohen, the effect size f is low if f ≤ 0.2 and it is high if f > 0.5. According to the table, there exists a weak effect between usefulness, ease of use and satisfaction with the threshold (f ≤ 0.2) while there exists a large effect between ease of learning and satisfaction with the threshold (f > 0.50). The implication of this is that the ease at which the students found the Imikode virtual reality game in learning object oriented programming gave them more satisfaction compared to other constructs.

The bar chart depicting the correlation values of the research instrument in Fig. 7 is represented below.

Bar chart showing the correlation values of the USE Instrument

Full size image

In Fig. 7, all the correlation values for each item (Q1-Q30) were presented, and it was discovered that the values exceeded the r-value (0.25) at the level of 0.05 significance. Therefore, each correlated value is considered valid, satisfying the criteria. Hence, this instrument is useful for the evaluation of educational virtual reality games.

4.3 Participants

Participants in this study included undergraduate students in computer science studying introductory programming courses in their first year of studies at Usmanu Danfodiyo University, Department of Computer Science, Sokoto, Nigeria. The students had fundamental knowledge of computer studies from secondary school education. The participants comprising 103 undergraduate students (80 males and 23 females) between the ages of 16–20 years evaluated the VR game using the USE questionnaire (see Appendix I). The participants willingly availed themselves to be recruited for the study in order to experience the power of virtual reality technology in programming education.

4.4 Procedure

The VR application was installed on a mobile device and then connected to a Google cardboard headset which was presented to all participants in this study to learn basic OOP concepts. In the first instance, a 30-min presentation of content using the traditional lecture method was presented to the students on virtual reality technology and basic OOP concepts such as objects, methods (behaviors), setters and getters methods, method arguments, and garbage collection to destroy unused objects. The role of VR in learning, specifically, how to use VR technologies to learn the OOP concept, was also discussed. Furthermore, the students were guided on how to insert the mobile device on the Google headset and launch the installed VR game. During the lecture, a presentation was conducted on how to use the VR headset to walk and navigate various scenes in the simulated environment. Finally, the students were admonished on the need to provide honest feedback when completing the questionnaire. During this participatory and discussion method, teachers and students asked questions and discussed the concept of VR for teaching and learning. Some of the questions from the students included “what will I see in the simulated world?”, “do I keep holding the VR headset while navigating the simulated environment?” “Is the simulated environment different from the real-world environment?” and finally, “what happens if something happens to me in the virtual environment?” The lecturer answered each of the students' questions to their satisfaction. In this question and discussion session, which took about 45 min, the teacher responded to the first question by informing the students that what they would see in the simulated world represents the real world.

For the second question, the lecturer told the students that it is not necessary to keep holding the headset as the Google cardboard headset was designed to fit appropriately to the position the user feels like placing it during the learning process. For the third question, the lecturer's response was that the simulated environment is a representation of the real-world environment and that they are not the same in both theory and practice. Finally, to the last question, the lecturer informed the students that since the virtual environment creates an illusion of the real-world environment, anything that happens to an individual in that environment does not affect such individuals in the real world. The learning process lasted for one (1) week, and each student played the Imikode VR game for 30 min. After each student completed the learning process, the USE questionnaire, which is a paper-based 7-point Likert scale instrument, was immediately administered to the student. Filling the questionnaire by each student, however, took approximately 10 to 15 min. It should be noted that the data were collected using the convenience sampling method because the students under consideration willingly availed themselves to participate in this research. In addition, the students were within the reach of the researchers, implying that the researchers experienced no difficulty bringing them to the fore to participate in the study.

The students were instructed to carefully read the questions and select just one option from seven options for each question that applies to them. The participants answered all 30 questions, including the questionnaire's open-ended questions, and their responses were collected immediately. However, it is worth noting that each response was anonymous and honest based on the actual utilization of the VR game in enhancing their academic performance concerning programming. The procedure of the research was illustrated in Fig. 8 and the different scenes of the learning process in Fig. 9a–d.

The procedure of the research showing the time taken to complete each activity

Full size image

Different scenes a–d of students learning OOP by playing VR game using Google Cardboard Headset in a University classroom environment

Full size image

In Fig. 9a, the students demonstrated the utilization of the VR game inside the classroom; in this case, they wore the Google cardboard headset while holding the controller. As this happens to be the first time such a device is inserted, the students were busy viewing the simulated world and the beautiful environment. In addition, the students were also learning about the various features of the VR game and how to go about playing it.

Two different sets of students played the VR game in the computer laboratory as depicted in Fig. 6b; this time around, the students were busy creating objects and invoking methods to boost their programming skills. However, at this stage of the learning process, the students had already mastered navigating the virtual environment and various VR game features.

In Fig. 9c and d, however, the students used the Google headset to play the VR game directly facing other students in the classroom. Thus, showing all the students in the class how to put on the Google Cardboard headset, how to navigate in the virtual world, and finally, how to move in the simulated environment.

4.5 Data Analysis

IBM SPSS statistics version 21 (IBM Corp., 2012) and Smart PLS applications for Windows operating systems were used for analyzing the quantitative data. The gleaned data was coded according to the construct categories into the SPSS and Smart PLS applications. In a bid to foster high quality data, three of the co-authors checked the data in their various computers in order to ensure that the integrity and consistency of the data is realized across platforms. Descriptive statistics, Analysis of variance (ANOVA) and multiple linear regression analysis were employed including the f & t test investigations on the data using these tools. The students’ responses were analyzed on the feedback form of the USE Instrument based on qualitative content analysis and the procedure outlined by Mayring (2014) were followed on content structuring and thematic analysis. In this case, the open-ended feedback forms were read on several instances in order to understand and classify each student’s responses based on some themes and in line with the research questions. Furthermore, duplicity of themes among the feedback forms were realized and were categorized according to the usefulness, ease of use, ease of learning & satisfaction constructs respectively. Some responses which did not fall nicely into these constructs were realized because it seems more of a recommendation and or drawbacks; hence, it was profiled and classified such responses under the recommendation and drawbacks theme of the VR tool. It was discovered from the analysis that out of the 103 students that responded to the open-ended feedback form, 10 students either forgot or deliberately decided not to answer the open ended questions in the USE questionnaire. The 93 remaining students offered various comments ranging from some positive comments (88) to providing a recommendation (1) while others provided some drawbacks of the technology (4). Finally, these themes were carefully selected and then segmented them into separate units (see Table 9).

5.1 How do the Students Find the Imikode Educational Virtual Reality Game Usable for Learning?

Before analyzing the students’ responses to the questionnaire using descriptive statistics, a usability measurement score was carried out for Imikode educational virtual reality game. This measurement score represents the degree of acceptance or rejection of the Imikode VR game by the users hence, in this study each variable’s mean value was utilized to describe the usability measurement results, as suggested by Nielsen (1994). Also, each variable's mean score (see Table 7) was utilized by taking the average of the scores and grouping them according to each variable representation on a seven-point Likert scale. However, the direction of responses on this scale shows the level of acceptance or rejection, as put forward by Babbitt and Nystrom (1989). In this case, it was observed that the mean of each variable was within the acceptable range since no value was below 5.0, hence, each value draws closer to the peak of being firmly accepted. This means that as each mean score goes higher after satisfying the acceptance threshold of 5.0, the acceptance rate further increases (see Table 7). In addition, and similar to the work of Marreez et al. (2013) the seven-point Likert rating score were converted to two categories of “binomial data'' of acceptance and rejection according to the participants' responses of agree or disagree, where a score of 5 (somewhat agree), 6 (agree) and 7 (strongly agree) were grouped under the acceptance category. This was similarly done for the reject category, where score ratings of 1 (strongly disagree), 2 (disagree) and 3 (somewhat disagree) were merged and classified into the rejection category. However, the rating score for neutral was excluded from the analysis because it denotes uncertainty or neutrality. The reason the Likert score was converted and harmonized to 0–100 scores was to enable more analysis of the data from a binomial perspective where 0–49 indicates below average and 50–100 indicates acceptable value (Debevc & Bele, 2008). From Table 7, it can be observed that the trends of the constructs as the usefulness, ease of use, ease of learning and satisfaction constructs all exceeded the score value of 50, thus satisfying the criteria of being accepted.

Furthermore, when taken a cursory look at the average score of the four variables the score is 90.94, exceeding 50. Based on these results, it was concluded that the students found the Imikode VR educational game to be very useful for learning object oriented programming. Similarly, from the 0 to 100 score of the USE questionnaire in Table 7, it was discovered that 90.94 percent of the users were satisfied with its use as a means of boosting their programming skills. However, the implication of this is that, for instance, when there is a population of 100 students in the class, over 90 percent of students who play the Imikode VR educational game will be satisfied with it as an excellent means of learning programming. We present the model construct in Fig. 10 which shows the degree of usability of the Imikode VR game.

Construct model of the USE Instrument showing the degree of usability

Full size image

Furthermore, with regards to the distribution of the scores as illustrated in Table 8, it was observed that the scores were skewed negatively for all the variables, implying that the left tail of the distribution is longer relative to the right tail, and thus not symmetric. In addition, none of the students responded with N/A. However, this could be explained because the students were guided correctly on the different aspects of the USE questionnaire, including learning with the Imikode VR game.

5.1.1 Student’s Open-Ended Feedback

User feedback as suggested by Hattie and Timperley (2007) involves providing information about the user's level of performance or understanding of a particular tool which is capable of guiding future development. In this study and in addition to quantitative responses, the students provided their responses in the form of an open-ended question in the quantitative feedback form. These questions enable students to provide comments or suggestions on what they feel based on their VR game experience. Of the 103 students who participated in this study, 10 students failed to provide feedback on the VR game and the remaining 93 students provided both positive feedback and some vital recommendations for improving the VR learning system including some drawbacks of the technology. The feedback is classified under the variables represented in Table 9.

It is clear from the above comments that the students enjoyed playing the VR game and it has come as a big booster in enhancing their knowledge of object-oriented programming despite the drawbacks of the technology. It is interesting to note that all the student’s responses were not positive as seen in Table 9, where only one student provided a recommendation and very few students provided some drawbacks of the technology.

5.2 How Does Ease of Use, Ease of Learning and Usefulness Attributes Contribute to User Satisfaction with Imikode?

This section explores multiple linear regression analysis assumptions: testing for normality test, checking for multicollinearity, and testing heteroscedasticity. Whenever multiple linear regression is used, it is essential to check if the assumption is satisfied. This study used multiple linear regression to examine the relationship between ease of use, satisfaction, ease of learning, and Imikode educational virtual reality game usefulness.

5.2.1 Requirements for Multiple Linear Regression to Understand Attributes of the Usability of the Imikode Educational Virtual Reality Game Through the USE Questionnaire

The prerequisites for conducting this analysis were derived from the classical assumption model of the tested variables (Hair et al., 2009). Accordingly, conducting this test is necessary to make the result worthy of trust and enable other researchers to have more confidence. These tests include the normality test, multicollinearity test, and heteroscedasticity test. Hence, it is necessary to ensure that the variables used should be normally distributed and devoid of multicollinearity and heteroscedasticity, respectively. In subsequent sections, how the classical assumption model for the tested variables is achieved was shown.

5.2.2 Test for Multivariate Normality of the USE Questionnaire

In multiple linear regression, the first assumption among the classical assumptions requires that we perform a normality test on the specified variables to check whether the variables are distributed normally. This can be achieved by closely examining the histogram and data plots. In this research, a cursory look at Fig. 11 shows that the graph is approximately bell-shaped and symmetric about the mean. Most of the data values revolve around the mean, signifying the presence of a normal distribution.

Histogram for normality test

Full size image

In Fig. 12, the data plots show a slight deviation from the line of least squares fit bars, which indicates a conventional normal distribution.

5.2.3 Test for Multicollinearity of the USE Questionnaire

The second assumption in the multiple regression model is that there is no multicollinearity in the data. Data are deemed free of multicollinearity if and only if each independent variable possesses a Tolerance (T) that is higher than 0.1 and also has a Variance Inflation Factor (VIF) that is lower than 10 (Kim, 2019). In this study, however, it can be seen from Table 10 that the VIF for usefulness (1.918), ease of use (2.489), and ease of learning (2.481) were lower than 10. Similarly, the T for usefulness (0.521), ease of use (0.402), and ease of learning (0.403) were greater than 0.1. Hence, we conclude that these variables are devoid of multicollinearity, implying that the independent variables are truly independent.

Data plot for normality test

Full size image

Table 11 presents the model summary, which shows the variability of the students' satisfaction using the VR game as a tool to learn programming, explained by the usefulness, ease of use, and ease of learning variables. Model summaries show how the dependent variable is used to measure the closeness of the data to the fitted regression line. In Table 11, R² indicates that 70.1% variability in students’ satisfaction is explained by the ease of use, ease of learning and usefulness variables.

5.2.4 Test for Heteroscedasticity of the USE Questionnaire

Finally, the heteroscedasticity test was performed to investigate whether the data satisfy homoscedasticity, which is the last assumption in the multiple linear regression. In this case, however, it was assumed that there is no heteroscedasticity in the data. The best way to check this is through a scatter plot diagram where the standardized residual is regressed onto the standardized predicted value. It should be observed that when homoscedasticity is present or when the assumption cannot be rejected because there will be no pattern in the data. In Fig. 13, it can be seen that the data are closely packed on one end and then start fanning out as you progress to the other end, representing a typical conventional reflection of homoscedasticity in the regression model.

Scatter plot diagram of homoscedasticity

Full size image

5.3 Multiple Linear Regression to Determine the Relationship Among the Attributes of the USE Questionnaire

Multiple linear regression was conducted to show the linear relationship between students’ satisfaction with ease of use, ease of learning, and usefulness of educational VR games. Hariyanto et al. (2020) stated that the purpose of using multiple linear regression analysis is to understand the nexus between two or more independent variables and one dependent variable. Hence, in this research, the f-test distribution was explored to understand the ease of use, ease of learning, and usefulness variables. Likewise, the partial t-test distribution was conducted to establish each of the independent variables' contributions to the students' satisfaction using VR tool for learning.

5.4 F-test Investigation to Determine the Influence of the Attributes of the USE Questionnaire

The F-test is used to show the contributions of the ease of use, ease of learning and usefulness attributes of the Imikode educational VR game on students' satisfaction. Analysis of variance (ANOVA) were also carried out to determine the effect of these attributes of the VR educational tool on student satisfaction. From Table 12, the F-test showed that the variables were statistically significant (F = 77.508, p < 0.05). Usefulness, ease of use, and ease of learning affect students' satisfaction using the VR tool for educational purposes.

5.5 Partial t-test to Understand Whether there is a Partial Influence on the Attributes of the USE Questionnaire

To understand whether there is a partial influence on the variables, a t-test investigation was conducted. Table 13 presents the partial influence of these variables on students' satisfaction using the VR tool for educational purposes. From Table 13, the t-test result showed a significant influence (t = 2.720, p < 0.05) between the usefulness and satisfaction of the students using the VR tool for educational purposes. The students found the Imikode educational VR game very useful, which made them satisfied anytime the game was played.

Furthermore, the t-test showed no significant influence (t = 1.100, p > 0.05) between ease of use and satisfaction of the students using the VR tool for educational purposes. This means that the ease of use of the Imikode VR game has no significant influence on students' satisfaction using the VR tool for educational purposes. Lastly, the t-test showed a significant influence (t = 7.072, p < 0.05) between ease of learning and students' satisfaction using the VR tool for educational purposes. This implies that the students were satisfied with the ease with which they used the Imikode VR tool to learn OOP.

In this study, the goal is to evaluate the usability of the VR game for learning object-oriented programming and to further validate the USE instrument to ascertain the correlation of the various constructs and determine its suitability for use by other researchers. Furthermore, some design considerations on programming tools were recommended.

The developed VR Game was presented to one hundred and three (103) computer science students in the first year of their studies who are currently studying introductory programming courses in the north-west region of a Nigerian university. In the subsequent sections, the outcomes of this study were discussed based on the research questions presented in the introduction section.

6.1 How do the Students Find the Imikode Educational Virtual Reality Game Usable for Learning?

To provide a solution to the first research question, reliability and validity analyses were conducted on the instrument used as a prerequisite for further analysis. The students interacted with the Imikode VR educational virtual reality game using the Google cardboard headset in classroom settings to learn object oriented programming. This is evident from the analyzed result of the USE questionnaire, as the usability measurement score which was converted to binomial data and harmonized to the 0–100 score in order to view the data from another dimension suggests that over 90 percent of the students accepted Imikode VR as another way of enhancing their coding skills.

In addition, from the open-ended feedback gathered from the participants, it was observed that more than 80% of the students alluded to the fact that the VR game enhanced their understanding of OOP which will in turn, lead to increased performance in tests and exams (see Table 9). This further shows that almost all the study participants confessed how usable the Imikode VR game is in enhancing their OOP skills. However, very few students provided some drawbacks while only one gave a recommendation of the VR technology based on their experiences with the virtual reality environment. The overall results of this analysis show that the instrument used in this study satisfied the requirements. Furthermore, it shows that the usability study using this instrument could further be used in subsequent studies as the factor loading, Cronbach’s alpha, composite reliability, effect size, average variance extracted and confirmatory test in Smart PLS using the factor algorithm including a bar chart of the correlation values that was used (see Fig. 7) all satisfied the minimum threshold of been accepted.

6.2 How does Ease of Use, Ease of Learning and Usefulness Contribute to the User’s Satisfaction of Imikode?

To check the correlation among the usability questionnaire attributes, multiple regression analysis was performed to show the relationship between the usefulness, ease of use, ease of learning, and satisfaction of the students using the VR tool for educational purposes. To achieve this, the multiple linear regression requirements derived from the classical assumption model of the tested variables were firstly analyzed. In this model, some tests were performed to ensure that the variables used were i. normally distributed ii. free from multicollinearity and iii. devoid of heteroscedasticity. This is in line with Hair et al. (2009), as conducting this test will lead to more trust in our results. A check if the data were normally distributed were confirmed by using a histogram and data plot and discovered that it satisfies the first assumption. A further check was carried out if the data were free of multicollinearity; the results also showed that the minimum benchmark was satisfied. To satisfy the third assumption, a test for heteroscedasticity with the aid of a scatter plot diagram was performed and it was discovered that it satisfies homoscedasticity as there was no pattern in the data.

Having satisfied the prerequisites of multiple linear regression, the F-test and t-test analyses were conducted. The F-test results show that usefulness, ease of use, and ease of learning affect students' satisfaction using the VR tool for educational purposes (F = 77.508, p < 0.05). There was a considerable correlation between the variables in the USE questionnaire. However, the t-test result shows that the students' satisfaction with the VR tool for educational purposes was influenced by both the usefulness and ease of learning variables (t = 7.072, p < 0.05; t = 2.720, p < 0.05). However, ease of use did not significantly influence students' satisfaction using the VR tool for educational purposes (t = 7.072, p < 0.05).

It should be noted however, that the normality test and homoscedasticity test focused on the satisfaction variable because they represent the dependent variable the authors presented earlier as outlined in the USE questionnaire (see Appendix I). ANOVA and t-test were conducted to determine the influence of the variables of the use questionnaire. The results of the t-test investigation showed no significant influence between the ease of use of the Imkode VR educational tool and students' satisfaction. However, the usefulness, ease of use, and ease of learning in tripartite were combined and analyzed concurrently using the F-test investigation and it was ascertained that there was a significant degree of influence on students' satisfaction.

Despite the huge successes recorded from the utilization of the VR game in improving the coding skills of students and the subsequent validation of the research instrument for use by other researchers, there are still some limitations with regard to its use and some future directions were proposed. Even though there were a reasonable number of participants for this study, further research in this area to involve more participants from other educational settings across institutions in Nigeria should be conducted so as to determine the learning effectiveness of students across those institutions. Due to the general lack of awareness regarding the utilization of VR technology for aiding teaching and learning programming in Nigeria, it was advocated that a series of publicity awareness should be introduced across campuses on the need to engage VR technology for programming education.

While appreciating the considerable impact this technology has achieved in improving students’ understanding of object oriented programming as attested by the students, there is also the need to enhance technological inadequacies. This is because acquiring the educational VR headset seems to be a daunting task, especially for students in Usmanu Danfodiyo University where this research was conducted, as shown in Fig. 6. In this scenario, only two (2) headsets were available for use by students. However, instead of spending less time on the experiment, the researchers ended up spending one (1) week for the entire one hundred and three (103) computing students in playing the game.

This study focused on the usability evaluation of VR games for learning object oriented programming and did not consider VR headset technology including mobile devices and their various features, all of which was believed will have an effect on the learning process. Another challenging factor was the difficulty in acquiring the VR headset in the region where the study was conducted coupled with the lack of technical support for users of the VR game. As the evaluation results of this study shows a general satisfaction of the VR game and its potential in improving the programming skills of students, future work in this regard will include the refinement of the VR game to include more sophisticated OOP concepts such as polymorphism, recursion, inheritance, interface, applets, and data structures. Furthermore, future research in this area should capture the analysis of AI components in the VR technology.

In the first instance, Imikode—a VR game for teaching object-oriented programming were presented. This study could be attributed to the difficulty students face in understanding OOP concepts and the need to factor in students’ consideration in building efficient and effective systems that will enhance the learning process. The research conducted by Pantelidis (2009) highlighted the significant contribution of using VR in education. It was concluded that children who are on the verge of failure would likely succeed in utilizing virtual reality systems. In addition, the use of VR tends to boost learning by creating objects in the virtual world and through the depersonalization of abstract ideas that have further defied representation.

The study VR game is a technology that facilitates the learning of object-oriented programming, especially the aspect where students learn how to create a new object, invoke methods, create constructors, overload methods, and override methods all taking place in a simulated environment and receiving real time feedback on errors made. The stages of learning OOP in the VR, which was configured to be from simple to complex at each stage, will further enable students to grasp the intrinsic properties of OOP. The feedback from students’ experiences of using the VR game has further provided great insight into the suitability of using VR technology to enhance the programming skills of students at various levels and the need for further design of programming tools. Despite the benefits of playing the Imikode VR game in learning object oriented programming; we believe a lot of work still needs to be done by software developers in building software tools that will facilitate the learning of object oriented programming paradigms. For example, one of the students’ open-ended feedback stated that beholding the sight of an object’s appearance after appending the correct code syntax on the white board for object creation and instantiation marveled him and enabled him to become more interested in programming. As a follow up with this; another student was thrilled when he used the object ‘’fox’’ to invoke a method ‘walk’ and discovered the movement of this object in the virtual space. Can these aspects be replicated in subsequent design of programming tools? It is believed that integrating these aspects and lots more will go a long way in improving the programming skills of students across all levels and not just the usual cramming of code syntax just to pass examinations.