Towards Intelligent Computer Assisted Educational Role-Play Mei Yii Lim1 , Ruth Aylett1 , Sibylle Enz2 , Michael Kriegel1 , Natalie Vannini3 , Lynne Hall4 , and Susan Jones4 1 School of Mathematical and Computer Sciences, Heriot Watt University, Edinburgh, EH14 4AS, Scotland {myl, ruth, michael}@macs.hw.ac.uk 2 Otto-Friedrich-Universitaet Bamberg, Kapuzinerstrasse 16, D-96045 Bamberg, Germany sibylle.enz@unibamberg.de 3 Universitaet Wuerzburg Lehrstuhl für Psychologie IV Roentgenring 10, D-97070 Wuerzburg natalie.vannini@psychologie.uni-wuerzburg.de 4 School of Computing and Technology, University of Sunderland {lynne.hall, susan.jones}@sunderland.ac.uk Abstract. This paper investigates how graphically displayed intelligent virtual actors, mobile devices and innovative interaction modalities can support and enhance educational role-play as well as deepen the sense of engagement and presence in participants to produce more successful learning. The discussion will be presented using a showcase from the eCIRCUS project, ORIENT, an application combining virtual and real life role-play for social and emotional learning. Keywords: Educational role-play, social and emotional learning, intelligent virtual actors, innovative interaction modalities 1 Introduction Role-play (RP) is a vital tool across a wide range of education and training scenarios, from adults in organisational or therapeutic contexts to children in schools. RP has been proposed as an engaging way to help solve inter-personal problems and increase team cohesion (eg. [1, 2]). It covers several aspects of Bandura’s Social Learning Theory [3], for example, in emphasizing the difference between the acquisition and the performance of behaviour as well as the importance of providing secure settings for the experimentation with new behavioural strategies. Role-play supports the creation of knowledge and meaning from concrete - though imagined - experiences (“prehearsal”, [4]) and uses social interaction and emotional engagement as mechanisms for a learner-centred constructivist approach. It can be used for organisation-based learning where more conventional eLearning approaches are not effective given that the social setting may be a crucial factor to the success of the learning process. If RP is embedded in an educational setting and is guided by a facilitator, the testing of new behavioural strategies is immediately followed by feedback from the social environment (the facilitator and the other players) serving as a source of information for the role-player about the appropriateness or suitability of their actions. Social interaction is used as the stimulus for challenging and changing existing beliefs and can result in significant behavioural changes because it is easier to empathise with another person under certain circumstances if one has experienced something similar. Therefore, through RP, empathy can be evoked when an individual attempts to understand another by imagining the other’s perspective. It consequently influences the subjective evaluation of a person regarding his or her social skills and accordant self-efficacy. The educational impact of role-play draws on the ‘willing suspension of disbelief’ by participants who commit to the roles they have been asked to play. Producing this feeling of social presence in the role-play world involves willingness to participate, the production of supporting materials, skilled facilitation and often the use of extra supporting actors. None of this is easy. For these reasons, the use of virtual worlds, providing opportunities for exploration, practice, experiments and construction, seem a desirable way of augmenting traditional learning tools. These allow for experiential learning in safe and secure learning environments, in particular for social and emotional learning in complex social situations without the risks that the learner faces in a real social situation. This paper presents an attempt to take advantage of the aforementioned benefits of role-play in cultural awareness education. 2 State-of-the-art The obvious example of technology-based role-play is of course massively multiplayer online games such as World of Warcraft and Everquest. These are however entirely computer based and do not take advantage of new generations of interaction technology allowing the role-play to take place in the real rather than the virtual world. Though strongly social in nature, the rich human interaction modalities of voice, face, body are severely limited by the impoverished repertoire of graphical avatars, the use of keyboard and mouse as the interaction mechanism, and the lack of geographical co-location. One of the earliest attempts to deal with these issues was the so-called Alternate Reality Game, Majestic, launched by Electronic Arts in 2001. It aimed at blurring the lines between fact and fiction, and players were given clues via mysterious midnight phone calls, anonymous e-mails and faxes, and fake websites. It failed as a game, but subsequently a distinct genre of pervasive games both for entertainment and education has come into being, often drawing on the wide availability of mobile devices. Educational examples include Virus [5], Paranoia syndrome [6], Virtual Savannah [7] and Environmental Detective [8]. A challenge for a technology-supported role-play lies in the role usually played by the human facilitator (often referred to as game master) in shaping the narrative experience and the pedagogical outcomes, which usually depends on postrole-play reflection or debriefing. Without feedback prompting reflection on the role-play, the transfer to real-world settings cannot be ensured. Current work in Interactive Narrative divides into ‘plot-based approaches, in which events are pre-authored, and ‘character-based approaches, in which narrative structure is generated by interaction - emergent narrative [9]. An example of work that adopted the ‘character-based’ approach is the software FearNot! [10]. In FearNot!, characters run the FAtiMA [11] architecture, including a generative planner. Plot-centred approaches may also involve generative planning [12] but risk conflict between pre-determined outcomes and character freedom-to-act, problematic in role-play where human actors require interactional freedom with each other and the virtual actors. Our software showcase for the eCircus project, ORIENT addresses issues mentioned above and takes a first step towards intelligent computer assisted educational role play. In the following sections we are going to discuss this project in detail. 3 Role-playing in ORIENT The motivation for creating ORIENT was to transfer the empathic learning concepts applied in FearNOT! for anti-bullying education to the domain of intercultural empathy. ORIENT (Overcoming Refugee Integration with Empathic Novel Technology) provides a RP and story framework for virtual actors that establish credible and empathic relations with the learners. It aims to develop models within which narrative engagement and empathy can be used to understand social, cognitive and emotional learning processes through RP. Its focus is on integration of refugee/immigrant children in schools. It attempts to create an environment to support affective engagement in social and emotional learning by investigating methods that allow individuals and groups to interact in physical as well as the virtual world, with user roles including both spectator and actor. Although there has been plentiful research on empathy, there is limited knowledge and understanding of the role of empathy for technology-enhanced learning processes. This problem is exacerbated by a lack of holistic models for the social and emotional process that consider intrapersonal (cognitive, affective), interpersonal (role-play), and situational variables that are at the basis of learning processes. ORIENT tries to address this issue. Its innovative approach offers the possibility to explore physiological, psychological and cognitive aspects of social and emotional learning provided through interactive educational RP to reach an improved understanding of learning processes on an intrapersonal as well as an interpersonal level. 3.1 The Game ORIENT is designed to be played by a group of 3 teenage users between the age of 13 and 14, each a member of a spaceship crew. Their mission takes them to a small planet called ORIENT, which is inhabited by an alien race, the nature loving Sprytes. These are autonomous affective characters with individual personalities able to express emotions. The Sprytes culture has been defined based on a subset of Hofstede dimensions [13]. The dimensions of cultural variability according to Hofstede individualism-collectivism; uncertainty avoidance; power distance; and masculinityfemininity - exist in all cultures at the cultural and individual level and one characteristic within each dimension tends to predominate. Individualism pertains to societies in which the ties between individuals are loose while collectivism pertains to societies in which people from birth onwards are integrated into strong cohesive ingroups. The dimension of uncertainty avoidance deals with the degree, to which members of a culture try to avoid uncertainty. The power distance is the extent to which the less powerful members of institutions and organisations accept that power is distributed unequally. Lastly, masculine and feminine cultures are particularly differentiated through gender role distribution. The Sprytes are collectivistic, being largely compassionate with each other and live in a group where the majority holds power. They are hierarchically organised (high power distance) hence believe in using force to influence others and they take uncertainty as threat (high uncertainty avoidance). Gender is absent in the culture and their graphical representation as can be observed in Figure 1(a) and Figure 1(b). Additionally, they are gestural creatures where gestures form an important mean of communication as shown in Figure 1(b) where one of the Spryte is scolding the users while the other is expressing its anger through a gesture, unique to the Spryte culture. Fig. 1. (a)A Spryte explaining their life cycle; (b)Angry gesture to the user for stepping on a little tree According to Baylor and Ryu [14], a social relationship between learners and agent builds a key mechanism to foster interaction and promote learning within a computer-based learning system. The users task in ORIENT is to prevent a catastrophe in the form of a meteorite that is on destruction course with the planet. During the mission, the users will have a chance to witness the Sprytes eating habits, life cycles - recycling the dead (Figure 1(a)), educational styles, family formation and value system - trees are sacred (Figure 1(b)). This general story framework allows the users to appreciate cultural differences by trying to integrate themselves into an alien culture in order to gain the aliens’ trust and eventually work together with them to save the planet. They can achieve this by empathising and exhibiting acceptable social conduct at different stages of the interaction to help and avoid enraging the Sprytes. Once a positive relationship is established between agents and users, motivation and interest are enhanced, hence constructive learning is possible [15]. In addition to the cultural-awareness function, ORIENT can be seen as a team building exercise where users play as a single entity rather than as individuals, distinguishing the application from other role-playing games. Although each user has a different role, these roles are closely related and all users have the same goal in the game. The main purpose of communication is to engage in social interaction as opposed to accomplish a task as efficiently as possible. Attitudes, emotions and behavioural reactions are prompted through the Sprytes that are reflected upon and discussed in the team of learners that collaboratively try to meet a common goal through understanding and empathising with members of this other culture. The virtual world setting is of crucial importance here: it encourages the experimentation with different behavioural strategies and thus leads to the exploration of other experiences, behaviour and ways of thinking. The setting presents a possibility for the users to experience encounters and relations with members of another culture and engage affectively. The users can learn in safe and secure environments without the risks that they might face in real complex social situations, allowing them to experience the character’s emotions and problems in a distanced way, while being at the same time engaged in what happens to the characters. Running real life role-plays can be very expensive with respect to manpower and time for developing appropriate scenarios, for executing and facilitating them, analysing them and providing proper feedback for the learner. The use of virtual actors in ORIENT both reduce the expense and complication of organising role-play where real actors might otherwise have to have been used, and help to reinforce ‘in-role’ behaviour in learners by supporting the believability of the role-play world. 3.2 The ORACLE Moreover, ORIENT includes the ORACLE (Onboard Resource Agent for Cultural and Liaison Engagements 2), an embodied computer character running on a mobile phone device that plays the role of a human facilitator. The ORACLE performs its pedagogical function by asking appropriate questions and providing help during the mission to motivate and keep the users engaged. It stimulates users’ reflection on events and experiences in ORIENT by commenting on users’ actions and by encouraging the recording of both personal and collective diaries. It can interact with the users reactively (users ask a question and ORACLE replies) or proactively (intervenes when necessary to help the users fulfil a task). When the mission is completed, the ORACLE carries out a “debriefing” session with the user where they report on the information collected during the mission, allowing transfer of acquired experiences to real world situations. 4 4.1 Technology System Components Fig. 2. ORIENT system components Figure 2 gives an overview of the main components that constitute the ORIENT system. At the core of the system is a world model that stores various elements of the virtual world using logical representations. The virtual world is presented to the user as 3D graphics rendered in the Ogre3D graphics engine. The users can interact with the world through a variety of tangible interaction devices (see section 4.2). The characters inhabiting this world are represented by autonomous agents, each of whose cognitive processes are simulated by a separate agent mind program that receives perceptions from and sends actions for execution to the global world model. These characters have autobiographical memory, individual personalities, a model of the Spryte culture, and improvisational capabilities to enable social and emotional learning through RP and affective engagement in a complex social environment. The affective mind is being built upon FAtiMA [11] combined with the PSI [16] model. Affective models are seen here as a key component of virtual actors given the need to affectively engage participants and spectators in the dramatic environment. For more information on the agent mind, please refer to [17]. A language engine transforms speech-acts into utterances and subsequently a speech synthesizer creates gibberish speech that matches the utterance. Similar to the agent mind, the ORACLE perceives events in the world and responds to them by displaying feedback and help to the users. 4.2 Interaction Modalities Empathic engagement can be enhanced through the use of innovative interaction devices to provide children with more intuitive interfaces to express affect. An empirical study showed that sensor-equipped toys, for example, SenToy [18], provide an affective means of self expression which is an essential requirement for successful RP. The work by Fails and collegues [19] who investigated different versions of the Hazard Room Game that contains elements of RP and interactive story telling indicated that interacting in a physical interactive environment may increase the learner’s interest and understanding compared to a traditional desktop-based version. Additionally, Dow and colleagues [20] found that Augmented Reality contributed to an enhanced sense of presence as opposed to a desktop-based or speech-based interface. Another issue of concern in ORIENT or any other educational RP applications is how interfaces can help to foster social interaction between learners. Inkpen and colleagues [21] observed that by giving each learner an input device, a positive effect on collaboration when solving a puzzle can result even if only one learner could interact at a time. A study by Mandryk and colleagues [22] revealed that learners preferred to play the game with friends than by themselves and that the learners spent a great deal of time interacting with each other. Another study exploring different multi-user settings for pervasive games [23] revealed that a positive effect on collaboration in RP can be achieved through an appropriate distribution of interaction devices. Clear evidence was observed that organised interaction within a group is possible through a setting where each user was assigned a role via an interaction device with a dedicated function, hence, balancing the level of interactivity and avoiding dominant users. Overall, there is empirical evidence that learners seem to be more engaged and more active when playing on a computer with multiple input devices than when using a computer by themselves. Fig. 3. Interactions: (a)performing a gesture using the WiiMote, (b)navigating using the Dance Mat and (c)RFID scanning Based on these studies, ORIENT’s user interface was designed to be physical and tangible as shown in Figure 3. Full body interaction and movement in the physical space, particularly important in social behaviour and culturally specific interaction are supported. Each user is assigned a role which relates to a specific interaction device that has unique functions. These devices are a mobile phone, a Dance Mat and a WiiMote. All these devices are necessary to achieve the overall goal of the game. Bluetooth communication is utilised for both the mobile phone and the WiiMote while the Dance Mat is connected to the computer through USB. The interaction techniques supported by the mobile phone, a Nokia NFC 6131 include RFID-based input and speech recognition. RFID tags are embedded in real world objects that also have existence in the virtual world. The user who is assigned to use the mobile phone selects objects by touching them and uses a “magic word” (character’s name) to grab a character’s attention. In response, the Sprytes speak an alien gibberish language, which is generated on the fly by a speech synthesizer and appears as translated subtitles on the screen. Navigation in the virtual world is achieved through the Dance Mat which is operated by a second user. The user can move forward, backward and turn left or right allowing them to explore the virtual world. The WiiMote is used for three-dimensional gesture recognition based on motion data derived from accelerometer sensors. As aforementioned, the Sprytes are gestural creatures, thus, the third user performs different gestures for expressing communicative content. With interaction supported through large and micro screens, physical interfaces and multi-modal interaction devices, we hope the users’ motivation to learn about the Sprytes’ culture, their engagement in the interaction and collaboration between them can be enhanced. 5 Evaluation The evaluation of ORIENT was designed as an in-role experience for adolescent users in UK and Germany: Four groups of three adolescents in each country interacted with ORIENT. Participants were given the roles of applicants to a mission of Space Command with the aim of exploring the alien planet of ORIENT, while the evaluation team played the roles of members of Space Command. One session comprised a duration of approximately two hours. The evaluation of ORIENT focused on two inter-related themes: participants’ inter-cultural awareness and behaviour; and their immersion and engagement in interactions with ORIENT. The pedagogical and psychological evaluation aimed to investigate the effectiveness of ORIENT in fostering cross-cultural acceptance through the promotion of collaborative practices and the appreciation of similarities and differences between cultures. In addition to the psychological and pedagogical aims of evaluating ORIENT, our aim was also to evaluate the technology, focusing on intelligent computer assisted RP and our approach to accessing it using unusual interaction devices. Table 1 details the constructs that we aimed to measure and the approach taken to achieve this. Table 1. Constructs and Corresponding Instruments CONSTRUCTS INSTRUMENT/MEASUREMENT APPROACH Demographic characteristics & Cultural Participant Questionnaire Profile Cultural Intelligence Cultural Intelligence Scale [24] Perception & expectations of game play Qualitative/open instrument to assess preinteraction views Knowledge about the Sprytes/ORIENT Story/Task Comprehension exercise Outgroup/cultural view (with regards to Cultural Activities Questionnaire (amalgathe outgroup ”Sprytes”) mating the Intergroup Anxiety Scale [25] & the General Evaluation Scale [26]) Device Use Usability evaluation questionnaire with closed questions/discussion Response to ORIENT & Sprytes Based on the Character Evaluation Questionnaire [27] and by interaction observation Feedback on interactions with ORIENT Questionnaires, discussion groups, creation of (e.g. graphics, speech, storyline, agent be- diaries / storybooks lievability, etc.) The main findings from the evaluation were that: – Overall participants rated the prototype positively and readily engaged with it and with one another, with interactions indicating that this approach has the potential to foster cooperation among the user group. However, the – – – – prototype needs additional content and structure to make the encounter with the Spryte culture a pedagogically meaningful experience. Participants were able to identify similarities and differences between their own and the culture of the Sprytes. The Sprytes and their culture were considered to be engaging, the differences appeared to be believable and credible with participants interested in participating in Spryte activities. Sprytes have no obvious individuality at present but appear as cultural stereotypes thus reducing the potential of the formation of empathic relationships between users and Sprytes. This lack of individuality also reduces the coherence of the storyline and the believability of the characters. Differences were seen between the German and UK samples. Although UK teenagers had an increased awareness of intercultural norms and experiences, they were more negative than German teenagers towards the Sprytes. UK participants felt that the Sprytes considered them to be enemies, whilst German teenagers felt that the Sprytes thought they were friends. Participants found the interaction devices and approach challenging but engaging. However, the effort and challenges of such interactions frequently absorbed more user time than the Sprytes and ORIENT did, with an inappropriate focus on devices rather than interaction. In order to enable the users to explore, understand, and flexibly react to the cultural encounter with and the specific problems of the Spryte culture, the interaction with the virtual world should be more intuitive and seamless as is currently the case. These findings highlight the potential of ORIENT to foster cooperation amongst the participants and that participants could readily identify differences and similarities between the virtual culture of ORIENT and the participants’ own culture. However, although participants could appreciate the distinct culture of the Sprytes they were unable to distinguish between the individual Sprytes themselves. This provides the challenge of making the agents individual and discrete personalities and behaviours within a coherent culture more obvious. The user experience would also be improved through making the interaction more intuitive and easier to control, and by adding content to the story in order to provide more room for exploration. 6 Conclusion ORIENT provides a novel approach to social learning through RP by integrating pedagogical self-learning and learning in a collaborative environment providing individuals with new ways to acquire, contribute and exploit knowledge and thereby learn. It presents a model of narrative engagement and empathy. Novel interaction modalities have also been applied to improve user engagement. Some evidence has been observed from the initial evaluation that ORIENT enhances collaboration among learners and learners are able to see a distinct culture in the Sprytes. Acknowledgements This work was partially supported by European Commission (EC) and is currently funded by the eCIRCUS project IST-4-027656-STP with university partners Heriot-Watt, Hertfordshire, Sunderland, Warwick, Bamberg, Augsburg, Wuerzburg plus INESC-ID and Interagens. The authors are solely responsible for the content of this publication. It does not represent the opinion of the EC, and the EC is not responsible for any use that might be made of data appearing therein. References [1] Marsh, D.T., Serafica, F.C., Barenboim, C.: Effect of perspective-taking training on interpersonal problem solving. Child Development 51(1) (1980) 140–145 [2] Lane, C., Rollnick, S.: The use of simulated patients and role-play in communication skills training: A review of the literature to august 2005. Patient Education and Counseling 67 (2007) 13–20 [3] Bandura, A.: Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, N.J.: Prentice-Hall (1986) [4] Kanfer, F.H.: Self-management strategies and tactics. (1979) 195–224 [5] Collella, V., Bororvoy, R., Resnick, M.: Participatory simulations: Using computational objects to learn about dynamic systems. In: SIGCHI conference on Human factors in computing systems (CHI98). Los Angeles, USA. (1998) [6] Heumer, G., Carlson, D., Kaligiri, S.H., Maheshwari, S., Hasan, W.U., Jung, B., Schrader, A.: Paranoia syndrome - a pervasive multiplayer game using pdas, rfid, and tangible objects. In: Third International Workshop on Pervasive Gaming Applications on Pervasive Computing 2006. Dublin, Ireland. (2006) [7] Benford, S., Crabtree, A., Flintham, M., Drozd, A., Anastasi, R., Paxton, M., Tandavanitj, N., Adams, M., Row-Farr, J.: Can you see me now? ACM Trans. Comput.-Hum. Interact. 13(1) (2006) 100–133 [8] Klopfer, E., Squire, K.: Environmental detectives: the development of an augmented reality platform for environmental simulations. In: Educational Technology Research and Development. (2008) 203–228 [9] Aylett, R.: Narrative in virtual environments: Towards emergent narrative. In Press, A., ed.: Fall Symposium, Technical report FS-99-01. (1999) 83–86 [10] Paiva, A., Dias, J., Aylett, R., Woods, S., Hall, L., Zoll, C.: Learning by feeling: Evoking empathy with synthetic characters. Applied Artificial Intelligence 19 (2005) 235–266 [11] Dias, J., Paiva, A.: Feeling and reasoning: A computational model for emotional agents. In: 12th Portuguese Conference on Artificial Intelligence (EPIA 2005), Springer (2005) 127–140 [12] Riedl, M., Young, R.M.: An intent-driven planner for multi-agent story generation. In: Proceedings of the 3rd International Conference on Autonomous Agents and Multi Agent Systems. (2004) [13] Hofstede, G.: Cultures and Organisations. McGraw-Hill, London (1991) [14] Baylor, A.L., Ryu, J.: Evidence that multiple agents facilitate greater learning. In: International Artificial Intelligence in Education (AI-ED). Sydney, Australia. (2003) [15] Moreno, R., Mayer, R.E., Spires, H., Lester, J.: The case for social agency in computer-based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction 19 (2001) 177–213 [16] Dörner, D.: The mathematics of emotions. In Frank Detje, D.D., Schaub, H., eds.: Proceedings of the Fifth International Conference on Cognitive Modeling, Bamberg, Germany (Apr, 10-12 2003) 75–79 [17] Lim, M.Y., Dias, J., Aylett, R., Paiva, A.: Improving adaptiveness in autonomous characters. In: The 8th International Conference on Intelligent Virtual Agents, Tokio, Springer (2008) 348–355 [18] Paiva, A., Prada, R., Chaves, R., Vala, M., Bullock, A., Andersson, G., Höök, K.: Towards tangibility in gameplay: building a tangible affective interface for a computer game. In: ICMI ’03: Proceedings of the 5th international conference on Multimodal interfaces, New York, NY, USA, ACM (2003) 60–67 [19] Fails, J., Druin, A., Guha, M., Chipman, G., Simms, S., Churaman, W.: Childs play: A comparison of desktop and physical interactive environments. In: Conference on Interaction Design and Children. (2005) 48–55 [20] Dow, S., Mehta, M., Harmon, E., MacIntyre, B., Mateas, M.: Presence and engagement in an interactive drama. In: Proceedings of the SIGCHI conference on Human factors in computing systems. (2007) 1487–1484 [21] Inkpen, K., Ho-Ching, W., Inkpen, K.H.C., Scott, S., Shoemaker, G.: This is fun! we’re all best friends and we’re all playing: supporting children’s synchronous collaboration. In: Proceedings of the 1999 conference on Computer support for collaborative learning. (1999) [22] Mandryk, R., Inkpen, K., Bilezikjian, M., Klemmer, S., Landay, J.: Supporting children’s collaboration across handheld computers. In: Proceedings of the SIGCHI conference on Human factors in computing systems, New York, USA. (2001) [23] Leichtenstern, K., Andr, E., Vogt, T.: Role assignment via physical mobile interaction techniques in mobile multi-user applications for children. In: European Conference on Ambient Intelligence, Darmstadt, Germany. (2007) [24] Ang, S., Van Dyne, L., Koh, C., Ng, K.Y., Templer, K.J., Tay, C., Chandrasekar, N.A.: Cultural intelligence: Its measurement and effects on cultural judgment and decision making, cultural adaptation and task performance. Management and Organisation Review 3 (2007) 335–371 [25] Stephan, W.G.: Intergroup relations. The Handbook of Social Psychology (1985) 599–659 [26] Wright, S.C., Aron, A., McLaughlin-Volpe, T., Ropp, S.A.: The extended contact effect: Knowledge of cross-group friendships and prejudice. Journal of Personality and Social Psychology 73 (1997) 73–90 [27] Hall, L., Woods, S., Aylett, R., Paiva, A., Newall, L.: Achieving empathic engagement through affective interaction with synthetic characters. In: International Conference on Affective Computing and Intelligent Interaction, Beijing, China. (2005)