Computers & Education 44 (2005) 135–154 www.elsevier.com/locate/compedu Heterogeneity of learning material in synchronous computer-supported collaborative modelling Christos Fidas a, Vassilis Komis b, Spyros Tzanavaris b, Nikolaos Avouris a a,* ECE Department, Human–Computer Interaction Group, GR-University of Patras, 26500 Rio Patras, Greece b Department of Early Childhood Education, University of Patras, GR-26500 Rio Patras, Greece Abstract This paper examines the effect of heterogeneous resources, available to students, during computersupported collaborative problem solving. A study of collaborative modeling has been conducted in the frame of an authentic educational activity in a secondary school. The students involved were provided with sets of primitive resources of varying degrees of heterogeneity to be used during synchronous computermediated modeling activities. Analysis of students’ peer interaction and of the produced solutions revealed that, contrary to our expectations, the group with heterogeneous resources produced solutions of similar quality to those of the reference group, although they were more active, they exchanged more messages, they were involved in deeper discussions and collaborated more for building the constituent parts of the solution.
2004 Elsevier Ltd. All rights reserved. Keywords: Computer-mediated communication; Cooperative/collaborative learning; Human–computer interaction 1. Introduction Modern approaches in teaching and learning put emphasis on problem solving activities that involve collaboration. It seems that there is a wider acceptance of the fact that these approaches encourage construction of knowledge and building of meaning. The main benefits of collaborative learning are related to the active character of the learning process, the deep level of information processing and the requirement of deep understanding from the students involved (Dillenbourg, 1999; Scardamalia & Bereiter, 1994). Through such approaches, skills of critical thinking, * Corresponding author. Tel.: +30-610-997349; fax: +30-610-997316/30-2610-996820. E-mail address: n.avouris@ee.upatras.gr (N. Avouris). 0360-1315/$ - see front matter
2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2004.02.001 136 C. Fidas et al. / Computers & Education 44 (2005) 135–154 communication and coordination can be developed and conscious knowledge construction mechanisms can be built (Stahl, 2002; Steeples & Mayers, 1998). Network-based computer systems offer new possibilities in this context and at the same time raise new questions related to the feasibility and effectiveness of distance collaboration. Also questions relate to the factors that affect collaboration, the role of the symbolic and physical tools that support human activity and communication in this context as well as the role of human interaction and peer support during collaborative learning (Gassner, Jansen, Harrer, Herrmann, & Hoppe, 2003; Martinez, Dimitriadis, Rubia, G omez, & de la Fuente, 2003; Suthers, Hundhausen, & Girardeau, 2003). In computer-supported collaborative problem solving, knowledge building takes place through student peer interaction, interaction between the students and external representations, between students and teachers or students and software agents. Communication often takes place through specially designed tools, which should remain transparent in order not to interfere with students’ problem solving activity (Reiser, 2002). Synchronous communication, for instance, can take place through exchanged text messages using chat tools (Baker, Hansen, Joiner, & Traum, 1999) and through shared activity boards, in which problem solutions are constructed. The collaborating partners share this way representations of cognitive artifacts, supporting common understanding. In this context, special interest has been recently shown on the investigation of the conditions under which computer-supported collaborative problem solving can be effective. Investigation of these conditions often involves design of collaborative learning environments, e.g. environments, which provide learning resources and in particular primitive entities that can be used in the process. In most cases these primitive entities belong to a pre-determined closed set. Examples of these primitives can be abstract objects, like rectangles, ellipses, squares, different statement types, etc., as it is the case in Belvedere (Suthers & Jones, 1997), COLER (ConstantinoConzalez & Suthers, 2001), C-CHENE (Baker & Lund, 1997), Modeler Tool (Koch, Schlichter, & Trondle, 2001). These can take on a special meaning for the students during problem solving. Common understanding among collaborators is based on the existence of these common basic primitives and the solution is built using these shared available resources. This is one of the mechanisms provided for scaffolding the collaborative activity. These common primitives are the items about which the users argue and discuss before converging to a commonly acceptable solution (Suthers, 2000). According to Stahl (2002) the students can start their argumentation only after they have built a common understanding of their meaning and use it in the modeling activity. However this ‘‘closed environment’’ assumption is not always true. Today collaborative problem solving activity can take place within open systems, which permit additional resources to be built or sought by the students themselves. In addition, pedagogical motivations often encourage this ‘‘open’’ approach. As a consequence the building blocks are not shared among all the partners who therefore need to negotiate the available resources before even start getting engaged in problem solving. The collaborators search for primitive entities in a wider space like the Internet or even build new entities themselves during the process. This is the case of ModelsCreator 3.0 (MC3) (Fidas, Komis, Avouris, & Dimitracopoulou, 2002) the environment used in our study. MC3 permits synchronous distance collaboration for building and exploring models made out of an open set of primitive entities. These entities represent concepts with properties and visual behaviour. In this environment a student before entering in a specific collaborative modeling C. Fidas et al. / Computers & Education 44 (2005) 135–154 137 session may search for or build individually a new set of primitive elements to which meaning can be assigned. The student is provided with adequate tools (editors) that permit creation of these new entities or modification of existing ones. As a consequence, collaborating students may find themselves in possession of heterogeneous sets of primitive objects. Even if the collaborating partners share a problem definition and given data set, one or more of the partners may have access to additional basic constructs or compound primitives, making the process of grounding of interaction and common understanding particularly complex. In the reported research we have attempted to investigate this aspect of collaborative learning, by studying the role of not-shared primitive constructs in collaborative modeling activities (Komis, Avouris, & Fidas, 2003). This is a key question relating to the heterogeneous nature of the context of collaborating partners, who do not necessarily share common conventions, cultural and cognitive backgrounds, tools and resources. Building a common understanding in such a case is considered particularly difficult. In particular, during the reported study we examined how heterogeneous primitives affect synchronous collaboration at a distance. The main premise has been to investigate if the heterogeneity of resources has any effect on collaboration, since the students need in this case to seek and agree on a common set of primitives before building a solution, while there was also a concern that the lack of a common set of primitive resources can create confusion and uncertainty to the students. In order to achieve these objectives we performed a case study involving collaborative problem solving by pairs of students at a distance, when the partners possessed sets of building blocks of the solution of varying degrees of heterogeneity. In Section 2 of the paper the context of the study, methods and tools used are described. In Section 3 we analyse the collected data and discuss the findings of the empirical study, finally in Section 4 the results are summarized and the implications of the reported research are outlined. 2. Methods and tools 2.1. Context of the study The reported study took place in the frame of educational activity of a Greek Technical Lyceum (ages 15–16). A class of 20 students (12 male and 8 female), that followed the Information Technology specialization track, participated in the experiment, in the context of their ‘‘Internet Technology’’ course, halfway through the spring semester of 2002. The instructor participated actively in the experiment. The students volunteered to participate in the study and they were not evaluated for their performance in this problem solving activity. The activity that they were given was well situated within the context of their course, as it involved gaining hands-on experience with an Internet-based groupwork tool (MC3). All students had developed computing skills, while half of them owned a personal computer. The class during the problem solving activity was divided in two groups (A and B), each one of them made of five (5) pairs of students: Ai , i ¼ 1,5 and Bj , j ¼ 1,5. Special attention was put on selecting the pairs of students in such way that their cognitive and subject matter skills to be of similar level, so that symmetric peer support to be facilitated. 138 C. Fidas et al. / Computers & Education 44 (2005) 135–154 2.1.1. The study process The members of each pair collaborated using the MC3 environment in order to provide a solution to the ‘‘Travel Agency problem’’, which is described in more detail in the next section and in the Appendix. First the students were trained in using the software tools. Subsequently they were given the activity sheet and instructions on the problem solving strategy to be followed during the provided time (50–55 min). The tutor did not intervene during problem solving. The physical location of the students in the lab was such that the collaboration within each pair was effected exclusively through the provided tools, thus simulating distance collaboration. The pairs of group A, which was used as a control group, shared a common set of primitive entities relevant to their problem, while those of group B possessed heterogeneous entity sets. A week later, the students participated in an open discussion in the class, during which they commended on the process, they discussed their solutions and the tools used, while at the end they filled a provided questionnaire. 2.1.2. The task The problem that was given to each pair was to form an offer for a package holiday, pretending they were two clerks of a travel agency, working at a distance. The students were provided with primitive entities representing key concepts, these were the holiday budget, the cost of lodging, the duration of holidays, the cost per person, the number of travelers and the means of transport, see Fig. 1. Each one of these entities was located in an entity library where a description, a visual representation, a set of attributes with corresponding ranges of values could also be found. These entities could be inter-related in the activity space in order to build a model of the holiday offer, on which various if-then-else scenarios could be tried. In the Appendix A summary of the Activity sheet is included. 2.2. The Models Creator 3.0 learning environment During this study the ModelsCreator 3.0 (MC3) (Fidas et al., 2002) collaborative modelling environment was used. This is an evolution of Models Creator 2.0 (MC2) (Dimitracopoulou, Fig. 1. Settings of research: group A (shared libraries of primitive entities) and group B (heterogeneous libraries of primitive entities). C. Fidas et al. / Computers & Education 44 (2005) 135–154 139 Komis, Apostolopoulos, & Politis, 1999; Komis, Dimitracopoulou, Politis, & Avouris, 2001), an earlier prototype supporting modeling activities in a stand-alone setting. MC3 is a collaborative modelling learning environment that facilitates building of semi-quantitative models, quantitative models, as well as static qualitative models (concept maps) (Fidas & Komis, 2001), with special emphasis on semi-quantitative modelling. An evolution of the MC3 concepts can be found in the more recent ModellingSpace environment (Avouris, Margaritis, Komis, Saez, & Melendez, 2003c). The structural elements of the models built in MC3 are the entities, the entities’ properties and the relationships among entities. Entities are the objects or the concepts that constitute a model (such as man, tree, holidays etc.). Properties are intrinsic characteristics of the entities that change, rendering the model dynamic behaviour. Relationships define the ways, according to which the entities’ properties change and affect each other. The student using MC3 can modify the primitive entities available. New entities can be defined using an Entities Editor or can be imported from the Internet or exchanged among users of MC3. These characteristics of MC3 make it an open environment, in which the primitive modelling constructs can vary between different installations. MC3 puts great emphasis on visualization of the modelling entities, their properties and their relations. Visualization is considered crucial in supporting the reasoning development of young students, favouring the transition from reasoning over objects to reasoning with abstract concepts. This feature is extended also to the simulation of executable models allowing their validation through representation of the phenomenon itself in a visual way and not in an abstract one, as it is usually the case. The visualisation of an entity is achieved by defining a number of states and associating images to each state of the entity. The user interface of MC3 during modeling activities is shown in Fig. 2. 2.2.1. Synchronous peer collaboration through MC3 MC3 supports synchronous and asynchronous collaboration at a distance. MC3 contains tools for exchange of text messages between collaborating partners and sharing of the common activity space, through a replicated architecture. The synchronous collaboration tools of MC3 have been used in our study. The shared Activity Space of MC3 (see main area of Fig. 2) can become a drawing space of synchronous collaboration, in which one of the two collaborating partners can insert primary objects (entities and relations), through direct manipulation. The coordination protocol used in this study is described here: When connection between two partners is established, a copy of the drawing board is build and maintained in both parts involved until the connection is terminated by one of the two partners. The two partners can exchange roles, playing either the passive or the active role. The active partner is the one who can manipulate objects in the activity space. These actions generate messages transmitted to the passive partner, thus reproducing the same effect on the screens of both workstations. So MC3 supports a shared WYSIWIS (what you see is what I see) environment. A mechanism is established for exchange of roles. The metaphor used is that of ‘‘passing the key ’’. The holder of the ‘‘action-enabling key’’ is the active partner. Through this key request/ key accept/ key reject protocol, the active role can change at any point during collaboration, provided that the passive partner requests the key and the active partner accepts the request. The keypassing tool is shown at the bottom of Fig. 2. This protocol maintains clear semantics of actions and roles in the shared activity space and therefore is an essential part of the architecture. This 140 C. Fidas et al. / Computers & Education 44 (2005) 135–154 Fig. 2. The ModelsCreator v. 3.0 (MC3) User interface during model building. consideration seems to be in agreement with the view expressed by researchers of similar environments, see (Soller, 2001) and (Komis, Avouris, & Fidas, 2002). An extensive discussion on the effect of various coordination support mechanisms on synchronous problem solving is included in (Avouris, Margaritis, & Komis, 2003b). Peer support in this context takes place through exchanged chat messages between the students and through actions in the activity space by the key holder. 2.3. Educational scenario for collaborative problem solving The educational scenario was based on studies of Pinelle and Gutwin (2002) on collaborative problem solving. The scenario involved formulation of a package holiday offer by the students who were supposed to be clerks of a Travel Agency working at a distance. They had to negotiate on the main entities affecting the characteristics of the offer, using the provided tools (chat tool and collaborative modelling tool). Many researchers argue that the problems that users should be asked to solve need to be authentic and should be based on realistic scenarios (Gutwin & Greenberg, 2000; Hansen, Holmfeld, Lewis, & Rugelj, 1999). The activity sheet that was given to the students explicitly requested from them to discuss the main entities that affect the offer, using the chat tool, and to define their relation (total cost, C. Fidas et al. / Computers & Education 44 (2005) 135–154 141 destination, duration, cost of lodging etc.) Subsequently, the students had to build jointly a model, which would represent their offer. Each pair of distant students produced a separate solution to the problem. 2.3.1. First findings on the produced solutions The setting of the experiment and the tools did not seem to cause any particular usability problems to this group of students, who were quickly acquainted with the tools and used them efficiently to hold conversations and build models of the problem. The fact that the students who participated in the study had Information Technology as the main focus of their studies, perhaps contributed to this. In contrast, the quality of the produced solutions to the problem was not particularly high, as there seemed to exist lack of understanding of the background domains involved in the given problem (tourist industry, holiday budgeting etc.). The solutions produced were evaluated in a 10 points scale, according to which, the solutions of group A took the following marks: A1 ¼ 3.0, A2 ¼ 3.0, A3 ¼ 2.5, A4 ¼ 4.5 while those of group B were B1 ¼ 3.0, B2 ¼ 4.0, B3 ¼ 2.0, B4 ¼ 4.0, B5 ¼ 3.0. The mean score per pair was 3.2 for both groups. So a first finding was that the heterogeneity of the primitive resources did not have any effect on the quality of the produced solutions. In the next section we proceed with analysis of the collaborative problem solving activity that produced these solutions. 3. Analysis of collaborative problem solving The findings of the study, discussed in this section are based on the following data, collected during the field experiment: (a) logfiles of activity which include exchanged dialogue messages and operations on the common activity space, in chronological order, (b) the solutions produced by the 10 pairs of students, (c) the filled questionnaires during the second phase of the study and (d) video recording of the open discussion and evaluation by the students themselves. Analysis of these data is done along the following dimensions: • Analysis of dialogue and activity (exchanged text messages and operation in the common activity space), based on the OCAF model of analysis (Avouris, Dimitracopoulou, & Komis, 2003a, 2003b, 2003c), described in Section 3.1, • quantitative and qualitative analysis of interaction, • detailed analysis of the history of the heterogeneous entities (this concerns mainly group B), • micro-analysis of dialogues concerning the heterogeneous entities. In the following section we provide a brief introduction to the main principles of the methodology used for analysis of dialogues and actions. 3.1. Methodology of analysis The Object Oriented Collaborative Analysis Framework (OCAF) (Avouris et al., 2003a) used in this study, is particularly suitable for analysis of collaborative problem solving activity, which involves interleaving of modeling actions and dialogue. This framework puts emphasis on the 142 C. Fidas et al. / Computers & Education 44 (2005) 135–154 Table 1 OCAF analysis model: the main functional types Functional role Derived from: Example I ¼ Insertion of the item in the shared space Action analysis P ¼ Proposal of an item or proposal of a Dialogue analysis state of an item C ¼ Contestation of the proposal Dialogue analysis R ¼ Rejection/refutation of the proposal Action and/or dialogue analysis X ¼ Acknowledgement/acceptance of the Action and/or proposal dialogue analysis T ¼ Test/Verify using tools or other means of Actions and dialogue an object or a construct (model) analyses Action: ‘Insertion’ of Entity ‘‘X’’ Message: ‘‘I believe that one entity is the ‘‘A’’ or ‘‘let us put the value of entity Y to state locked ’’ Message: I think that this should be linked to the entity B by the ‘‘analogue to’’ relation Message: ‘‘What their attributes will be ? I don’t agree’’. Or Action: ‘Delete’ Entity ‘‘X’’ Message: ‘‘That’s right’’ or Action: Insertion of a proposed entity Message: Let us run this model to observe this part of the model behavior Action: Activate ‘Graph Tool’, or ‘Barchart Tool’’ objects of the jointly developed model. Every object is assigned its own history of events (actions and messages) related to its existence. The history of each one of these objects is a sequence of events that refer to an actor and an action according to the functional types, shown in Table 1. An example of an object history is: E(budget) ¼ XP , XI , YP , XR , indicating that agents X and Y interacted in relation to entity budget taking the assigned functional roles: First X made a proposal (XP ), then X inserted the object (XI ), then Y made a new proposal (YP ), which subsequently was rejected by X (XR ). The activity of all pairs of groups A and B has been analysed following this framework. The actions of the partners and the exchanged messages have been classified according to this scheme. As a result, the parts of the produced models have been accordingly annotated. In addition, a quantitative analysis of occurrence of certain functional types has been performed, providing some indication of the quality and degree of collaboration, discussed in the following. 3.2. Quantitative analysis of dialogues and actions Comparison of the overall activity of the pairs of groups A and B is included in Table 2, also shown in Fig. 3. In this summary table, the average number of actions and exchanged text messages per partner is shown. Partners of group A produced as an average 16.5 events while partners of group B 19.9 events, while the corresponding standard deviations where 2.75 and 3.32. So overall group B was more active. In Table 2 the breakdown of this activity according to the OCAF functional roles is Table 2 Comparison of occurrence of OCAF functional roles in activities of Groups A and B Group (Insert) R(Reject) (Propose) (Test) (Accept) C(Contest) Mean SD A B Mean Difference 9.2 9.3 9.3 0.1 3.6 4.4 4.0 0.8 1.4 2.9 2.2 1.5 1.5 2.3 1.9 0.8 0.7 1 0.9 0.3 0.1 0 0.1 )0.1 16.5 19.9 18.2 3.4 2.75 3.32 3.03 C. Fidas et al. / Computers & Education 44 (2005) 135–154 143 Fig. 3. Comparison of average actions per partner of groups A and B broken down according to OCAF functional roles (see Table 1). shown. The main difference is contributed to Proposals (1.5 more actions for group B in average), Rejections (0.8 actions) and Testing (0.8 actions), while for the rest of the functional roles the difference is not significant. The unpaired two-tailed t-test has confirmed statistically significant difference of the mean values for Proposals (t ¼ 2:31, P ðtÞ ¼ 0:049), while the difference is not statistically significant for the other types of events. It is known that high value of operators C (Contest), and X (Accept), is considered strong indication of conflict (Avouris et al., 2003a). In our case these operators had low values in both groups. This is in accordance with similar observations made in other studies, e.g. by Wu, Farell, and Singley (2002), as it is observed that in cases of peer interaction in groups of symmetric characteristics, the conflict indices are low. I (Insertions) are of the same level for both groups, since the produced solutions where of similar complexity as discussed in the evaluation of the solutions in Section 2. Rejections (R) are higher in group B (not statistically significant difference though), since in group B more proposals (P) were made. The Propose (P) role occurred more often in group B than in group A. According to Avouris et al. (2003a), this functional role is a strong indication of ownership of entities and relations as well as strong indication of participation in collaboration and peer support. It has been observed that the pairs of group B took the T (test/verify) functional role more actively. This can be related to the fact that in this group there were more proposals, which needed to be evaluated and rejected after testing and verification, using the provided tools (i.e. the ‘‘run the model’’ tool, the handlers for manipulating the values for entities’ properties). The conclusion of this analysis is therefore that group B is more active and take roles that indicate collaborative activity, like the Propose operator, more often than group A. In the following a more detailed analysis of this activity is performed. 3.3. Analysis of interaction An additional point of view concerns the textual interaction that took place during problem solving, following a methodology also used by Komis et al. (2002). As expected, communication 144 C. Fidas et al. / Computers & Education 44 (2005) 135–154 between members of group B was more intense. The overall number of exchanged messages was 150 between partners of group A and 175 between partners of group B. Thus the mean number of exchanged messages per pair of group A was 30 and for group B was 35. By performing a t test on these two mean values of the two groups we obtained P ðtÞ ¼ 0:24253, t ¼ 1:26, considered not significant. However for all major categories of messages, the mean value of exchanged messages was higher for pairs of group B than pairs of group A as shown in Table 3. In particular, exchanged messages were classified as: (a) strategy related or control messages, (b) task related : task A (compilation of offer and discussion of entities) and task B (investigation of relations and model building and testing) (c) related to the usage of tools, (d) off-task/social, In all categories, except the tools-related messages, group B exchanged more messages, as seen in Table 3. While in either of these groups the difference is not statistically significant, the trend indicates that in group B interaction related to problem-solving tasks was more intense, while interaction related to the modelling task was higher in group B as a percentage of exchanged messages (21% in group B against 17% in group A). An analysis of distribution of messages to the partners of the pairs of groups A and B has also been performed, as shown in Table 3. The degree of symmetry of interaction indicates the contribution of each partner to the exchanged messages. The average value of this index of both Table 3 Analysis of interaction Group ID Message distribution in partners Total number of messages Message typology Task related Partner #1 Partner #2 Degree of symmetry A1 A2 A3 A4 A5 15 23 12 17 15 16 14 11 18 9 0.94 0.61 0.92 0.94 0.60 31 37 23 35 24 Total 82 68 0.80 150 B1 B2 B3 B4 B5 16 11 18 21 23 22 14 15 18 17 0.73 0.79 0.83 0.86 0.74 38 25 33 39 40 Total 89 86 0.79 175 Strategy/ control Task A ‘‘Offer’’ Task B ‘‘Model’’ 18 13 1 5 7 5 14 9 18 8 44 29% 54 36% 26 17% 4 3% 22 15% 13 6 4 7 17 0 11 17 17 14 15 6 7 5 3 0 0 0 0 0 10 2 5 10 6 47 27% 59 34% 36 21% 0 0% 33 19% 5 4 6 5 6 Tools 1 1 1 0 1 Off-task/ social 2 5 6 7 2 C. Fidas et al. / Computers & Education 44 (2005) 135–154 145 groups is around 0.8. In Fig. 4(a) the pairs of the two groups are shown in a scatter plot diagram. The closest each point of this graph is to the diagonal, the more symmetrical the interaction for this particular pair has been. In addition, the north-eastern points in the graph indicate more active pairs, while the south-western ones are the less active ones. Second order polynomial trendlines have been added for the two groups. By inspecting the graph, it is deduced that group B in addition to stronger interaction is more uniform in terms of symmetry of interaction (indicated in this diagram as the group B trendline is closer to the diagonal to that of group A). In Fig. 4(b) the graph of message categories per group, shown also in Table 3, are depicted. Finally it is worth studying the content of exchanged messages, in order to establish the degree of collaboration. In particular the interaction related to strategy in group B is much more rich. In group A, messages of this category, seem to refer mostly to the contents of the holiday offer, and to the activity-enabling key exchange, while in group B strategy-related interaction is mostly related to the modelling task thus involving deep interaction on the domain. For instance in pair B2, initially student 1 requests from 2 to send him/her all the entities, which is accepted by partner 2. In pair B1 the two partners discuss first verbally the content of their libraries and subsequently 25 (16,22) messages of partner #2 (14,23) (17,23) (18,21) group B 20 (17,18) (15,18) (9,15) 15 (15,18) (11,14) group A (11,12) Group A 10 Group B 5 51 (a) 0 15 20 messages of partner #1 25 number of messages 70 60 Group A 50 Group B 40 30 20 10 to of ol f-t s as k/ so ci al (b) st ra te gy /c on tro l ta sk "O ffe r" Ta sk "M od el " 0 Fig. 4. (a) Symmetry of interaction scatter plot and (b) breakdown of messages per category. 146 C. Fidas et al. / Computers & Education 44 (2005) 135–154 Table 4 Average history length per entity broken down to shared and not-shared entities Pair Not-shared entities Shared entities All entities A1 A2 A3 A4 A5 Mean SD 4.5 2.0 2.8 5.5 3.3 3.6 1.4 4.0 6.0 0.0 6.0 0.0 3.2 3.0 4.3 4.0 1.4 5.8 1.6 3.4 1.9 B1 B2 B3 B4 B5 Mean SD 5.3 5.0 5.8 4.5 2.0 4.5 1.5 3.0 0.0 2.0 8.0 2.0 3.0 3.0 4.1 2.5 3.9 6.3 2.0 3.8 1.7 they negotiate on their activity, e.g. event 73, partner (1) says ‘‘you add the property of entity holiday duration first and then pass me the key so that I relate it to holiday budget ’’. 3.4. Analysis of entities history The primitive entities used in problem solving were six (6) and are listed in Appendix B. They can be distinguished in two categories. The two entities that were shared by the partners of all pairs (called shared entities) and the four entities that were split between the two partners of the pairs of group B (called not shared entities). In this section we examine the history of all entities that participated in the produced solutions. In particular, one aspect worth investigation is whether the not shared entities carry longer history in pairs of group B than in pairs of group A, where they were shared. For all pairs we have built a table of the length of their entities history, derived from the OCAF model, in Table 4 a summary of these attributes is shown. From this table it can be observed that in not shared entities 1 there were 4.5 events per entity for group B, while there were 3.6 events per entity for the same entities in group A, where they where shared among the partners. The standard deviations for both groups were of similar value (1.5 and 1.4). The average history length per entity for the shared entities (cost per traveler and number of travelers) for group A was 3.2, while for group B it was 3.0, while the standard deviations were equal (3.0) i.e. there is no significant difference in the two groups activity related to them, while it seems that group B focused more their activity in the not-shared entities. Further analysis of the difference in the behaviour of the two groups, involves the detailed analysis of types of actions related to each entity that participated in the solution of each group. We performed this analysis separately for the shared and the non-shared entities. The results for groups A and B are shown in Table 5. 1 Entities Holiday budget, Lodging, Means of transport, Holidays duration where split among the partners of group B. 147 C. Fidas et al. / Computers & Education 44 (2005) 135–154 Table 5 Break down of activities’ types per entitya Entities Types of actions (OCAF) I Group A Shared Cost per traveller Number of travellers Mean (shared) SD (shared) Not shared Budget Lodging Means of transport Holidays duration Mean (not shared) SD (not shared) Group B Shared Cost per traveller Number of travellers Mean (shared) SD (shared) Not shared Budget Lodging Means of transport Holidays duration Mean (not shared) SD (not shared) a Total R P T X C 9 6 7.5 2.1 7 2 4.5 3.5 4 1 2.5 2.1 1 1 1.0 0.0 1 0 0.5 0.7 0 0 0.0 0.0 22 10 16.0 8.5 13 12 14 4 10.8 4.6 8 2 9 0 4.8 4.4 3 0 0 0 0.8 1.5 0 5 0 0 1.3 2.5 1 0 0 0 0.3 0.5 0 0 1 0 0.3 0.5 25 19 24 4 18.0 9.7 8 7 7.5 0.7 2 5 3.5 2.1 1 2 1.5 0.7 3 1 2.0 1.4 0 0 0.0 0.0 1 0 0.5 0.7 15 15 15.0 0.0 15 10 10 13 12.0 2.4 3 4 5 1 3.3 1.7 9 0 3 3 3.8 3.8 0 0 4 4 2.0 2.3 3 0 2 1 1.5 1.3 0 0 0 0 0.0 0.0 30 14 24 22 22.5 6.6 See Table 1 for description of the OCAF functional types. From Table 5 it is shown that there is not significant difference for the shared entities between actions of groups A and B, while for the not shared entities, it appears that type P actions (proposals) appear to differ significantly, a similar finding to that of Section 3.2 that concerned the overall activity. So the proposals seem to be the actions that contribute mostly to the observed (however not statistically significant due to the small sample) difference in the entities-related activity as well as the overall activity between the two groups. This is also shown in Fig. 5, which depicts graphically the number of history events per entity for the not shared entities, broken down according to the OCAF functional roles. 3.5. Entities ownership analysis An additional result of entities history analysis concerns the ownership of parts of the solution. The ownership of entities establishes which partners participate actively in an entity of a model. This is a key notion in the analysis framework used. Any actor that participates in the history of a part of the solution according to OCAF, either by proposing or contesting its creation is defined as ‘‘owner’’ of this part of the solution. If an entity has just one owner, this is an indication that there has been no collaboration in relation to this entity, while it is a good indication of C. Fidas et al. / Computers & Education 44 (2005) 135–154 number of history envents 14 12 3,5 3,0 2,5 2,0 1,5 1,0 0,5 0,0 -0,5 -1,0 -1,5 -2,0 #messages dif (B-A) 10 8 6 4 Dif(B-A) 2 0 ? R ? T X C entities events functional types dif (B-A) 148 A B Fig. 5. Activities’ types for not shared entities (light colour ¼ group A, dark colour ¼ group B). collaboration if both participants are the owners. It is obvious that there can be various collaboration schemes used by the students, i.e. one could be a ‘‘divide and conquer’’ strategy, which means that the partners decide explicitly or implicitly to build separate parts of the model and not interfere with each other’s activity or an alternative collaboration scheme can involve a more synergistic strategy when both discuss and argue about the constituent parts of the model. This collaboration can be measured through the entities ownership attribute. In Table 6 the ownership of entities index is shown. From this table, it seems that the overall degree of collaboration cannot be considered very high in either group, since there were considerable parts of the produced solutions that were owned by a single partner. This finding is similar to other studies, (Komis et al., 2002; Wu et al., 2002) and can be attributed to the lack of tutor’s intervention during problem solving as well as to symmetry of partners cognitive skills, as discussed in the description of the context of the empirical study. In group A just half of the entities are of joined ownership, while in group B they are more than two thirds. It seems from these tables that in group B more often than in group A, the entities that take part in the solution of the problem become subjects of discussion and collaboration. In other words, sectionalisation of the primitive entities among partners has not created, as one might expect, a more sectionalised solution, but rather contributes to joint solutions of the problem and therefore to more collaboration. 3.6. Qualitative analysis of dialogues So far it has been observed that in the activities of the pairs of group B, there were more proposals (P) of entities and there was more interaction about the not-shared entities introduced, than in those of group A. It was also observed that there were more jointly owned entities in group B than in group A, as discussed above. The existence of not homogeneous primitive entity sets in group B seemed to be the reason for this observed behaviour. However it was considered important to examine in more detail the activity that concerned the not-shared entities. In the following logfile extract (Fig. 6) the discussion relates to the search for adequate entities. Student 2 asks his/her peer which entities exist in the library (message 20). The partner gives a C. Fidas et al. / Computers & Education 44 (2005) 135–154 149 Table 6 Ownership of Groups A and B entities Pair Group A A1 A2 A3 A4 A5 Total % Group B B1 B2 B3 B4 B5 Total % Single ownership entities Joined ownership entities 3 1 1 1 1 1 1 2 2 2 7 47% 8 53% 0 1 1 1 2 3 3 2 2 1 5 31% 11 69% Fig. 6. Dialogue extract from interaction of pair B1. reply (message 22) and puts these entities in the common activity space; subsequently a relation is created by 1, between budget and means of transport, as agreed earlier on. At the end of this dialogue extract student 1 asks if the partner has any more useful entities (action 42). Student 2 replies by listing the entities in his/her library and asks for the action-enabling key in order to insert the requested entities (actions 62 and 63 in dialogue extract). In this extract it is also interesting to observe the terminology used by one of the partners referring to the entities. The entities are referred, using their visual images instead of the verbal descriptive terms. A second example of this process is shown in Fig. 7, which includes a dialogue extract of pair B2. The two students in this case investigate the relation between the holiday budget and the duration of the holidays. The two entities that are needed, in order to build this relationship, are 150 C. Fidas et al. / Computers & Education 44 (2005) 135–154 Fig. 7. Dialogue extract from interaction of pair B2. possessed by different partners: holiday budget belongs to student 1 and duration of holidays to student 2. Student 1 inserts the holiday budget (action 21) in the activity space and then the key is passed to Student 2 who inserts entity holiday duration (action 30). Finally an inverse proportional relation is built between the two entities. In general, the pairs of group A, first discussed the contents of the offer using the chat, subsequently they worked together building the requested model of the holiday budget. In contrary, the pairs of group B first discussed the entities available in their libraries in order to establish a common understanding of the available primitive entities representing concepts. They were engaged in a discussion on which of the entities could be of use for the model. As a result, they started collaborating on the entities earlier on. 4. Conclusions This study focused on the effect of heterogeneous sets of primitive entities on synchronous collaborative problem solving (Komis et al., 2003). In particular we studied the effect of notshared entities on collaborative modeling. The findings of this study are summarized here. Two groups of secondary education students were formed, made of five pairs each. The students were distributed uniformly in the groups in terms of their cognitive skills. The two groups differed only in terms of the primitive entities available to each pair, out of which they were asked to build a model of a holiday offer in a collaborative way. In group A, both partners of each pair shared the same primitive entities, while in group B some of the necessary entities were owned by only one of the two partners. The solutions produced and the activity that led to these solutions were analysed using the OCAF framework (Avouris et al., 2003a, 2003b, 2003c). A prime observation is that the degree of collaboration was not particularly high for either of the two groups. This was attributed mainly to lack of intervention by the tutors, who could have encouraged more collaborative activity. Also the produced solutions by the two groups were of similar quality. C. Fidas et al. / Computers & Education 44 (2005) 135–154 151 Some distinct differences were observed between the two groups. Group B was overall more active in terms of actions and dialogue. In addition it was observed that the pairs of group B made twice as many proposals concerning parts of the solution (2.9 against 1.4 proposals per partner of group A), an indication of stronger collaboration. Furthermore, by studying the history of the solution components we found that in group B considerably more components were owned by both partners (69%) than in group A (53%). From these observations, we concluded that in group B there was more discussion and collaboration relating to the constituent parts of the solution. This was mainly due to the not-shared entities, which stimulated more discussion and negotiation than the common ones. The existence of not shared entities instead of creating additional difficulties for the collaborating partners, as originally expected, rather provided a stimulus for more involvement and deeper discussions, without any deterioration of the quality of the produced solutions. Analysis of the problem-solving strategies used by the two groups revealed that while group A started straight away with problem solving, the pairs of group B first searched and discussed available building blocks (entities) and the concepts that they represent, which helped them build a more collaborative attitude. These findings were confirmed by the students’ own reaction, in the post-test discussion in the class, where they considered the existence of not-shared entities as one of the prime incentives for collaboration. Taking into account the importance of the primitive entities in collaborative problem solving, as these are the main constructs that support common understanding and building of a shared meaning (Baker et al., 1999; Seitamaa-Hakkarainen, Raunio, Raami, Muukkonen, & Hakkarainen, 2001), the findings of this study are of more general value. Open systems are inevitably characterized by heterogeneity of primary resources. One might expect that this ‘‘openness’’ can be a source of uncertainty and confusion, inhibiting effective collaboration. However the findings of this study reveal that there might be some positive aspects relating to the deeper engagement of partners who attempt to work together towards a common understanding. It should be observed that in the reported experiment the students shared a common cultural, cognitive and social context, as members of the same class. Luck of this condition could have inhibited further sharing of understanding the heterogeneous entities, a premise requiring further validation. In conclusion, open collaboration environments, like MC3, as they become available, set new challenges in collaborative problem solving, imposing new functionalities (Dillenbourg, Baker, Blaye, & O’Malley, 1995; Muehlenbrock, Tewissen, & Hoope, 1998). It was shown that the openness of these environments can eventually provoke more semantically rich patterns of interaction and development of new grounding mechanisms (Baker, de Vries, Lund, & Quignard, 2001). The introduced uncertainty by the existence of heterogeneous entity libraries has been overcome by closer collaboration of the partners, which seemed to have produced as good solutions, following different strategies. It seems that when certain resources are not shared, their owners feel obliged to negotiate their usefulness with their peers, deepening further the collaborative nature of the activity. Acknowledgements Special thanks are due to the students of 2nd Technical Lyceum of Patras who participated in the reported here empirical study, to H. Sakonidis of the Democritus Univ. of Thrace, for 152 C. Fidas et al. / Computers & Education 44 (2005) 135–154 providing the main idea for the ‘‘Plan the Holiday’’ activity sheet, used in the study and to projects ModelsCreator/Pinelopi (Greek Ministry of Education) and IST-2000-25385/ModellingSpace (European Union) for financial support on development of the tools used in the study. Appendix A. Summary of the activity sheet Suppose that you work in a travel agency and you would like to create a package holiday offer. Collaborate using the ModelsCreator 3.0 tool (chat and modeling area) with another agent and build a joint offer. Decide the destination, discuss the factors that you need to take in consideration using the chat tool, build an offer (e.g. 7 day holidays for €600, including transport). Study the effect of various factors, establish which ones affect the offer more. More specifically discuss and decide the relationship between holiday cost and duration, cost of traveling and overall cost, cost of lodging and overall cost, cost of traveling and cost of lodging. Build a model of the holiday offer using the provided entities and relationships. Appendix B. Available entities and relations B.1. Entities shared Not-shared entities in group B Holiday duration Lodging Shared entities Means of transport Holiday budget B.2. Relations Proportional Inverse proportional Cost per person Number of travelers C. Fidas et al. / Computers & Education 44 (2005) 135–154 153 References Avouris, N., Dimitracopoulou, A., & Komis, V. (2003a). On analysis of collaborative problem solving: An objectoriented approach. Computers in Human Behavior, 19(2), 147–167. Avouris, N., Margaritis, M., & Komis, V. (2003b). Real-time collaborative problem solving: a study on alternative coordination mechanisms, Proceedings of the third IEEE international conference on advanced learning technology (ICALT) (pp. 86–90), Athens, July 2003. Avouris, N., Margaritis, M., Komis, V., Saez, A., & Melendez, R. (2003c). ModellingSpace: interaction design and architecture of a collaborative modelling environment, Proceedings of the sixth conference on computer based learning in science (CBLIS) (pp. 993–1004), Nicosia, Cyprus. Baker, M., Hansen, T., Joiner, R., & Traum, D. (1999). The role of grounding in collaborative problem solving tasks. In P. Dillenbourg (Ed.), Advances in Learning and Instruction series, Collaborative-learning: cognitive and computational approaches (pp. 31–64). Pergamon, Elsevier. Baker, M., & Lund, K. (1997). Promoting reflective interactions in a computer-supported collaborative learning environment. Journal of Computer Assisted Learning, 13(3), 175–193. Baker, M. J., de Vries, E., Lund, K. & Quignard, M. (2001). Computer epistemic interactions for co-constructing scientific notions: lessons learned from a five-years research program. In P. Dillenbourg & A. Eurelings, Proceedings of the Euro computer supported collaborative learning (pp. 89–96), Maastricht, 22–24 March 2001. Constantino-Conzalez, M., & Suthers, D. (2001). Coaching collaboration by comparing solutions and tracking participation. In P. Dillenbourg & A. Eurelings, Proceedings of Euro computer supported collaborative learning (pp.173–180), Maastricht, 22–24 March 2001. Dillenbourg, P. (Ed.). (1999). Advances in Learning and Instruction series, Collaborative-learning: cognitive and computational approaches. Pergamon, Elsevier. Dillenbourg, P., Baker, M., Blaye, A., & O’Malley, C. (1995). The evolution of research on collaborative learning. In E. Spada & P. Reiman (Eds.), Learning human and machine: towards an interdisciplinary learning science (pp. 189–211). Oxford: Elsevier. Dimitracopoulou, A., Komis, V., Apostolopoulos, P., & Politis, P. (1999). Design principles of a new modelling environment supporting various types of reasoning and interdisciplinary approaches. In Proceedings of the ninth international conference of artificial intelligence in education (pp. 109–120). Ohmsha: IOS Press. Fidas, C., & Komis, V. (2001). The architectural design of a real time collaborative concept-mapping environment from distance. In P. Dillenbourg & A. Eurelings, Proceedings of Euro computer supported collaborative learning (pp. 656– 659), Maastricht, 22–24 March 2001. Fidas, C., Komis, V., Avouris, N., & Dimitracopoulou, A. (2002). Collaborative problem solving using an open modelling environment. In G. Stahl (Ed.), Computer support for collaborative learning: foundations for a CSCL community, Proceedings of CSCL 2002, Boulder, Colorado, USA, 7–11 January 2002 (pp. 654–655). Lawrence Erlbaum Associates, Inc.. Gassner, K., Jansen, M., Harrer, A., Herrmann, K., & Hoppe, U. (2003). Analysis methods for collaborative models and activities. In B. Wasson, S. Ludvigsen, & U. Hoppe (Eds.), Designing for change in networked learning environments, CSCL 2003: Computer support for collaborative learning, Bergen, Norway, 14–18 June 2003 (pp. 369– 378). Kluwer Academic Publishers. Gutwin, C., & Greenberg, S. (2000). The mechanics of collaboration developing low cost usability evaluation methods for shared workspaces. In Proceeding of the ninth IEEE WETICE Workshop 2000 (pp. 98–103). IEEE Press. Hansen, T., Holmfeld, L. D., Lewis, R., & Rugelj, J. (1999). Using telematics for collaborative knowledge construction. In P. Dillenbourg (Ed.), Advances in Learning and Instruction series, Collaborative-learning: cognitive and computational approaches (pp. 169–196). Pergamon, Elsevier. Koch, J. H., Schlichter, J., & Trondle, P. (2001). Munics: modeling the flow of information in organisation. First EuroCSCL 2001, (pp. 348–355). Komis, V., Avouris, N., & Fidas, C. (2003). A study on heterogeneity during real-time collaborative problem solving. In B. Wasson, S. Ludvigsen, & U. Hoppe (Eds.), Designing for change in networked learning environments, CSCL 2003: Computer support for collaborative learning, Bergen, Norway, 14–18 June 2003 (pp. 411–420). Kluwer Academic Publishers. 154 C. Fidas et al. / Computers & Education 44 (2005) 135–154 Komis, V., Avouris, N., & Fidas, C. (2002). Computer supported collaborative concept mapping: study of interaction. Education and Information Technologies, 7(2), 169–188. Komis, V., Dimitracopoulou, A., Politis, P., & Avouris, N. (2001). Experimentations sur l’utilisation d’un logiciel de modelisation par petits groupes d’eleves. Sciences et techniques educatives, Hermes, 8(1–2), 75–86. Martinez, A., Dimitriadis, Y., Rubia, B., G omez, E., & de la Fuente, P. (2003). Combining qualitative evaluation and social network analysis for the study of classroom social interactions. Computers and Education, 41, 353–368. Muehlenbrock, M., Tewissen, F., & Hoope, H. U. (1998). A framework system for intelligent support in open distributed learning environments. International Journal of Artificial Intelligence in Education, 9, 256–274. Pinelle, D., & Gutwin, C. (2002). Groupware walkthrough: adding context to groupware usability evaluation. In Proceedings of CHI 2002 (pp. 455–462). New York, NY: ACM. Reiser, B. J. (2002). Why scaffolding should sometimes make task more difficult for learners. In G. Stahl, Computer support for collaborative learning: foundations for a CSCL community, Proceeding of CSCL 2002, Boulder, Collorado, USA, (pp. 255–264). Scardamalia, M., & Bereiter, C. (1994). Computer support for knowledge – building communities. The Journal of the Learning Sciences, 3(3), 265–283. Seitamaa-Hakkarainen, P., Raunio, A.-M., Raami, A., Muukkonen, H., & Hakkarainen, K. (2001). Computer support for collaborative designing. International Journal of Technology and Design Education, 11, 181–202. Soller, A. L. (2001). Supporting social interaction in an intelligent collaborative learning system. International Journal of Artificial Intelligence in Education, 12, 40–62. Stahl, G. (2002). Introduction: foundations for a CSCL community. (pp. 1–2), In G. Stahl, Computer support for collaborative learning: foundations for a CSCL community, Proceedings of CSCL 2002, Boulder, Collorado, USA. Steeples, C., & Mayers, T. (1998). A special section on computer – supported collaborative learning. Computers and Education, 30(3/4), 219–221. Suthers, D. & Jones, D. (1997). An Architecture for intelligent collaborative educational systems. In B. du Boulay, R. Mizoguchi, Eighth world conference on artificial intelligence in education (AIED’97) (pp. 55–62). Suthers, D. (2000). Initial evidence for representational guidance of learning discourse. Proceedings of international conference on computers in education, November, Taiwan. Suthers, D., Hundhausen, C. D., & Girardeau, L. E. (2003). Comparing the roles of representations in face-to-face and online computer supported collaborative learning. Computers and Education, 41, 335–351. Wu, A., Farell, R., & Singley, M. (2002). Scaffolding group learning in a collaborative network environment. In G. Stahl, Computer support for collaborative learning: foundations for a CSCL community, Proceedings of CSCL 2002, Boulder, Collorado, (pp. 245–254), USA.