[go: up one dir, main page]
Next Article in Journal
A Retrospective Analysis of the COVID-19 Infodemic in Saudi Arabia
Previous Article in Journal
Smart Cities in Russia: Current Situation and Insights for Future Development
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Future Internet
Volume 13
Issue 10
10.3390/fi13100253
futureinternet-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Designing a Collaborative Virtual Conference Application: Challenges, Requirements and Guidelines

by
Teo Rhun Ming
,
Noris Mohd Norowi
*,
Rahmita Wirza
and
Azrina Kamaruddin
Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang 43400, Malaysia
*
Author to whom correspondence should be addressed.
Future Internet 2021, 13(10), 253; https://doi.org/10.3390/fi13100253
Submission received: 31 July 2021 / Revised: 8 September 2021 / Accepted: 17 September 2021 / Published: 29 September 2021
(This article belongs to the Topic Advances in Online and Distance Learning)
Browse Figures
Figure 1
<p>The time space matrix for CSCW: (<b>a</b>) The interface of Video Conferences for different location same time Tabletop System matrix [<a href="#B10-futureinternet-13-00253" class="html-bibr">10</a>]; (<b>b</b>) Example of different location and time using email application [<a href="#B11-futureinternet-13-00253" class="html-bibr">11</a>]. (<b>c</b>) Tabletop System for same location and time matrix [<a href="#B12-futureinternet-13-00253" class="html-bibr">12</a>]; (<b>d</b>) Kiosk System for same location different time matrix [<a href="#B13-futureinternet-13-00253" class="html-bibr">13</a>].</p> "> Figure 2
<p>Experimental set up and layout of equipment.</p> "> Figure 3
<p>Distribution of users’ usage frequency of features on virtual conferencing tool.</p> "> Figure 4
<p>Distribution of user’s perceived ease of use of the features on virtual conferencing tool.</p> "> Figure 5
<p>Top reasons for disabling the web camera during virtual conferences.</p> "> Figure 6
<p>Bar chart of web camera usage during a video conference meeting.</p> "> Figure 7
<p>The breakdown of emotional feedback when peers disabled their web cameras during a virtual conference meeting.</p> "> Figure 8
<p>Distribution of users’ prior collaborative experience.</p> "> Figure 9
<p>The system usability score (SUS) of the current virtual conference application.</p> "> Figure 10
<p>The proposed natural and affective virtual conference application.</p> "> Figure 11
<p>This is the flow process of the proposed natural and affective virtual conference application: (<b>a</b>) Flow process of detecting the facial expressions of the user and displaying it onto the avatar; (<b>b</b>) Flow process of detecting hand gesture used by the users and displaying as an action cues.</p> "> Figure 12
<p>The user interface of the proposed virtual conference application for remote collaboration when users enter the group.</p> ">
Versions Notes

Abstract

:
Due to the recent COVID-19 pandemic that has swept the globe, more people are working from home. People use synchronous applications to communicate remotely because they are not able to meet face-to-face. However, few research studies on the issues surrounding the virtual conference application, particularly those that include collaborative activities, have been conducted. The usability study recruited 16 participants (in four groups of four) to communicate synchronously while performing collaborative activities, such as drawing together on a shared screen. According to the findings of the usability study, users do not often use the collaborative tools provided by the current virtual conference application. This is due to low exposure and unfamiliarity with the use of collaborative tools. The findings also show that users frequently do not turn on the web camera due to several reasons, including privacy, connectivity issues, the environment, and background distraction. Turning on the web camera can also cause anxiety due to shyness in front of the camera. However, some participants prefer to turn on the web camera so that they can see each other’s reactions when performing collaborative activities. The article provides several guidelines to assist in the design of virtual conference applications, including a simple familiar intuitive interface to encourage the use of collaborative tools and also introduces the use of virtual avatars as a way to represent oneself during online meetings to allow affective sharing while respecting the privacy of its users.

1. Introduction

This research is primarily focused on a virtual conference application that has been overlooked for the past few years. With the recent COVID-19 pandemic still taking its course around the world, more people are resorting to online communication applications to adapt to the new normal [1]. This includes performing different online tasks such as meeting, learning, discussing, and problem solving through a video conference. However, the current pandemic has shown that 96 percent of users are frustrated when collaborating on any online communication format [2]. A previous study found that users prefer interactive sharing spaces to just video in one of its user studies [3]. Current virtual conference applications still use text, audio, and video to perform complex collaborative activities [4,5,6,7]. This paper focuses on the virtual conference application by identifying the issues and limitations encountered while performing collaborative activities. The following is how the paper is organized: Section 2 presents several theories and concepts on Human–Computer Interaction (HCI), collaboration, video conference, communications and the increase in the use of virtual conferences application during the current pandemic, and user experience. Section 3 describes the usability study that was conducted to evaluate user experience with the current virtual conference application. The results of the usability study are presented in Section 4. Section 5 discusses the findings of the usability study. Section 6 contains design recommendations, proposed system architecture, and an introduction to the user interface design for the virtual conference application. Finally, the work of this paper concludes with a discussion on its limitations and recommendations for future work.

2. Related Works

This paper focuses on the current user experience of virtual conference applications when performing collaborative activities. Due to the current pandemic outbreak, there appears to be very limited research in this area. However, the study is still able to learn from previous research on remote collaboration. To understand the user experience of the current virtual conference application, the paper outlines human–computer interaction and collaboration. The paper also discusses how the COVID-19 pandemic affects online collaboration, as well as issues surrounding the communication of emotions and non-verbal interaction. Then, we conduct a review on literature related to user experience and remote collaboration. Finally, the study should be able to assist in determining the indicators based on the influencing factors of user experience.

2.1. HCI and Collaboration

Human–Computer Interaction (HCI) research has focused on computer systems that have a group impact, mainly on Computer-Supported Cooperative Work (CSCW). CSCW is based on how groups work together and how technology affects group behavior [8]. A groupware is a set of software tools that support a group of people working together. Different theories have been created to combine the cooperative work of people through hardware, software, and networking. One of the theories that have been created is the CSCW matrix theory. The time space matrix is used in CSCW matrix theory to categorize groupware [9]. The first dimension is concerned with whether the interaction can be performed at the same time (synchronous) or at different times (asynchronous). The second dimension is considered when the individuals are located at the same location (collocated) or at different locations (remote). Combining these two-dimensions has resulted in four categories of groupware for CSCW: co-located and synchronous, remote and synchronous, collocated and asynchronous, and finally remote and asynchronous. Figure 1 shows the different example applications being categorized in the time space matrix.
Referring to Figure 1, each of the quadrants has its own set of advantages and disadvantages. For example, quadrant 1 is doing collaborative tasks at the same time and same location. This benefits tasks that are performed directly face-to-face. There is no input and output delay between users, as they can collaborate quickly without the use of the internet. However, performing collaborative work at the same place and time requires a larger space for users to move around and a bigger system for it to collaborate effectively. Collaborative Multi-Mobile System (CMMS) is an example of CSCW with the same time and location, where users can place multiple mobile devices on the table in a collocated manner to create a large surface for collaborative activity [14,15]. Quadrant 2 is carrying out the tasks at different times but at the same location, such as the kiosk. The benefit is that information can be shared at any time without the need for the parties to meet. However, users must return to the same place to retrieve the information required. Quadrant 3 has the advantage of performing collaboration at different locations at the same time without meeting face-to-face. However, the downside is that it requires a higher bandwidth to stream videos over the distance. Depending on the internet quality, the streaming can sometimes be delayed by a few seconds. In collaborative works, users want to make quick changes but the delay in real time application can cause frustrations. Finally, quadrant 4 collaborates at different locations and times. Although there are no delays with this method, users are rarely able to perform any real time-based activities for quick actions and changes.
This paper focuses on synchronous collaborative interaction groupware in quadrant 3 of the time space matrix. Examples of groupware that support synchronous collaborative interaction groupware are virtual conference applications, remote desktop applications, instance messaging, and multiuser text editing [16]. The research has mainly focused on the current video conference or telecommunication used by current users to communicate at a distance and at the same time using text, audio, and video.

2.2. COVID-19 Pandemic and Online Collaboration

The COVID-19 pandemic has generally forced people to use online applications such as virtual conference application or social media platforms to help connect people at a distance. People have relied on these platforms to perform a variety of activities such as learning, meeting, and collaborating with their peers. Schools have forced teachers and students to shift from the usual face-to-face settings to online settings for teaching and learning. Meetings between workers and co-workers in companies also take place in a virtual setting to share, present, and discuss information at hand. Some people use virtual conference applications to perform a collaborative task and to discuss ideas and solutions to a given problem.
Several previous studies on online communication have been conducted in order to find ways to support remote collaboration during the pandemic. One previous study has used remote collaboration to design homes without the need for the designer to be present [17]. Collaborative activity can also be used for group learning, which has been shown to be more effective than individual learning [18]. According to a previous study, students now learn better by collaborating with peers online, which is preferred by students [19]. For remote collaborative learning, previous research has found that the provision of opportunities to collaborate, discuss, and reflect on their professional development is beneficial [20]. Previous research identified opportunities for new and augmented approaches for remote collaborative research through Distributed Participatory Design by distributing geographies, backgrounds, ages, and abilities to overcome these new barriers during and after COVID-19 [21]. Previous studies have also looked at security issues such as user privacy, which is more vulnerable to be abused, especially during remote collaboration [22]. Currently, there has been a lack of study on the issues and limitations of performing a remote collaboration using the current virtual conference application, particularly during a pandemic period. Section 2.3 compares the various virtual conference applications available for real-time collaboration and its features.

2.3. Virtual Conference Application

There are several virtual conference applications or video conferences available on the market that can be used for remote collaboration. The most popular virtual conference applications have a 2D interface with a mouse, keyboard, and web camera for remote interaction/collaboration. Table 1 compares several virtual conference applications based on the features provided. Collaborative tools such as whiteboard, hand raise, room breakout, and annotation are available in all virtual conference applications. However, a previous study identified multiple weaknesses in the current virtual conferences application, such as security, privacy, media quality, reliability, capacity, and technical difficulties [23]. The virtual conference application also has a limited view as it displays on a flat 2D screen at a reduced scale [4]. The virtual conference application also has a fixed web camera placed at the edge of the display [24]. As a result, the current setup has a fixed viewing angle that requires the user to be in front of the screen. There are also issues with security, quality, and latency when communicating at a distance [23,25,26].

2.4. Communicating Emotions and Non-Verbal Interaction While Online

The current virtual conference application does not allow for the sharing of emotional awareness while communicating between local and remote participants. Previous research has shown that limited bonding capability becomes an issue, especially during remote collaboration, and the researchers of this paper would like to study the impact of collaborative teams on affective states [34]. This means that remote collaborators are not aware of other people’s emotions, such as happy, sad, neutral, frustrated, excited, and more. According to Meluso et al. [26], the greatest challenge for the current virtual conference application is conveying emotion. Emotions may be able to provide valuable information about the relationship between collaborators, allowing them to understand each other’s emotions and feelings. Chikersal et al. [35] have suggested that, in the future, the virtual conference application will be able to synchronize members’ facial expressions, as sharing facial expressions may enhance team performance. During this difficult time, many people have been using virtual conference applications for meetings, learning, and working. However, collaborators often turn off the video feed due to privacy, unstable connections, and limited data. Therefore, this research would like to understand more about the underlying issues that arise when the webcam is not available, particularly during a collaborative activity.
There is also a lack of non-verbal interaction options in the current virtual conference application. The keyboard and mouse are the main modes of interaction in the current video conference. The current virtual conference applications lack non-verbal interaction options such as hand gestures, eye gaze, and body movement [36]. A comparison revealed that non-verbal remote collaboration is better than the conventional virtual conference applications that rely solely on the mouse [37]. Participants have shown a general interest in non-verbal communication compared to only verbal communication [38]. Previous research has investigated ways to include non-verbal communication into areas such as communication behaviors, remote collaborations, and sharing cues. Faucett et al. [39] have developed a system that can sense and provide real-time feedback about non-verbal communication to improve communication behaviors. Non-verbal gestures can be useful in social interactions, especially in remote collaboration [7]. Therefore, this research would like to understand more on the underlying issues during non-verbal communication during collaborative activities.

2.5. User Experience (UX)

User Experience (UX) has been broadly defined as a method of understanding the feeling of users while using a system or application [40]. This includes all of the feelings, thoughts, actions, and sensations of engaging in some activity while using the system or application [41]. According to a systematic review conducted, there are four core factors that affect user experience, namely the user, the system, the context, and the temporal aspects [42]. The first major aspect of user experience is the user themselves, as they can come from different backgrounds in terms of age, gender, culture, etc., which can affect how the system is used. The interactive system can be designed to meet various goals and user needs; however, new technology has been introduced over the years, resulting in a different user experience than the previous system with old technology. Context is another important factor that influences user experience, as different conditions, physical environment, or even culture, can affect how users interact with the system or application. Finally, the temporal factor is a critical factor in user experience, as there would be no experience without time. For the time being, the study focuses solely on the user, system, and context aspect of the user experience. According to Virpi Roto [43], the user experience is subjective because the user’s state influences system perception, which affects the experience and the user’s state.
Previous studies have performed usability evaluation as a compulsory process to help determine the user experience of an application or system. The data can be obtained through questionnaires and interviews as a tool in a usability evaluation. In most commercial platforms, the evaluation of user experience is widely used. For example, Mishra et al. [44] used an online questionnaire to assess the ease of use and responsiveness of using augmented reality technology for online shopping. Hussain et al. [45] used a questionnaire to determine the user experience of Amazon Kindle application in terms of perceived ease of use, perceived visibility, perceived enjoyability, and perceived efficiency. Another variable that can determine the user experience is how the applications or systems are used. Chen et al. [46] calculated usage by ranking selected platforms for education and business based on the number of keyword user comments. Pal et al. [47] used a system usability scale questionnaire (SUS) to determine the user experience of smart voice assistants. Furthermore, interviews can be used to determine the user experience of products. Van Der Linden et al. [48] inquired about the usage frequency of interactive devices and 3D applications in virtually immersive environments and tablet usage for learning, respectively. For example, Følstad and Skjuve [49] used interviews to determine the user experience of using chatbots to provide better services to customers. Kelleci and Aksoy [50] used focus group interviews and observation to study the user experience of virtual classroom simulations used for the training of teachers.
Several studies on user experience for virtual conference applications for remote collaboration have been conducted. Thomaschewski et al. [51] designed an augmented reality awareness interface to support remote teams by using the usability and user experience score. Huidong Bai et al. [38] evaluated the user experience of mixed reality remote collaboration using gaze and hand gesture interaction. To validate the user experience, the study used the variables of mental workload, system usability, and preference. Another remote collaboration study looked into whether wearables could help improve user experience in remote collaboration by providing spatial annotation and by view-sharing data to a mixed reality headset [52]. It also measured the mental workload of the user to evaluate user experience. The subjective collaborative quality, satisfaction, and performance are used for understanding user experience in the remote collaboration study by projecting a gesture on a remote user from a local user using a mixed reality headset [53]. Most of the remote collaboration studies evaluate user experience either using virtual reality, augmented reality, or mixed reality 3D technology, but a few studies involve 2D-based remote collaboration. Anton et al. [54] compared the user experiences of 2D-based video conferences, 3D video interfaces, and 3D with annotations. The results showed that 3D with annotation has a higher rating compared to just 2D-based interface. Currently, there is a lack of study on remote collaboration using virtual conference applications, especially during the COVID-19 pandemic, which is what motivated this study. Moreover, in order to understand the user experience, this research would like to focus more on the users’ comments and feedback to evaluate the current virtual conference application for collaborative tasks. Finally, based on the previous study, several measurements have been identified to aid in the construction of a user study, particularly on remote collaboration on the current virtual conference applications. A guideline is also proposed based on the comments and feedback provided by the users.

3. Methodology

The aim of this study is to identify and understand the issues with the current virtual conference application with respect to usability and collaborative experience while performing collaborative activities. The study adapted the usability study to identify issues related to the usability and layout of the current virtual conference applications based on users’ feedback. It also implemented a user-based evaluation to evaluate user experience on current virtual conference applications.
The study focuses on identifying the limitations and issues encountered while performing collaborative activities on the current virtual conference application. The key research questions are: (1) What are the main issues and limitations of using current virtual conference applications while performing collaborative activity? (2) How does it affect the usability and user experience while using the current virtual conference application? (3) What are some recommendations or guidelines for improving the usability of the current virtual conference application? To answer these research questions, the exploratory experimental design with mixed method was used to identify the problems and limitations faced when using a virtual conference application for collaborative purposes.
The Zoom meeting application was used to test the software in this experiment. This was because, when compared to other commercial meeting applications, the participants in this study were more familiar with using the Zoom meeting application for day-to-day online synchronous activities, such as meeting, learning, and collaborating. The study designed its own data-collection tools to help validate the analysis to achieve the objectives because very few reviews have been conducted to explain the area of this study. The study collected data through a questionnaire and focus group interviews. The questionnaire required participants to answer 46 questions relating to usability and user experiences. The study collected self-reporting answers from the participants, which included attributes such as terms of usage frequency, ease of use, affective input, collaborative experience, and system usability scale (SUS). Self-reporting was used in lieu of log tools, as the questionnaire answers provided a better insight on the overall perception and preferences of the participants when using virtual conference applications outside of the experiment. Additionally, five questions were asked during the focus group interview. The questions asked participants to elaborate on their overall experience using the virtual conference application, to state why they may or may not switch on their web cameras during virtual conferences, whether they noticed any affective feedback from their fellow meeting buddies, and to provide additional comments and suggestions to improve the current virtual conference application. The data for the questionnaire was collected and analyzed using Google forms. Prior to the analysis, the focus group discussion was transcribed using a speech-to-text transcription software (YouTube transcription).

3.1. Participants

The experiment recruited 16 undergraduate and postgraduate (13 female, three male) students, aged between 16 and 40 years old from the Faculty of Computer Science and Information Technology, University Putra Malaysia, due to their familiarity with computers and technological background. A facilitator was also present during the experiment to help with instrument setup, distribution of materials, and to give instructions to the participants. The 16 participants were divided into five groups of three or four members, with only one group being studied at any one time.

3.2. Apparatus and Materials

Participants in the usability study were required to prepare their own computer to perform collaborative activities. Their computer should be equipped with a web camera, microphone, keyboard, and mouse, and the laptop should be set up on a desk. Each participant required a stable internet connection in order to complete the remote collaboration task that had been set. The participants were also required to install a virtual conference application (Zoom) to perform most of the collaborative activities.

3.3. Tasks

The tasks designed in this study were intended to understand users’ interaction and navigation when they are performing remote collaboration activities. Before the start of the experiment, participants were briefed on the outline of the study and consent was obtained. Although each participant joined remotely from different locations, the facilitator served as a host and set up the camera, laptop, WiFi, and virtual conference application from his location to record the interaction, as well as welcomed the participants to the virtual experiment space. Figure 2 shows the experimental setup for the usability study.
The participants were required to use the virtual conference application (Zoom). The session was recorded. Each group had a total of three participants. One of the participants was made the host and was asked to create a new meeting. The other participants were then invited to join the meeting. The group was given an instruction that read “You are required to collaboratively draw Your Dream Holiday”, and they were given 15 min to complete the task collaboratively using the tools readily available on the virtual conference application they were using. After the participants had completed the experiment, the recording was saved.
The group then moved to the next section of the study, in which they each had to fill out a questionnaire about their recent collaborative virtual conference experience. To better understand the issues and challenges associated with performing online collaborative tasks, a focus group discussion was then carried out.

4. Results

The following sections show the results of the questionnaire that was given to the participants after they completed the tasks in the study. The data collected include the usage frequency of the several features available on the virtual conference application (e.g., annotation, screen sharing, etc.), the perceived ease of use, affective input, collaborative experience, and the usability of the application as measured by the System Usability Scale (SUS).

4.1. Usage Frequency of Features on the Virtual Conferencing Application

The frequency of feature usage describes how frequently participants utilized the features available on the virtual conference application during the study. The features included were items such as the web camera, the microphone, the chat function, and so on. Based on the results of the questionnaire, the most frequently used tools in the study when performing collaborative activities were primarily the share screen feature and the microphone feature (18%). The less frequently used tools were video recording, the reaction to indicate emotions, the raising hand feature (i.e., to signal permission to speak up), the breakout room feature, the file sharing feature, the whiteboard feature, the chat feature, and the web camera feature (i.e., enabling the camera to be switched on). This suggests that the participants only used the features marginally. It is possible that the participants were not familiar with the virtual conference application and had not fully explored all of its features. However, it strongly indicates that, in general, only a few features were necessary to carry out collaborative tasks during a virtual conference. Figure 3 depicts a diverging stacked bar chart of feature usage frequency for the current virtual conference application for collaborative purposes.

4.2. Ease of Use

According to the questionnaire, 73% of the participants stated that the virtual conference application was easy to use, while 27% stated otherwise. Further analysis reveals that the recording feature, the reaction feature, the raise hand feature, the breakup room feature, the screen sharing feature, the chat feature, and the microphone feature were rated as the easiest to use on the virtual conference application by participants. Participants also reported that the file sharing feature, along with the whiteboard and annotation features were more difficult to use. This finding is interesting, as the annotation feature, the file sharing feature, and the whiteboard feature, were all designed to facilitate the completion of collaborative tasks but were rated poorly by the participants. Figure 4 shows the diverging stacked bar chart of ease of use for each feature provided by virtual conference application.

4.3. Web Camera Usage/Affective Input

Moods, feelings, and emotions, or affective input, can be communicated across the virtual conference application by using features such as switching on the web camera to provide live input of emotional expressions. Another way to express emotions is through the reactions feature. In one section of the questionnaire, participants were asked if they could identify the emotions of other group members who were participating in the collaborative task in the same user study session as them. It was found that participants agreed that it was hard to read the emotions of other group members in the session, because they had all turned off their web cameras most of the time. Several reasons were cited by the participants for their less affective behavior, including: (1) reducing the consumption of their mobile data, (2) hiding the less-than-ideal background environment in which they participated in the user study, e.g., busy, messy, etc., (3) protecting their privacy, and (4) avoiding showing their self-body image due to low confidence. Two participants mentioned hardware issues with their web cameras, and one participant admitted to being away from her computer for a period of time. Figure 5 highlights the reasons provided by the participants for disabling their web camera during virtual conference meetings.
The participants in the usability study were also asked when they would be willing to turn on the web camera. It was discovered that participants neither enabled nor disabled the web camera continuously. The participants stated that they usually enabled the web camera “when being asked”, which rated the highest. The second most important factor influencing whether they switched on the web cameras was having to “give a presentation”, followed by “performing group activity”. Figure 6 shows the results of different situations in which participants were willing to turn on their web cameras during a video conference meeting.
The usability study also asked participants how they felt when other group members’ web cameras were turned off while performing collaborative activities. The majority of the participants (69%) were neutral and impartial, meaning it did not matter to them if the web camera was switched on or off by other group members. Around 20% of the participants associated the web cameras being turned off with negative feelings, with 13% describing a sense of loneliness and isolation and another 6% describing feelings of boredom and demotivation. On the contrary, the remaining 12% of the participants reported feeling more comfortable and relaxed (less anxious) when their peers turned off the camera. Figure 7 depicts the breakdown of the feelings experienced by the participants when the web cameras of their peers were turned off during a virtual meeting.

4.4. Collaborative Experience and System Usability Scale (SUS)

The SUS was used to investigate the collaborative experience of the participants when using the virtual conference application, as well as the usability of the application. Overall, the participants rated the collaborative experience to be good, (µ = 4.27). Figure 8 shows the distribution of collaborative experience gained through the use of a virtual conference application.
The SUS also reveals that the current virtual conference application received an average total score of sixty-three (63), indicating that the score is average [55]. Two items that the participants had rated poorly were brought to light through the SUS: (1) it was difficult to learn how to use the current virtual conference application quickly due to the many features, and (2) it was time-consuming. Figure 9 depicts a diverging stacked bar chart for the SUS of the virtual collaborative application when collaborative activities are completed.

5. Discussion

According to the findings of the study, users generally viewed that current synchronous applications are better, compared to asynchronous applications in terms of collaborative experiences and system usability. However, some features in the application were found to be underutilized, such as the record feature, the reactions feature, the raise hand feature, the breakout room feature, the file sharing feature, the whiteboard feature, the chat feature, the annotation feature, and the camera feature. These features were essential for collaborative purposes, but participants did not use them frequently enough. From the study, users believed that the file sharing, whiteboard, and annotation features were not easy to use, which perhaps contributed to their underutilization.
The study also gathered feedback on the web camera usage during the virtual conferences. Most of the participants preferred to turn off their web cameras unless specifically instructed not to do so. According to the results of the survey, most of the participants were unconcerned about their other members not turning on their cameras during an online engagement. This was further supported by the findings of the focus group discussion, in which participants discussed the issues they faced while performing collaborative activities on the current virtual conference application. Qualitative data extracted from the transcription of the focus group discussion is labeled using the G[Number]M/F[Number] arrangement, where G[Number] denotes the group number, e.g., Group 1 to 5, M/F denotes the sex of the participants, e.g., Male or Female, and [Number] denotes the participant’s identification number, e.g., 1 to 16. The indented text was the feedback from the participants.

5.1. Underutilization of the Collaborative Tools

The majority of the participants in this study expressed that they had very little experience in drawing collaboratively on their virtual conference platform. Previous work supported this by stating that, despite knowing how to use a video conferencing application, participants had never performed any collaborative task via video conference [56]. Some of the participants in this study found it very interesting to be able to draw collaboratively, as they had not experienced this in their previous virtual conferences:
G2F4:
Actually, I’d only just found out a function like this on Zoom, so it’s interesting. I’d just noticed it.
G2F5:
Haven’t used it before, Um. Interesting.
G3F7:
I think this is a fun experience because it’s my first time, I think.
G1F1:
For me, I also think it’s quite fun and kind of new for me because, uh, we usually draw by ourselves right, and suddenly someone comes in and draw together with us, so it’s kind of a fun experience and new for me.
However, being unfamiliar with the features had also resulted in some challenges and frustrations, as observed with some of the participants:
G3F7:
How do you join the drawing? Where is it (the feature)?
G3F8:
Can you see what we are drawing right now?
G3F7:
Yes, I can see it, but where can I (click to) draw on it?
G3M9:
It’s at the top. It’s under ‘View Option’.
G3F8:
Annotation.
G3F7:
Oh okay. (A join the drawing)
Participants also had difficulty activating the annotate feature from drawing, as it was designed to be hidden within a menu. As a result, the participants were unable to locate the functions they desired to use, and there was no indication whether the function had been activated or not. The following excerpt highlights a conversation during the usability study, where a participant in Group 5 was trying to find the draw function:
G5M14:
I want to change to go back to the drawing.
G5M15:
Oh, uh you click on the draw right click on the draw and then it will go back to the drawing yeah.
G5M14:
Where is the draw? The draw okay yeah. (G5M14 searches through all the functions by clicking the menu of the annotate function) I’m still in the erasing icon, what’s this. (G5M14 is still figuring out where is the draw menu)
According to Nielsen Norman Group, the menus such as the hamburger or the hidden menu can degrade the user experience [57]. In their studies, the hidden menus can reduce the functions’ discovery time by half, with longer task time and an increase in the perceived task difficulty. It is more obvious in our usability study, as the participants find it difficult to locate the required function, therefore increasing task time and perceived task difficulty. Hence, it is evident that there is a clear need to redesign a more intuitive interface for collaborative purposes in virtual conference applications. A previous study presented several ways to improve the usability of the virtual conference application through an immersive interface [58]. However, using virtual reality and augmented reality technologies to improve usability can be costly and difficult to implement. A simpler solution must be devised to address this issue.

5.2. Connectivity and Other Technical Issues

Another issue discovered during the study was related to connectivity and other technical issues, such as poor quality of internet connection. Unstable and slow internet connection results in the users’ disconnection from the group. Moreover, disconnection can result in the loss of any collaborative work that the group has already completed:
G3F7:
Wait, is Windows not responding?
G3F8:
Okay, can you erase?
G3F7:
Oh, not responding.
G3F8:
Oh, it’s loading. Do we need to change color?
G3M9:
Where is it? Oh, let’s change the color.
G3F7:
Yes, I think it is loading, isn’t it? Why is it so lagging?
G3M9:
I think so, G3F7 disconnected just now.
(A notification message to confirm disconnection appeared on the screen and all the whiteboard content disappeared)
Previous work also demonstrated that the participants encountered the freezing problem due to an unstable internet connection [59]. During an interview session with the participants in the usability study, they mentioned that, since their internet was not stable, they turned off their cameras while interacting to reduce the bandwidth consumption in order to maintain a more stable connection:
G3F7:
Because when we do the usual class, if all students switched on their cameras, the internet will be slower, it would freeze or lag a lot.
G3F8:
Sometimes, Zoom takes a lot of internet bandwidth if we switched on the camera and it will always cause disconnection.
G5M16:
The biggest reason is that I’m not sure when will I be (accidentally) disconnected so I’m trying to reduce the numbers, uh, data that transfer in between. Yeah so that’s the biggest reason I decide not to show myself to others.
Some participants also expressed their frustration with technical issues, such as action delay. For instance, it took a few seconds for their drawing to appear on the screens of others due to poor internet connection. This could be because the virtual conference application was not completely in real time:
G3M9:
Yes, because of the internet connection, we cannot all talk at the same time. So, there is going to be lag issues. Sometimes, I’d thought that they could not hear me because of the internet.
G3F7:
Same… it is difficult because of the unstable network connection.
G3F8:
And, sometimes it’s not real time. They’re talking about something else, and then suddenly we interact about something else. So, it’s a bit difficult.
Some participants mentioned that virtual conference applications for collaborative activities consume a lot of data. The participants admitted that the more fun an activity is, the more data it consumes. This situation was not ideal because most participants had limited data plans from their internet service provider:
G1F1:
For me it’s fun to do this activity, but if we are using Zoom platform, it uses a lot of data yeah so, I don’t really like.
G1F2:
Because if we are using Zoom, it actually needs a lot of Internet (data) consumption.
Based on a previous survey, data usage via virtual conference applications can be very expensive [60]. The reason for this is the high bandwidth consumption required by video conferences to transmit video and audio data across the internet. A previous study using video conferencing also encountered technical and connectivity issues such as having no audio or video feed, software incompatibility, and sometimes being unable to connect to the internet [61]. Previous research showed that disabling video feed and using only audio in rural areas with high internet latency can greatly improve the user experience [62].

5.3. Limited Sharing of Emotions and Expressions

Most of the participants in the usability study preferred to turn off their web camera during online communication due to several reasons such as privacy, a less-than-ideal background environment, and a lack of confidence or low self-esteem. Some users stated that they preferred their web cameras to be switched off by default in adherence to the Islamic teachings that require Muslims to dress modestly in public:
G5M15:
I just like it like that, maybe due to privacy because when we do something, like when we draw, our faces look too focused, and we don’t want other people to look at your face at that time. I don’t know, for me, I just like the privacy and since others have also not turned on their cameras, so I choose to not (turn on my camera) too.
G1F1:
Because we are Muslims, so we need to wear a hijab for that.
Another reason is the surrounding environment, which may not be suitable for sharing with others. One of the participants performed the study outside, therefore he decided not to turn on the web camera.
G1F2:
I’m outside, so my line is not so good. That’s why I just closed the video.
Interestingly, a few participants from the study also mentioned that they felt more relaxed when their web cameras were turned off. They had associated being shy with making them feel more anxious when the camera was turned on.
G4F10:
(I) feel more relaxed if the camera is not switched on.
G4F11:
Yeah, we are shy.
G4F10:
Oh, I just feel more relaxed we don’t switch on the camera, but it’s okay if it’s necessary, to switch it on, uh, when the lecturer requests.
G4F11:
Yeah, when we are requested upon.
However, some participants expressed that they preferred to turn on the web camera while doing the collaborative work in order to see how others reacted.
G3M9:
I also love to switch on the camera because it’s more fun because we can see their reactions and so on.
Thus, when users turn off their web camera during an online interaction, they are unable to share and communicate affective feedback with one another. Due to this isolation, the results from the usability study earlier show that most of the participants feel neutral or lonely when performing collaborative sync communication. Therefore, a balanced solution is required that can express the affective feedback of all parties involved, while also respecting the wishes of others who value their privacy highly.
According to a previous study, nearly half of the students did not agree to keep their web cameras on during classes due to anxiety, shyness, and privacy concerns [63]. Currently, the research has highlighted a lack of study on the causes of this behavior and would like to further investigate the matter in the future. Several theories relating to shyness and sociability (the likeliness of users to be socially engaged with others) are discussed in this article [64]. Although some users prefer privacy due to shyness, anxiety, or to increase focus, others still rely on visual cues such as facial expressions and hand gestures to help them interpret the message. Previous research has suggested that a lack of non-verbal and visual cues may increase the risk of clients and therapists misinterpreting what is communicated [65].
One possible solution for sharing non-verbal cues without jeopardizing privacy is by replacing the live video feed with virtual forms of a person, such as through an avatar. Previous studies have shown that avatars can help improve subsections of social presence such as co-presence and behavioral interdependence [66,67]. However, using avatars for this purpose has some limitations because they can only capture and display a limited range of emotions and expressions. Previous work has also shown that lower graphical detail on the avatar further limits the expressions that can be shown, and in turn lowers the emotional and affective states [66]. Basic graphics in avatars also present a trust issue with users, with previous results showing that users have a higher level of trust in the avatar when a more realistic avatar is used, as compared to basic-looking avatars [67,68,69].

5.4. Background Distractions

One recurrent problem when users try to collaborate synchronously online is the inevitable background noises and visuals that interrupt their activities. The background noise can be subtle or too loud to ignore. In most cases, participants had to repeat their sentences due to their voice being muffled by the background noises:
G2F4:
I can draw using color, there is the box where the color located (There is some background noise behind the scene)
G2F5:
I use the box at the top, number 2 (Background noise—C draw orange rectangle, B draw blue rectangle) (G2F4 distracted by his little brother)
Sometimes, participants were distracted by something else while performing collaborative tasks on the virtual conference application. From the observation, one of the participants in Group 3 was interrupted while doing the collaborative activity. Distractions can also take the form of an unwelcome presence in the background:
G3F7:
Please wait for a moment. (Others were coloring the sea while G3F7 is being interrupted by talking to someone behind the background).
G3F7:
Can you move over? (G3F7 was letting someone to move across behind the scene while the web camera was disable)
G4F10:
There might be distractions from brothers, or like maybe their parents always come into the room without knocking, without realizing they’re in class, so that’s why.
Distractions can occur in the virtual world as well, such as when the default layout of the platform is perceived to be cluttered and interfering with the flow of the tasks.
G5F13:
Okay, I don’t think that turning on the camera is, uh, it’s not, it’s not, it’s unnecessary because I want the whiteboard to be full screen. I think having the, having the video on the side might clutter the experience.
Another research study found that distraction can come from background sound, background video movement, or an unwanted presence [70]. Other factors that may cause distractions include a poor camera angle, bad lighting, walking while holding the camera, behavioral distraction (eating in front of the camera), and background noises [68]. There are also issues due to the home environment, such as seeing yourself on screen and other forms of interruption, which can be distracting.
Previous research has suggested that using a larger face and reducing background visibility could help reduce distraction [70]. The larger face can help reduce background distraction by allowing users to focus more on its facial expression. The previous study designed its system to be as simple as possible to minimize distraction when performing video annotation [71]. That is, by reducing the number of features available on the screen, simplifying the icons, and reducing background visuals and noise in the video feed.

6. Design Guidelines and Recommendations

The result and discussion of this user experience study serve as the basis for a set of design recommendations that will be useful in the future when designing virtual conference applications for collaborative purposes. According to the findings of the user experience study, there is a need for an intuitive interface to help increase the usage and ease of use of such synchronous, collaborative tools. For instance, the interface should be designed to be as close as possible to the real-world settings of a meeting or collaborative site. This would put the participants at ease as they would recognize the virtual location as being somewhat familiar to the real-world meeting site. Likewise, the tools and additional features provided by the synchronous, collaborative virtual conference application should also have been designed to encourage exploration when engaging in online collaboration activities with others.
Another key finding from this study is the need to provide for an alternative method of representing facial expressions when communicating online. Although respondents stated that they were mostly shy or reluctant to switch on their cameras and show themselves (and their backgrounds) to others, they had also stated that they would appreciate the facial cues and reading responses from others. This study proposes the use of an avatar to achieve this. Avatars use less bandwidth to display expression than a live video feed from a web camera, which consumes a lot of data. Avatars also help to improve privacy as they are less intrusive compared to using a web camera. The virtual character should not be too distracting and can be disabled if users want to. This could help increase user engagement and awareness, as the virtual avatar can share expressions without having to show the user’s actual face or background.

6.1. Proposed Architecture System Design

The overall architecture of the proposed natural and affective virtual conference application is illustrated in Figure 10. The proposed system is made up of both local and remote clients. A server is also included in the system to connect multiple local and remote clients. For the local client, the web camera captures the video as an input for the virtual conference application. For privacy and data management, the local user can control the input through the input control layer by enabling, switching, and disabling. The video source data is then used to track facial expressions and hand gestures through image preprocessing. Once the facial expressions and gestures have been detected, the system then identifies the correct actions to produce based on the expressions and gestures input, which is done through the affect and action module, respectively. The affect module determines the users’ facial expression, while the action module determines the users’ gestures. The affect and action data are packaged and ready to be sent to the server through a real time network layer. The real time network layer connects to a cloud server and then transfers the affect and action data from the local client to the server. The server then distributes the affect and action data to all remote clients that connect to it.
At the remote client, the affect and action data received from the server are then interpreted and sent to the avatar and action controller, respectively. The avatar controller manages and displays the avatar animation by showing real-time emotion based on the affect data received from the local client. The action controller displays actions such as the raising of one’s hand, waving one’s hand, and signing a thumbs up. The actions are hovered on the avatar based on the action data received from the local client. The avatar and actions are then displayed on the user interface. Finally, remote users can see other users’ facial expressions and actions through the avatar without revealing their own faces to others. The avatar partially substitutes the need for a public video display while still allowing users to share affects and actions.
Figure 11 depicts the architecture flow process of the proposed affective virtual conference application for remote collaboration. Figure 11a shows the flow process for determining emotion based on the facial expressions of users. To detect a facial expression, a face tracking algorithm based on previous research must be implemented [72]. Similar to the previous flow process, a frame needs to be captured and image preprocessing must be performed. The image preprocessing process includes converting an image to grayscale and removing the background. Then, the image must be adjusted through pose correction to straighten out the face. If a face is detected, the image extracts the facial features of the face into three main cues: the eyes, face, and eyebrows. If no errors are found, the data from the feature extractions are used in the Support Vector Machine (SVM) to classify the facial expression. Using the provided training dataset, the system helps to classify different types of emotion such as happy, sad, angry, neutral, excited, curious, and more. This information is then used by the avatar module to help the avatar express emotion. However, only one expression is displayed at a time.
On the other hand, Figure 11b depicts the flow process of providing action based on hand gestures provided by users. For hand tracking, the proposed virtual conference application for remote collaboration makes use of some of the techniques developed by a previous study [73]. The flow starts by capturing a frame from a video source; then, the frame image is processed by converting it to grayscale, performing background subtraction, Gaussian blurring, and image thresholding. After that, the system extracts a contour and convex hull from the processed image. The system then proceeds to detect convexity defects using the processed image. Based on the number of convex hulls and defects, this information can help determine whether the image processed contains the hand or not. If the system fails to detect the hand, it will return to capture another frame and the process is repeated. After detecting the hand, the system uses a simple algorithm to determine the number of convex hulls and defects. If the gesture cannot be determined by the algorithm, the system employs machine learning using Support Vector Machine (SVM) with a training dataset to try to recognize the gesture so that it can be accurate enough to detect different types of gesture. When the gesture is successfully determined, the system proceeds to find the appropriate actions. Finally, a virtual action, such as thumbs up, raising hand, or clapping, can be displayed on the screen.

6.2. Proposed User Interface Design

The proposed user interface design for the proposed virtual conference application is shown in Figure 12. The user is required to enter his or her name, the group name that will be created or joined, and the group password that will be set or entered into the group. After entering their information, users can choose to join or create a group by clicking on the Join Group or Create Group button. When a user clicks the Create Group button, the proposed virtual conference application creates a new empty group with the group name and password set by the user. When the user clicks Join Group, he or she enters the group based on the group name and password provided by the users who created the group. When the user enters a group, there are multiple functions that can be accessed by the users. The top left corner section contains multiple buttons for annotating, such as sketching, erasing, and coloring. There are also some application functions such as screen sharing, recording, video mode, and microphone control. At the top right corner is the settings menu, where there are options to invite, leave, or edit group settings. The chat message between group members is displayed in the bottom left corner. To highlight actions taken by the users, a yellow color is displayed for each action. The bottom-right corner is used to manage group members, such as muting them.
The most prominent feature of this proposed virtual conference application when compared to others is the viewing mode, which allows users to switch between avatar, video, or off mode. When avatar mode is turned on, the avatar appears at the bottom of the screen, depending on the number of users who have joined. The avatar is able to express itself based on the user’s facial expression. When the user switches to video mode, the avatar is replaced with a web camera video stream from their computer. The application also allows the user to turn off both video and avatar mode if the user wants more privacy.
Another feature that differentiates this system from others is that it provides actions cues such as hand up and love when a certain gesture is performed. Users will become much more aware as the actions show up beside the avatar. A text message is also displayed on top of the avatar to also help improve awareness. This can be helpful for keeping track of the current message chat. The proposed virtual conference application also improves awareness by displaying color-coded name cues while annotating on the whiteboard or screen sharing. This is used to indicate who is annotating, which can be useful during collaboration.

7. Conclusions and Future Works

In this paper, a usability study was carried out to determine the issues faced by users while performing collaborative activities on current virtual conference applications. Our findings show that most of the features offered by existing applications are not fully utilized. It was also discovered that users found common collaborative features such as file sharing, whiteboard, and annotation features difficult to use. A simpler, more intuitive, interface design is needed to better assist users. The study also revealed that most participants would prefer to turn off the web camera for reasons of privacy, internet instability, and reducing mobile data consumption. Participants, on the other hand, rated positively on the collaborative experience and system usability as acceptable.
Following that, the paper described a set of design guidelines to address the issues raised. This includes ways to encourage collaborative usage by redesigning the interface to look more like the real world. Other solutions included providing avatar-based virtual characters that consume less data, improve privacy, and increase social awareness among participants.
One of the goals of this study is to understand more about human-to-human distance communication through technology, especially during the ongoing COVID-19 pandemic. The study also contributes to the research community by providing guidelines for improving collaboration and affective sharing, which are often lacking in current virtual conference applications.
One of the study’s limitations was that it only tested one application (Zoom) for the experiment. It would be interesting to compare different types of virtual conference applications to determine the issues and challenges. Another limitation is that the study did not consider participants with different backgrounds for the experiments due to the pandemic and the consequential restriction of movement control order. An extension of the study will be conducted, and participants from different backgrounds will be considered to gain a better representation.
The next step is to redesign the interface of a virtual conference application to make it more user-friendly, thereby encouraging collaboration among users. An avatar-based solution to increase social awareness and social presence of group members is also in the works, where affective qualities can be communicated among group members without the need to publicly enable the web camera. In the near future, this study will include log recordings in addition to the self-reporting that is currently in place. This should provide a better insight to further understand the behavior of users when performing collaborative activities online in a more natural setting.

Author Contributions

Project administration, T.R.M.; formal analysis, T.R.M.; data curation, T.R.M.; methodology, T.R.M.; investigation, T.R.M.; writing—original draft preparation, T.R.M.; Conceptualization, N.M.N.; writing—review and editing, N.M.N. and A.K.; Funding acquisition, N.M.N.; supervision, N.M.N., R.W. and A.K.; validation, R.W. and A.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Universiti Putra Malaysia under the High Impact Putra Grant project code GP/2017/9549600 and the Faculty of Computer Science and Information Technology Trust Fund code 6211200.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

The authors would like to express their great appreciation to the respondents who voluntarily took part in this study, as well as the students and researchers at the Human-Computer Interaction Lab, Faculty of Computer Science and Information Technology, UPM, for providing valuable editing and writing advice.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Felder, J. The Rise of the Video Call: How COVID-19 Has Changed the Way We Collaborate. Available online: https://www.uctoday.com/collaboration/video-conferencing/the-rise-of-the-video-call-how-covid-19-has-changed-the-way-we-collaborate/ (accessed on 18 February 2021).
  2. Malim, G. Poor Audio Quality the Biggest Frustration for Virtual Meeting Participants. Available online: https://www.uctoday.com/collaboration/poor-audio-quality-the-biggest-frustration-for-virtual-meeting-participants/ (accessed on 18 February 2021).
  3. Junuzovic, S.; Inkpen, K.; Blank, T.; Gupta, A. IllumiShare: Sharing any Surface. In Proceedings of the CHI’ 06: SIGCHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 22–27 April 2006; Association for Computing Machinery: New York, NY, USA, 2012; pp. 1919–1928. [Google Scholar] [CrossRef]
  4. Pejsa, T.; Kantor, J.; Benko, H.; Ofek, E.; Wilson, A. Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, San Francisco, CA, USA, 26 February–2 March 2016; Association for Computing Machinery: New York, NY, USA, 2016; Volume 27, pp. 1716–1725. [Google Scholar] [CrossRef]
  5. Khan, M.S.; Li, H.; ur Réhman, S. Postprint Tele-Immersion: Virtual Reality Based. In Proceedings of the 18th International Conference, HCI International 2016, Toronto, ON, Canada, 17–22 July 2016; pp. 352–357. [Google Scholar]
  6. Teo, T.; Lawrence, L.; Lee, G.A.; Billinghurst, M.; Adcock, M. Mixed Reality Remote Collaboration Combining 360 Video and 3D Reconstruction. In Proceedings of the CHI ‘19: 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK, 4–7 May 2019; Association for Computing Machinery: New York, NY, USA, 2019; pp. 1–14. [Google Scholar] [CrossRef]
  7. Lee, G.A.; Teo, T.; Kim, S.; Billinghurst, M. A User Study on MR Remote Collaboration Using Live 360 Video. In Proceedings of the 2018 IEEE International Symposium on Mixed Augmented Realality ISMAR 2018, Munich, Germany, 16–20 October 2019; pp. 153–164. [Google Scholar] [CrossRef]
  8. Nabben, B.; Nabben, B.J.T. The Impact of Groupware and CSCW on Group Collaboration: An Overview of Success Factors in CSCW. In Proceedings of the International Workshop on Cultures of Participation in the Digital Age, Turku, Finland, 20 June 2019. [Google Scholar]
  9. Pedersen, G.; Koumaditis, K. Virtual Reality (VR) in the Computer Supported Cooperative Work (CSCW) Domain: A Mapping and a Pre-study on Functionality and Immersion. In Proceedings of the International Conference on Human-Computer Interaction, Cham, Germany, 10 July 2020; Volume 12191, pp. 136–153. [Google Scholar] [CrossRef]
  10. Petrosyan, L. Video Conference Application. Available online: https://commons.wikimedia.org/wiki/File:GTM2020-2.png (accessed on 18 July 2021).
  11. Wolfyxvf Email Application. Available online: https://commons.wikimedia.org/wiki/File:Zimbra_email.jpg (accessed on 18 July 2021).
  12. Wikimedia. Microsoft Surface Table. Available online: https://commons.wikimedia.org/wiki/File:Surface_table.JPG (accessed on 18 July 2021).
  13. Zeorcher KIOSK Machine. Available online: https://commons.wikimedia.org/wiki/File:KIOSK_machine.png (accessed on 18 July 2021).
  14. Ming, T.R.; Liang, O.B.; Norowi, N.M.; Mansor, E.I.; Dey, P.; Jamil, I. Gaming Experience On The Mobitop System: A Collocated Ad-Hoc Collaboration System. J. Teknol. 2016, 78. [Google Scholar] [CrossRef] [Green Version]
  15. Rhun Ming, T. A Collocated Multi-Mobile Collaborative System with Hover Connectivity Initiation and Seamless Multi-Touch Interactivity. Master’s Thesis, Universiti Putra Malaysia, Selangor, Malaysia, 3 April 2018. [Google Scholar]
  16. St. Germain, S. Use of collaborative software to improve Nuclear Power Plant outage management. In Proceedings of the 9th International Topical Meeting on Nuclear Plant Instrumentation, Control, and Human Machine Interface Technologies (NPIC&HMIT 2015), Charlotte, NC, USA, 22–26 February 2015; Volume 1, pp. 608–615. [Google Scholar]
  17. Chia, F.Y.; Lee, B.; Saakes, D. Collaboration in the living room or how couples design together. In Proceedings of the NordiCHI ’16: 9th Nordic Conference on Human-Computer Interaction, Gothenburg, Sweden, 23–27 October 2016. [Google Scholar] [CrossRef]
  18. Chen, S.Y.; Chang, L.-P. The influences of cognitive styles on individual learning and collaborative learning. Innov. Educ. Teach. Int. 2016, 53, 458–471. [Google Scholar] [CrossRef]
  19. Ku, H.Y.; Tseng, H.W.; Akarasriworn, C. Collaboration factors, teamwork satisfaction, and student attitudes toward online collaborative learning. Comput. Human Behav. 2013, 29, 922–929. [Google Scholar] [CrossRef]
  20. Lei, M.; Medwell, J. Impact of the COVID-19 pandemic on student teachers: How the shift to online collaborative learning affects student teachers’ learning and future teaching in a Chinese context. Asia Pac. Educ. Rev. 2021, 1, 3. [Google Scholar] [CrossRef]
  21. Constantin, A.; Alexandru, C.; Korte, J.; Wilson, C.; Fails, J.A.; Sim, G.; Read, J.C.; Eriksson, E. Distributing Participation in Design: Addressing Challenges of a Global Pandemic. Int. J. Child-Comput. Interact. 2021, 28, 100255. [Google Scholar] [CrossRef]
  22. Hu, W.-C.; Bansal, B.; Kaabouch, N. Using Mobile and Wireless Computing to Facilitate Virtual Collaboration during a Pandemic. J. Syst. Cybern. Infromatics 2020, 8, 273–288. [Google Scholar]
  23. Woubie, A.; Zarazaga, P.P.; Bäckström, T. Users Perceptions about Teleconferencing Applications Collected through Twitter. arXiv 2020, arXiv:2010.09488. [Google Scholar]
  24. Kjeldskov, J.; Smedegård, J.H.; Nielsen, T.S.; Skov, M.B.; Paay, J. EyeGaze: Enabling eye contact over video. In Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces—AVI ’14, Como, Italy, 27–30 May 2014; pp. 105–112. [Google Scholar]
  25. Cutler, R.; Mehran, R.; Johnson, S.; Zhang, C.; Kirk, A.; Whyte, O.; Kowdle, A. Multimodal active speaker detection and virtual cinematography for video conferencing. In Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 4–8 May 2020. [Google Scholar]
  26. Meluso, J.; Johnson, S.; Bagrow, J. Making Virtual Teams Work Redesigning Virtual Collaboration for the Future. SocArXiv 2020, 1–14. [Google Scholar] [CrossRef]
  27. Zoom—Zoom. Zoom Video Conferencing Plans & Pricing. Available online: https://zoom.us/pricing (accessed on 27 May 2021).
  28. Webex.com. Cisco Webex Plans and Pricing. 2021. Available online: https://www.webex.com/contact-sales.html?TrackID=1096897 (accessed on 8 September 2021).
  29. Blackboard Help. Feature Guide for Blackboard Instructor. Available online: https://help.blackboard.com/Blackboard_Instructor/Feature_Guide (accessed on 27 May 2021).
  30. BigBlueButton Features|Built for Teachers|BigBlueButton. Available online: https://bigbluebutton.org/teachers/features/ (accessed on 27 May 2021).
  31. Teams, M. Compare Microsoft Teams Online Options. Available online: https://www.microsoft.com/en-my/microsoft-teams/compare-microsoft-teams-options (accessed on 27 May 2021).
  32. Google Meet (formerly Hangouts Meet). Pricing. Available online: https://apps.google.com/meet/pricing/ (accessed on 27 May 2021).
  33. Skype. Features|Find Out What Skype Can Do for You. Available online: https://www.skype.com/en/features/ (accessed on 27 May 2021).
  34. Benke, I.; Heinzl, A.; Maedche, A. The Relationship of Collaboration Technologies and Cognition and Affect. In Proceedings of the 54th Hawaii International Conference on System Sciences 2021, Kauai, HI, USA, 5–8 January 2021; pp. 1–11. [Google Scholar]
  35. Chikersal, P.; Tomprou, M.; Kim, Y.J.; Woolley, A.W.; Dabbish, L. Deep Structures of Collaboration. In Proceedings of the CSCW ’17: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, Portland, OR, USA, 25 February–1 March 2017; pp. 873–888. [Google Scholar] [CrossRef]
  36. Mazgaj, M.; Derrick, D. Exploring the Effects of Real-Time Hologram Communication on Social Presence, Novelty and Affect. In Proceedings of the 54th Hawaii International Conference on System Sciences 2021, Kauai, HI, USA, 5–8 January 2021; pp. 471–480. [Google Scholar]
  37. Manghisi, V.M.; Uva, A.E.; Fiorentino, M.; Gattullo, M.; Boccaccio, A.; Monno, G. Enhancing user engagement through the user centric design of a mid-air gesture-based interface for the navigation of virtual-tours in cultural heritage expositions. J. Cult. Herit. 2018, 32, 186–197. [Google Scholar] [CrossRef]
  38. Bai, H.; Sasikumar, P.; Yang, J.; Billinghurst, M. A User Study on Mixed Reality Remote Collaboration with Eye Gaze and Hand Gesture Sharing. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020. [Google Scholar] [CrossRef]
  39. Faucett, H.A.; Lee, M.L.; Carter, S. “I should listen more”: Real-time sensing and feedback of non-verbal communication in video telehealth. Proc. ACM Hum.-Comput. Interact. 2017, 1, 1–19. [Google Scholar] [CrossRef]
  40. Ens, B.; Lanir, J.; Tang, A.; Bateman, S.; Lee, G.; Piumsomboon, T.; Billinghurst, M. Revisiting collaboration through mixed reality: The evolution of groupware. Int. J. Hum. Comput. Stud. 2019, 131, 81–98. [Google Scholar] [CrossRef]
  41. Law, E.L.C.; Roto, V.; Hassenzahl, M.; Vermeeren, A.P.O.S.; Kort, J. Understanding, scoping and defining user experience: A survey approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, MA, USA, 4–9 April 2009; pp. 719–728. [Google Scholar] [CrossRef]
  42. Ahsanullah, S.S.; Kamil, M.; Muzafar, K. Understanding factors influencing User Experience of interactive systems: A literature review. ARPN J. Eng. Appl. Sci. 2015, 10, 18175–18185. [Google Scholar]
  43. Roto, V. User experience building blocks. In Proceedings of the 2nd COST294-MAUSE International Open Workshop, Oslo, Norway, 1 January 2006; pp. 1–5. [Google Scholar]
  44. Mishra, A.; Shukla, A.; Rana, N.P.; Dwivedi, Y.K. From “touch” to a “multisensory” experience: The impact of technology interface and product type on consumer responses. Psychol. Mark. 2020, 1–12. [Google Scholar] [CrossRef]
  45. Hussain, A.; Mkpojiogu, E.O.C.; Musa, J.; Mortada, S. A user experience evaluation of Amazon Kindle mobile application. In AIP Conference Proceedings; AIP Publishing LLC: Melville, NY, USA, 2017; Volume 1891, p. 020060. [Google Scholar] [CrossRef] [Green Version]
  46. Chen, T.; Peng, L.; Jing, B.; Wu, C.; Yang, J.; Cong, G. The impact of the COVID-19 pandemic on user experience with online education platforms in China. Sustainability 2020, 12, 7329. [Google Scholar] [CrossRef]
  47. Pal, D.; Funilkul, S.; Arpnikanondt, C.; Varadarajan, V. User Experience with Smart Voice Assistants: The Accent Perspective Human Factors in Cybersecurity View project User Experience with Smart Voice Assistants: The Accent Perspective. In Proceedings of the 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kanpur, India, 6–9 July 2019. [Google Scholar] [CrossRef]
  48. Van Der Linden, J.; Amadieu, F.; Vayre, E.; Van De Leemput, C. User Experience and Social Influence: A New Perspective for UX Theory. In Proceedings of the International Conference on Human-Computer Interaction, Cham, Germany, 3 July 2019; pp. 98–112. [Google Scholar] [CrossRef] [Green Version]
  49. Følstad, A.; Skjuve, M. Chatbots for customer service: User experience and motivation. In Proceedings of the 1st International Conference on Conversational User Interfaces—CUI ’19, Dublin, Ireland, 22–23 August 2019. [Google Scholar] [CrossRef]
  50. Kelleci, Ö.; Aksoy, N.C. Using Game-Based Virtual Classroom Simulation in Teacher Training: User Experience Research. Simul. Gaming 2021, 52, 204–225. [Google Scholar] [CrossRef]
  51. Thomaschewski, L.; Weyers, B.; Kluge, A. Using usability and user experience scores to design an Augmented Reality-based Ambient Awareness interface to support spatially dispersed teams. In Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, Rome, Italy, 7–9 September 2020; pp. 1–6. [Google Scholar] [CrossRef]
  52. Sasikumar, P.; Gao, L.; Bai, H.; Billinghurst, M. Wearable remotefusion: A mixed reality remote collaboration system with local eye gaze and remote hand gesture sharing. In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality Adjunct, Beijing, China, 10–18 October 2019; pp. 393–394. [Google Scholar] [CrossRef]
  53. Wang, P.; Zhang, S.; Bai, X.; Billinghurst, M.; He, W.; Sun, M.; Chen, Y.; Lv, H.; Ji, H. 2.5DHANDS: A gesture-based MR remote collaborative platform. Int. J. Adv. Manuf. Technol. 2019, 102, 1339–1353. [Google Scholar] [CrossRef]
  54. Anton, D.; Kurillo, G.; Bajcsy, R. User experience and interaction performance in 2D/3D telecollaboration. Futur. Gener. Comput. Syst. 2018, 82, 77–88. [Google Scholar] [CrossRef]
  55. Lewis, J.R.; Sauro, J. Item Benchmarks for the System Usability Scale. J. Usability Stud. 2018, 13, 158–167. [Google Scholar]
  56. Barathan, S. Generating an Efficient Remote Collaboration Environment Using Shared Gaze. Master’s Thesis, University of Canterbury, Christchurch, New Zealand, 1 July 2016. [Google Scholar]
  57. Hamburger Menus and Hidden Navigation Hurt UX Metrics. Available online: https://www.nngroup.com/articles/hamburger-menus/ (accessed on 8 June 2021).
  58. Park, W.; Heo, H.; Park, S.; Kim, J. A study on the presence of immersive user interface in collaborative virtual environments application. Symmetry 2019, 11, 479. [Google Scholar] [CrossRef] [Green Version]
  59. Hussain, A.; Wang, W.; Xu, D.L. Comparing WebRTC video conferencing with Skype in synchronous groupware applications. In Proceedings of the 2017 IEEE 21st International Conference on Computer Supported Cooperative Work in Design, Wellington, New Zealand, 26–28 April 2017; pp. 60–65. [Google Scholar] [CrossRef] [Green Version]
  60. A’yun, K.; Suharso, P.; Kantun, S. Google Classroom as the Online Learning Platform During he Covid-19 Pandemic for the Management Business Student at SMK Negeri 1 Lumajang. In IOP Conference Series: Earth and Environmental Science; IOP Publishing: Bristol, UK, 2021; Volume 747. [Google Scholar] [CrossRef]
  61. Hirschman, K.B.; Bowles, K.H.; Garcia-Gonzalez, L.; Shepard, B.; Walser, T.J.; Thomas, G.L.; Stawnychy, M.A.; Riegel, B. Lessons learned from the implementation of a video health coaching technology intervention to improve self-care of family caregivers of adults with heart failure. Res. Nurs. Health 2021, 44, 250–259. [Google Scholar] [CrossRef] [PubMed]
  62. Essel, H.B.; Vlachopoulos, D.; Adom, D.; Tachie-Menson, A. Transforming higher education in Ghana in times of disruption: Flexible learning in rural communities with high latency internet connectivity. J. Enterprising Communities 2021, 15, 296–312. [Google Scholar] [CrossRef]
  63. Gherheș, V.; Șimon, S.; Para, I. Analysing students’ reasons for keeping their webcams on or off during online classes. Sustainability 2021, 13, 3203. [Google Scholar] [CrossRef]
  64. Chan, M. Shyness, sociability, and the role of media synchronicity in the use of computer-mediated communication for interpersonal communication. Asian J. Soc. Psychol. 2011, 14, 84–90. [Google Scholar] [CrossRef]
  65. Rawlings, G.H.; Gaskell, C.; Rolling, K.; Beail, N. Exploring how to deliver videoconference-mediated psychological therapy to adults with an intellectual disability during the coronavirus pandemic. Adv. Ment. Health Intellect. Disabil. 2021, 15, 20–32. [Google Scholar] [CrossRef]
  66. Heidicker, P.; Langbehn, E.; Steinicke, F. Influence of avatar appearance on presence in social VR. In Proceedings of the 2017 IEEE Symposium on 3D User Interfaces (3DUI), Los Angeles, CA, USA, 18–19 March 2017; pp. 233–234. [Google Scholar] [CrossRef]
  67. Jo, D.; Kim, K.; Kim, G.J. Effects of Avatar and Background Types on Users’ Co-presence and Trust for Mixed Reality-Based Teleconference Systems. In Proceedings of the 30th Conference on Computer Animation and Social Agents, Seoul, South Korea, 22–24 May 2017; pp. 27–36. [Google Scholar]
  68. Campion, S.; Landowska, A.; Duckworth, T.; Moore, C.; Eachus, P.; Roberts, D. Removing the mask—Do people over trust avatars reconstructed from video? In Proceedings of the International Conference on Virtual Reality and Augmented Reality, Cham, Germany, 7 December 2017; Volume 10700 LNCS, pp. 134–152. [Google Scholar] [CrossRef] [Green Version]
  69. Pan, Y.; Steed, A. The impact of self-avatars on trust and collaboration in shared virtual environments. PLoS ONE 2017, 12, e0189078. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  70. Nelson, S.B.; Jarrahi, M.H. Too much information: Video conferencing for professional communication. Proc. Assoc. Inf. Sci. Technol. 2017, 54, 764–765. [Google Scholar] [CrossRef]
  71. Zhang, T.; El Ali, A.; Wang, C.; Hanjalic, A.; Cesar, P. RCEA: Real-time, Continuous Emotion Annotation for Collecting Precise Mobile Video Ground Truth Labels. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; pp. 1–15. [Google Scholar] [CrossRef]
  72. Svetleff, Z.; Anwar, S. Facial Expression of Emotion Assessment from Video in Adult Second Language Learners. Intern. J. Modern. Eng. Res. 2017, 7, 27–33. [Google Scholar]
  73. Leite, D.Q.; Duarte, J.C.; Neves, L.P.; de Oliveira, J.C.; Giraldi, G.A. Hand gesture recognition from depth and infrared Kinect data for CAVE applications interaction. Multimed. Tools Appl. 2017, 76, 20423–20455. [Google Scholar] [CrossRef]
Futureinternet 13 00253 g001 550
Figure 1. The time space matrix for CSCW: (a) The interface of Video Conferences for different location same time Tabletop System matrix [10]; (b) Example of different location and time using email application [11]. (c) Tabletop System for same location and time matrix [12]; (d) Kiosk System for same location different time matrix [13].
Figure 1. The time space matrix for CSCW: (a) The interface of Video Conferences for different location same time Tabletop System matrix [10]; (b) Example of different location and time using email application [11]. (c) Tabletop System for same location and time matrix [12]; (d) Kiosk System for same location different time matrix [13].
Futureinternet 13 00253 g001
Futureinternet 13 00253 g002 550
Figure 2. Experimental set up and layout of equipment.
Figure 2. Experimental set up and layout of equipment.
Futureinternet 13 00253 g002
Futureinternet 13 00253 g003 550
Figure 3. Distribution of users’ usage frequency of features on virtual conferencing tool.
Figure 3. Distribution of users’ usage frequency of features on virtual conferencing tool.
Futureinternet 13 00253 g003
Futureinternet 13 00253 g004 550
Figure 4. Distribution of user’s perceived ease of use of the features on virtual conferencing tool.
Figure 4. Distribution of user’s perceived ease of use of the features on virtual conferencing tool.
Futureinternet 13 00253 g004
Futureinternet 13 00253 g005 550
Figure 5. Top reasons for disabling the web camera during virtual conferences.
Figure 5. Top reasons for disabling the web camera during virtual conferences.
Futureinternet 13 00253 g005
Futureinternet 13 00253 g006 550
Figure 6. Bar chart of web camera usage during a video conference meeting.
Figure 6. Bar chart of web camera usage during a video conference meeting.
Futureinternet 13 00253 g006
Futureinternet 13 00253 g007 550
Figure 7. The breakdown of emotional feedback when peers disabled their web cameras during a virtual conference meeting.
Figure 7. The breakdown of emotional feedback when peers disabled their web cameras during a virtual conference meeting.
Futureinternet 13 00253 g007
Futureinternet 13 00253 g008 550
Figure 8. Distribution of users’ prior collaborative experience.
Figure 8. Distribution of users’ prior collaborative experience.
Futureinternet 13 00253 g008
Futureinternet 13 00253 g009 550
Figure 9. The system usability score (SUS) of the current virtual conference application.
Figure 9. The system usability score (SUS) of the current virtual conference application.
Futureinternet 13 00253 g009
Futureinternet 13 00253 g010 550
Figure 10. The proposed natural and affective virtual conference application.
Figure 10. The proposed natural and affective virtual conference application.
Futureinternet 13 00253 g010
Futureinternet 13 00253 g011 550
Figure 11. This is the flow process of the proposed natural and affective virtual conference application: (a) Flow process of detecting the facial expressions of the user and displaying it onto the avatar; (b) Flow process of detecting hand gesture used by the users and displaying as an action cues.
Figure 11. This is the flow process of the proposed natural and affective virtual conference application: (a) Flow process of detecting the facial expressions of the user and displaying it onto the avatar; (b) Flow process of detecting hand gesture used by the users and displaying as an action cues.
Futureinternet 13 00253 g011
Futureinternet 13 00253 g012 550
Figure 12. The user interface of the proposed virtual conference application for remote collaboration when users enter the group.
Figure 12. The user interface of the proposed virtual conference application for remote collaboration when users enter the group.
Futureinternet 13 00253 g012
Table
Table 1. The features provided for each video conference.
Table 1. The features provided for each video conference.
Online AppsWhite BoardHand RaiseRoom BreakoutAnnotateBrowser BasedHost Control
Zoom [27]YesYesYesYesNoYes
WebEx [28]YesYesYesYesYesYes
Blackboard [29]YesYesYesYesYesYes
Big Blue Button [30]YesYesYesYesYesYes
Microsoft Team [31]YesYesYesNoYesYes
Google Meet [32]YesYesNoNoYesNo
Skype [33]NoNoNoNoNoNo
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Ming, T.R.; Norowi, N.M.; Wirza, R.; Kamaruddin, A. Designing a Collaborative Virtual Conference Application: Challenges, Requirements and Guidelines. Future Internet 2021, 13, 253. https://doi.org/10.3390/fi13100253

AMA Style

Ming TR, Norowi NM, Wirza R, Kamaruddin A. Designing a Collaborative Virtual Conference Application: Challenges, Requirements and Guidelines. Future Internet. 2021; 13(10):253. https://doi.org/10.3390/fi13100253

Chicago/Turabian Style

Ming, Teo Rhun, Noris Mohd Norowi, Rahmita Wirza, and Azrina Kamaruddin. 2021. "Designing a Collaborative Virtual Conference Application: Challenges, Requirements and Guidelines" Future Internet 13, no. 10: 253. https://doi.org/10.3390/fi13100253

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop