Keywords

1 Introduction

The virtual environments have gradually been used universally. It is hard to find directions towards a destination in an unfamiliar virtual environment. There is a growing research demand on visual aids to ensure the users’ engagement in virtual environments. Many virtual environment applications use an overview interface showing a survey of the entire space to preserve users’ sense of position and context. A larger overview interface on a limited screen means a smaller detail interface displaying various details of the information space [2]. The size of the overview interface in virtual environments has not been researched as yet.

Studies have indicated that differences in gender, experience, age and other traits affect users’ performance and subjective experience in virtual environments [4, 10, 13]. Lee et al. [9] suggested that individuals who were familiar with mobile devices performed better in wayfinding with interactive maps. New information related to the usage of mobile devices that stored in users’ long-term memory is easy to understand. Designers of overview interfaces in virtual environments should pay attention to the issue of whether users are familiar with information technology devices.

The purpose of the study is to examine the use of overview interfaces by people with different levels of familiarity with mobile devices, specifically investigating how the size of the overview interface affects users’ performance and subjective experience in a virtual environment.

2 Related Work

Virtual reality technology is used to create virtual environments simulating different kinds of space, such as an architecture, a tourist attraction and a game space. Designers propose solving problems in virtual environments by a map, a grid, nodes, paths, audio cues, and other assistive aids [5, 12]. Overview and detail interfaces can display the entirety and details of an information space [11]. A detail interface is always used to control position at a short distance and undertake diagnostic tasks, while monitoring tasks might benefit from an overview interface [8]. Empirical studies have demonstrated that the overview interface helps increase user efficiency, satisfaction, preference and the sense of position and control [8]. Nevertheless, the authors note that integration of overview and detail interfaces increases the mental and motor efforts required from a user.

Visual variables including size, value, grain, color, orientation, and shape [1] are used to explicitly represent spatial relationships on an overview interface. Earlier studies have evaluated the effectiveness and efficiency of some visual variables of overview interfaces [4, 6]. In some cases, the size of an overview interface can be changed by clicking on a button on the screen. To enlarge the size of the overview interface can attract users’ attention and vary the amount of information that can be displayed at one point in time. However, there have not been much studies on the important factor that may affect users’ performance and mental workload in virtual environments.

The typical mobile devices, such as smartphone and tablet computer, provide the opportunity to move virtual environments from fixed places to more convenient locations. Mobile virtual environment platforms now are equipped with high display resolution, sufficient computing and networking capability. When the technology devices are unfamiliar to the users, it is prone to cause operation errors. With the accumulation of hands-on experience using the devices, there is less demand for the description of system operation and users might feel less pressure to meet performance expectations. Users expect to use familiar interaction techniques instead of unfamiliar ones in 3D interfaces [3]. Effects of users’ experience of using mobile devices on the usage of overview interfaces on a mobile virtual environment platform are still to be confirmed.

3 Methods

3.1 Participants

A total of thirty-two participants (9 men and 23 women) were invited to take part in a wayfinding experiment. Half of the participants used mobile devices for an average of less than six hours (1–6 h) a day. The other half used mobile devices for an average of more than or equal to six hours (6–12 h) a day. All participants used the device of experiment with no problem in basic operation.

All participants were university students, 30 undergraduates and 2 graduate students, between 18 and 24 years old. Seven people used overview interfaces in virtual environments once a week (21.88%). Participants who used overview interfaces in virtual environments less than once a week were 14 people (43.75%). Eleven people did not have experience of using overview interfaces in virtual environments (34.38%).

3.2 Materials and Apparatus

A simple virtual exhibition was created with 3DS Max software by reference to the memorial museum of Liu Zigu, who is a famous Chinese artist (see Fig. 1). The overview interface was created with Photoshop software (see Fig. 2). The experiment operation was configured with Unity 3D game engine. This experiment was conducted on an iPad Air 2 tablet computer with screen size of 9.7 in., resolution of 2048 × 1536 pixels, and iOS 9.7 operating system.

Fig. 1.
figure 1

View from the top to see the whole virtual model set up for this study

Full size image
Fig. 2.
figure 2

The overview and detail interfaces used in this study

Full size image

3.3 Experimental Design

This experiment adopted a two (overview interface size) x two (familiarity with mobile devices) between-subjects design. There were two types of overview interfaces adopted in this study: the larger overview interface (1024 × 1536 pixels) whose size is close to the detail interface, and the smaller overview interface (384 × 512 pixels) whose size is significantly smaller than the detail interface. Participant were randomly assigned to either of two experimental conditions described above. People who spend less than six hours a day and who spend more than or equal to six hours a day using mobile devices were compared experimentally in terms of their performance and experiences. Participants were divided into four groups. Three research questions were addressed:

  • Whether the size of the overview interface can affect users’ performance and subjective experience in a virtual environment?

  • Whether high familiarity with mobile devices can improve users’ performance in a virtual environment?

  • What is the interaction effect between overview interface size and users’ familiarity with mobile devices?

3.4 Procedure

Each participant was asked to complete three tasks with increasing complexities to get them interacting with the virtual exhibition. The time of completing each task was recorded. We asked participants to find virtual objects matching certain criteria (e.g., shape, color, distance, orientation). In the more difficult tasks, participants needed to memorize and compare the information of the space. Participants had to accomplish these search and comparison tasks without help from the experimenter.

After completing all the tasks, participants were required to fill out a questionnaire to gather data regarding their subjective experience in the virtual environment. Mental workload was gathered by NASA task load index (TLX) [7], since subjective experience is crucial to what extent an overview interface will be used. NASA TLX consists of six subscales: mental demand, physical demand, temporal demand, performance, effort and frustration. Lower is better for all measures (1-best; 7-worst).

4 Results

4.1 Analysis of Task Completion Time

The collected data were analyzed using IBM Statistical Package for the Social Sciences (SPSS) software in terms of two-way analysis of variance (ANOVA). Basic descriptive statistics including the means and standard errors regarding task completion time are presented in Table 1. Table 2 presents the ANOVA results of task completion time for each independent variable level.

Table 1. Descriptive statistics of task completion time (s)
Full size table
Table 2. Two-way ANOVA of task completion time
Full size table

The first task was the easiest task, which is to search for a specific object. Table 2 reveals no significant main effect of overview interface size regarding the Task 1 completion time (F = 2.178, p = 0.151 > 0.05). There was a significant main effect of users’ familiarity with mobile devices on the Task 1 completion time (F = 5.467, p = 0.027 < 0.05). The results suggest that participants who spend less than six hours a day using mobile devices (M = 52.614, Sd = 46.251) took significantly more time to complete the first task than those who spend more than or equal to six hours a day (M = 23.659, Sd = 20.583). There was no significant interaction effect between the variables of overview interface size and users’ familiarity with mobile devices on the Task 1 completion time (F = 1.156, p = 0.291 > 0.05).

The second task was a more difficult task in which participants needed to compare information about distance and look up relevant details. As is shown in Table 2, the main effect of overview interface size on the Task 2 completion time was not significant (F = 0.106, p = 0.747 > 0.05). The main effect of users’ familiarity with mobile devices on the Task 2 completion time was also not significant (F = 0.545, p = 0.466 > 0.05). There were no significant main and interaction effects (F = 1.674, p = 0.206 > 0.05) in terms of the Task 2 completion time. It indicates that in the second task, overview interface size and users’ familiarity with mobile devices did not significantly affect user performance, as participants might already be familiar with the virtual environment.

The third task was the most difficult task in which participants needed to compare information about orientation and search for a specific object. A significant main effect of overview interface size on the Task 3 completion time was detected (F = 5.358, p = 0.028 < 0.05). The results suggest that the Task 3 completion time for the larger overview interface (M = 33.956, Sd = 20.593) was significantly longer than that for the smaller overview interface (M = 22.253, Sd = 11.634). Table 2 shows that the main effect of users’ familiarity with mobile devices on the Task 3 completion time was not significant (F = 1.563, p = 0.222 > 0.05). An interaction effect of overview interface size and users’ familiarity with mobile devices was found for the Task 3 completion time (F = 5.266, p = 0.029 < 0.05).

According to Fig. 3, when using the larger overview interface, participants who spend more than or equal to six hours a day using mobile devices took more time (M = 43.684, Sd = 19.619) to complete the third task than those who spend less than six hours a day (M = 24.229, Sd = 17.537). Whereas, when using the smaller overview interface, participants who spend more than or equal to six hours a day using mobile devices took less time (M = 18.387, Sd = 14.005) to complete the third task than those who spend less than six hours a day (M = 24.119, Sd = 8.668). Participants who spend less than six hours a day using mobile devices took quite similar time to complete the third task using both overview interfaces.

Fig. 3.
figure 3

The interaction diagram of overview interface size and users’ familiarity with mobile devices in terms of the Task 3 completion time

Full size image

4.2 Analysis of NASA TLX

The descriptive statistics and two-way ANOVA of NASA TLX questionnaire are shown in Table 3 and Table 4. We analyzed the mental workload on six dimensions with each overview interface and found no significant difference between our tested conditions on the sections of mental demand, physical demand, temporal demand and frustration.

Table 3. Descriptive statistics of NASA TLX
Full size table
Table 4. Two-way ANOVA of NASA TLX
Full size table

Mental Demand.

Participants who spend more than or equal to six hours a day using mobile devices gave the lowest mental demand scores (M = 3.625, Sd = 3.335). Participants who spend less than six hours a day using mobile devices gave the highest mental demand scores (M = 6.000, Sd = 4.556). Participants supporting by the larger overview interface (M = 5.250, Sd = 4.468) and the smaller overview interface (M = 4.375, Sd = 3.822) reported similar mental demand scores.

Physical Demand.

All experimental groups’ reports on physical demanding were rather low. Participants who spend more than or equal to six hours a day using mobile devices gave the lowest physical demand scores (M = 0.125, Sd = 0.363). Participants who spend less than six hours a day using mobile devices gave the highest physical demand scores (M = 0.938, Sd = 2.059). Participants using the larger overview interface (M = 0.813, Sd = 2.062) reported higher physical demand scores than those using the smaller overview interface (M = 0.250, Sd = 0.552).

Temporal Demand.

The smaller overview interface leaded to the lowest temporal demand (M = 5.875, Sd = 4.778). The other conditions were considered to a higher temporal demand.

Performance.

A significant main effect of overview interface size was detected in terms of the performance score on NASA TLX (F = 5.313, p = 0.029 < 0.05). Participants using the smaller overview interface reported the lowest performance scores (M = 2.084, Sd = 3.663). Participants supporting by the larger overview interface reported the highest performance scores (M = 6.104, Sd = 5.893).

Effort.

An interaction effect of overview interface size and users’ familiarity with mobile devices was found on effort score (F = 5.363, p = 0.028 < 0.05). According to Fig. 4, when using the smaller overview interface, participants who spend more than or equal to six hours a day using mobile devices gave significantly lower effort scores (M = 2.375, Sd = 2.292) than those who spend less than six hours a day (M = 6.542, Sd = 4.528). Whereas, when using the larger overview interface, participants who spend more than or equal to six hours a day using mobile devices gave higher effort scores (M = 6.750, Sd = 6.031) than those who spend less than six hours a day (M = 3.749, Sd = 3.808).

Fig. 4.
figure 4

The interaction diagram of overview interface size and users’ familiarity with mobile devices in terms of the effort score on NASA TLX

Full size image

Frustration.

Participants reported low frustration scores in all conditions. Participants using the smaller overview interface gave the lowest frustration scores (M = 0.896, Sd = 1.298). Participants using the larger overview interface reported the highest frustration scores (M = 2.333, Sd = 3.445).

Sum.

We also analyzed data of the sum score on NASA TLX. Physical demand and frustration were less weighted contributing factors than the others. There was a significant difference between the larger overview interface and the smaller overview interface (F = 5.643, p = 0.025 < 0.05). As is shown in Table 3, participants using the larger overview interface reported the highest overall workload (M = 27.50, SD = 10.64). Users of the smaller overview interface reported the least overall workload (M = 17.94, SD = 12.383).

5 Discussion

5.1 Size Effects of Overview Interfaces

The results of task completion time indicate that the size of the overview interface can affect user performance in a virtual environment. Participants using the smaller overview interface performed significantly better in the most difficult search and comparison task. This might be because the smaller overview interface, by displaying more virtual objects, can help participants acquire more information and then reduce time spent at the task of finding and comparing information. This may indicate that participants did not take much time to integrate information on the overview and detail interfaces.

The results from NASA TLX suggest that the size of the overview interface has a significant influence on subjective experience in a virtual environment. Users with the smaller overview interface experienced significantly less overall workload and were more satisfied with their performance. For each measurement of NASA TLX, participants using the larger overview interface gave higher scores. Users’ subjective experience with the overview interface are consistent with the results generated from their performance in the most difficult task. Most probably, the smaller overview interface means a larger detail interface providing more information to make users feel easier to know the environment well.

Overall, measured by task completion time and NASA TLX, the smaller overview interface is a better visual aid than the larger overview interface in virtual environments.

5.2 Familiarity with Mobile Devices and the Usage of Overview Interfaces

The results of this study indicate some support for the role of users’ familiarity with mobile devices in virtual environments. Participants who were more familiar with mobile devices performed significantly better than those who were less familiar with mobile devices in their first visit to the unfamiliar virtual environment. It is consistent with the results presented by Lee et al. [9] where participants who were more familiar with mobile devices performed better using the interactive maps. This might be because users unfamiliar with the device first need to know the method of operation. This may indicate that the experience of using mobile devices reduces the temporal demand to understand system usage and become familiar with a new environment on a mobile device.

From the experimental results of task completion time and the effort score on NASA TLX, we found that the impacts of overview interface size vary with participants’ familiarity with mobile devices. In the condition of the larger overview interface, participants who were more familiar with mobile devices performed worse in the most difficult task and felt like they have to work harder to accomplish their level of performance. The opposite results from the condition of the smaller overview interface show that participants who were less familiar with mobile devices performed worse in the most difficult task and felt like they have to work harder to accomplish their level of performance. Subjective experience of participants who were more familiar with mobile devices were consistent with the results generated from their performance in the most difficult task. One possible explanation is that although the smaller overview interface means a larger detail interface displaying more detailed information, the overview interface showing a survey of the entire virtual environment can provide participants with relative position and distance information to help form cognitive map for decision making and preserve users’ sense of position and context. This may indicate that the requirement of overview interface as an aid might decrease as users’ familiarity with mobile devices increases. Users who were more familiar with mobile devices would like to pay more attention to the detail interface.

Concerning subjective assessments of mental workload, the low physical demand scores and frustration scores reflected the comfortable and effective interactions with overview interfaces. Although participants who were more familiar with mobile devices gave lower scores than those who were less familiar with mobile devices on most dimensions except temporal demand, the NASA TLX results show no significant main effect of users’ familiarity with mobile devices. It is perhaps due to the limited size and complexity of the virtual exhibition used in this study.

All of the participants in this study were university students and relatively young. Differences in education backgrounds and age may affect user performance in virtual environments. These limitations provide the focus for future work.

6 Conclusion

In this study, the size of overview interfaces and users’ familiarity with mobile devices were manipulated to assess possible effects on users’ performance and experiences in virtual environments. Our results show that the smaller overview interface can improve users’ performance in the most difficult task, reduce users’ overall mental workload and enhance their satisfaction with performance in virtual environments. High familiarity with mobile devices can improve users’ performance in their first visit to the unfamiliar virtual environment. The size effects of overview interfaces vary with users’ familiarity with mobile devices. Designers of virtual environment applications might be well advised to use the smaller overview interfaces and consider users’ familiarity with mobile devices.