Keywords

1 Introduction

Augmented Reality (AR) is the art and technology in which traditional field of real world view is augmented with additional information. This can be computer-generated 2D and 3D images or information superimposed on the real-world view captured from the camera of smartphone, computer or other device [1]. Augmented image appears to its users like virtual and real objects coexisted in the same space. In other words, AR immerses its users in virtually enhanced real world [2].

Augmentation of reality has been used in different medias and systems for decades, e.g. head-up displays in fighter planes, scoreboards in sportscasts. Due to the computerisation and rapid emergence and development of mobile technology, AR is now available for common users in their mobile devices and possibly soonish with AR-glass technology.

The interest towards the emerging AR technology has been rapidly growing among the museums and cultural heritage sites around the world. However, the acceptance of AR applications can vary in different populations. Lee et al. made a cultural comparison between South Korea and Ireland, both having high smartphone penetration rates but different cultural profiles, and noticed that aesthetics of AR have the strongest influence on perceived enjoyment. Also, as expected, South Korea, having high collectivism and high uncertainty avoidance culture, displayed stronger dependence on social influence and hedonic characteristics of AR [3].

Smartphone or tablet device meets the main requirements posed by AR since it has a camera and capability of rendering and displaying the augmented graphics. [4] Hence, with explosive growth of penetration rates of smartphone, application-based AR has been more accessible to users. Especially, cultural heritage tourism is one of the most important areas served by mobile AR app [5, 6] which provides digitally restored artifacts, thereby preventing degradation of cultural heritage sites aggravated by frequent access by tourists and let them perceive fun and usefulness [7]. A number of cultural heritage institutions around the world, such as the Louvre Museum in Paris and British Museum in London, have developed and provided with their mobile AR appsFootnote 1.

As this study is – as far as we know – the among the first studies about AR-games which involves end-users, the research questions reflect the overall requirement for these kinds of solutions and thus our research questions are as follows:

  1. (1)

    Is there a demand potential for AR adventures in cultural travel?

  2. (2)

    Does AR adventure generate added value to the museum experience?

  3. (3)

    Is an AR adventure suitable for museum and cultural travel atmosphere?

  4. (4)

    Are the customers willing to pay for it?

In this paper an Augmented Reality Adventure User Experience (UX) research results done in Finnish cultural heritage site Luostarinmäki Handicrafts Museum in Turku are introduced and discussed. The research team has been developing AR and Mixed Reality (MR) mobile applications (apps) for Finnish cultural and especially museum field to research this emerging technology’s possibilities in presenting historically accurate and entertaining experiences to audiences. The paper is organized as first introducing the AR technology and our prototype, then our research design and results, followed by discussion and conclusions.

2 Reality… Augmented?

“What is real? How do you define ‘real’? If you’re talking about what you can feel, what you can smell, what you can taste and see, then ‘real’ is simply electrical signals interpreted by your brain.”

– Morpheus, The Matrix [8].

2.1 What Is AR?

Whereas the more known virtual reality experiences attempt to recreate all of these signals, augmented reality only attempts to complement the natural ones with some artificial flavor. It thus sits somewhere in between the physical reality and completely simulated virtual reality, as seen in the famous Virtual Reality Continuum in Fig. 1 [9].

Fig. 1.
figure 1

Levels of mixed reality.

Full size image

Whereas virtual reality (VR) can deviate greatly from the real world, augmented reality productions however must fit into the physical context in order to achieve an immersive and believable experience. At the minimum a good AR application should (1) seamlessly combine the physical and virtual pieces of content, (2) be highly interactive in real-time and (3) allow users to experience the content with free movement in the real world 3D space. [10] The first condition is not to set limitations to artistic freedom in any sense, but to highlight the fact that at least the virtual content should in fact react to as many changes and parts of the real world as possible. The second condition separates for example pre-rendered movie productions from augmented reality, as even if the contemporary film productions feature highly believable computer generated imagery, they do not represent a real-time simulation of reality from the end-users point of view. The third condition then emphasizes how important it is to the end users not be limited in the ways they can “live” in the augmented space: they must be able to move freely and explore the content from any angle and location they like. The virtual content must always be fixed tightly in its place in the real world while the user moves about.

Augmented reality can be experienced via various types of devices. Currently the most used device platform is the mobile devices segment most consumers carry with them in their everyday life: mobile phones and tablets. The image from the rear camera of the device is displayed on the screen and the virtual content is drawn on top of it to create the seamless viewing experience. While traditional mobile devices allow widespread adoption of AR experiences, they can at most provide a window-type of experience into the augmented world. The next phase, currently taking its initial steps will be in form of wearable eyewear: with see-through displays both the real world and the virtual elements can in the future be overlaid on top of the whole field of vision (FOV). Such devices, but with limited FOV are being prepared for consumer grade release by several international corporations, such as Google, Microsoft and Sony and many others have announced intentions of going into the market [11–13].

The first wave of augmented reality applications for mobile devices used purely geolocation based solutions for aligning, registering, the content with the real world. With the user’s geolocation and the device’s orientation read from magnetometer and other sensors it is possible to do only very rough registering: the content might be meters away from the actual desired location and it usually also reacts to user’s movements sluggishly and with imperfect results. Mostly the first applications were thus about displaying information about distant enough points of interest around the user, such as showing the restaurants in radius of 5 km as seen in Here City Lens application in Fig. 2.

Fig. 2.
figure 2

Here City Lens [14].

Full size image

To acquire more accurate results the modern augmented reality technologies employ visual tracking as part of the process. The image from the device’s camera is algorithmically analyzed to find stable points that can be used as a reference to measure how the device is moved around. The reference points, or feature points, can then also be matched to pre-created set of reference points. If they match with high enough accuracy, the reference point can be deduced as the camera is pointed to predefined content area which has accurately placed virtual content. The currently more traditional approach with predefined content is to use basic printed imagery as recognizable 2D markers. Complex images with high contrast and thus high amount of stable feature points work more reliably, but in some cases even photographs can be used as markers.

The more developing way of tracking is to use 3D objects as markers. They can produce both more accurate and stable registration, but also allow more viewing directions than the 2D surfaces. 3D markers are however more time consuming to create and require more data and thus bandwidth to transfer and also require much more complex and computationally heavy algorithms to be used. Since in the last few years both mobile networks have gained higher transfer speeds and devices have become more powerful, it is now possible to employ large datasets of 3D markers as the basis for mobile augmented reality applications. The next step for accurate registration lies in actual environments being used as markers, instead of just small objects. This however requires even more complex solutions, as for example changing lighting in outdoor environments creates new challenges for the current algorithms.

Solutions come in the form of both new sensors such as depth cameras [15] and also as new solutions to registration, as perhaps more than the actual feature points the registration could be based on the geometry of the real scenery. Even without directly depth sensing devices some steps to this direction can be taken by employing Simultaneous localization and mapping technology, SLAM, where the end-user’s device’s camera image is used to recreate the viewed scene as 3D geometry [16]. This way just different types of objects and surfaces could be recognized – not just the very objects used as reference material. With this approach it will be possible to create AR experiences that are not limited to be used in a single real world environment, but could instead automatically adapt to new surroundings.

As augmented reality technologies become more and more complex, the importance of professionally developed and maintained dedicated software libraries has become vital part of the industry. The wide range of end-user devices available and different tracking techniques required for varying situations makes even maintaining such libraries a daunting task, let alone developing and researching new complex algorithms for better results. Thus, the currently wide adopted solutions are sparse and only a few competitors exist. The most used easily available AR software library is the Vuforia SDK, currently owned by PTC Inc. after acquiring it recently from Qualcomm [17]. One of the many, but likely the most feature rich and most used open source library is the ARToolkit, which also was recently acquired by DAQRI [18]. For more simple use cases and targeted for non-technical users exist for example the Aurasma and Layar platforms, which both are mostly built around augmenting printed or other fiducial material [19, 20]Footnote 2.

In addition to just displaying information and small gadget type of application, there are some examples of more complex augmented reality products. Likely one most known one currently is the AR conversion of the famous sandbox game MineCraft, that Microsoft has used as part of its HoloLens demonstrations and advertising [21]. Even if VR is likely the more suited medium of the two for gaming, AR will probably also gain much traction from the sector.

2.2 Our Prototype

One of the first augmented reality adventure games for mobile devices was developed during 2014–2015 as part of the Futuristic History project at Technology Research CenterFootnote 3 of the University of Turku for use in an outdoor museum environment which is illustrated in Fig. 3. The game was to provide visitors with a more lively experience in the museum, as currently the once busy streets of the 1800s town are now mostly empty. By bringing virtual characters into the court yards it was possible to demonstrate in a very concrete manner how people lived in the area and what they were like.

Fig. 3.
figure 3

Luostarinmäki adventure [6].

Full size image

Much care was placed on clothing, speech and especially building backstories for the characters. In a very classical point-and-click adventure game approach, the players were given small tasks to complete in order advance in the story that in end turned out to be about finding a missing wedding ring. Mostly the tasks were completed by visiting different areas and getting the right virtual characters to give the player the right information to proceed. The gaming experience was targeted to be as seamless as possible, even though the advanced ALVAR 3D tracking technology provided by VTT [22] was still experimental and not completely production level reliable [6].

Most frequent individual problems while testing the application was related to finding the AR content with 3D tracking. While the technology allowed us to not print 2D markers in the museum area thus preserving the original look of the cultural heritage site and structures which was a requirement from the museum. This unfortunately caused some problems with technical functionalities because some of the users had to spend some time before finding the augmented spaces thus weakening the experience.

3 Research Method

Even though several related mobile Augmented Reality applications have recently emerged also in Cultural Heritage sites, most of them are developed to enhance the visitor experience. AR applications which focus on gaming are limited as well as the AR User experience studies with actual end-users [23].

As the UX is in connection with the user environment and varying content, it creates challenges in setting research assumptions for Museum AR-Game User Experience. There is no existing supporting or contradicting theories with similar combination of interactive content and actual museum audience as a target group. Also, according to Yin, the results of this kind of case studies are not generalizable to populations [23, 24].

Our research assumptions are based on use case- and concept evaluations of AR applications for Cultural Heritage sites [23]. They include that (1) regardless of the technological limitations that can poverish the AR-User Experience and the users see augmented reality as suitable for museum and cultural travel environment, and (2) it generates added value to the visit [23, 25]. We also assume that there are (3) market potential for future AR solutions in museum and cultural travel sector.

The user experience study was conducted June 1st to June 18th 2015 and it included themes from various different technical features to the content and marketing. The data was gathered in a structured questionnaire, build in the application. The basic user data was gathered before the testing and the usability data after the testing was done. Target sample size was 150 visitors. Accidental sampling was used. Existing study subjects were encouraged to recruit more subjects into the sample. Full data set is gathered from 129 visitors.

The main objectives of the testing were to collect feedback for the AR/MR-application and its content from various user groups, to discover its possible usability problems and to validate consumer demand for related AR/MR applications and interactive content.

The practical arrangements included a research base outside the museum gate where the test persons receive the equipment and a free museum ticket as a reward, four iPads and a selection of headphones for the audio content. Research personnel included 1–2 researchers on site during opening hours. The testing event was advertised throughout media.

Themes of the research were:

  1. (a)

    Sample structure (Age, gender, gaming experience, etc.)

  2. (b)

    Reception (How the idea of AR was received)

  3. (c)

    Usability (Technical functionality, UI, easiness of use, learning, interactivity, use of audio)

  4. (d)

    Content (How the app guides the user/tutorials etc., storyline, gamified content)

  5. (e)

    Consumer business related (Demand, Willingness to pay for contents/apps, How the AR-experience is connected to the museum experience)

Sample structure (n = 129) is shown by age and sex in Table 1 and level of education in Table 2:

Table 1. Age and gender structure in study
Full size table
Table 2. Education structure in study
Full size table

This sample consists of museum visitors that were available to test the system in the outdoor museum during the test period. It represents the typical Finnish customers of Luostarinmäki Museum during the beginning of the summer season. This mostly consists of the customer base of the museum except those that could not participate for personal reasons or because of traffic (e.g. visitors within tourist groups with dedicated guides and thus unable to participate at all). Moreover the customer base for the museum varies by the time of the year and therefore e.g. school groups or those people having holidays later in the summer were not represented [26]. The prototype app had content ready in Finnish thus restricting the participation of non-Finnish speaking people.

4 Results

First questions after the background were about attitudes. Most of our test subjects were found to be quite enthusiastic about the app and thus almost all of them agreed with the questions “Museum visit can be entertainment” (99 %) and “Museum visit can be learning” (97 %) as illustrated in Fig. 4. 78 % of the people were interested in technology, 89 % in culture and 88 % in museums. Exactly 90 people out of the 129 (70 %) were interested in gaming as is presented in Fig. 5. (To note, the eldest were not that enthusiastic about it). Therefore, an indication towards a group of enthusiastic testers was found. Most of the test users were regular users of desktop computers and smartphones but only 60 % of them used regularly tablets. Answers towards activities in digital gaming gave more heterogenic answers.

Fig. 4.
figure 4

Attitudes towards museums.

Full size image
Fig. 5.
figure 5

Interest in the app.

Full size image

As shown in Fig. 6, in our study the share of active gamers (gaming every week) was found to be about 49 % and the share of those who play digital games every month as 68 %. This – even though the definition differs a bit – follows study [27] of playing in population, where 69 % were considered players and 48 % as active players.

Fig. 6.
figure 6

Gaming activity.

Full size image
Fig. 7.
figure 7

AR experience overall and by gender.

Full size image
Fig. 8.
figure 8

“The app was pleasant to use” by age.

Full size image

It must be noticed that even tough 91 % thought the first impression of the app was interesting, as shown in Fig. 9 but yet only 81 % stated that they understood the idea of the app. Moreover, it must be remembered that the actual percentage might be even lower. Those still in primary school and those who have at least master’s degree got the best impression about the app (Table 3).

Fig. 9.
figure 9

App functionality.

Full size image
Table 3. “First impression of the app was interesting” and number of museum visits
Full size table

As shown in Fig. 7, the AR app and the pleasantness of the AR experience was seen quite differently: the experience can be pleasant even though the app is still seen less so. While 84 % saw the experience as a pleasant one, only 59 % of users confirmed that the app was pleasant to use thus indicating clear problems in the usage of the app. The pleasantness of the app’s use was one of the lowest scores gathered and can be combined with the problems and feedback in functionality (see below). Males were more demanding for the experience than females.

Whether the app is seen as a pleasant one depends hugely on the age group. People from 25 to 34 years of age are most critical towards the app and demanded better experience while children of 7 to 12 years of age were quite pleased with the application which is illustrated in Fig. 8.

Even though the app got a lot of positive feedback, the functionality got even less praises than the pleasantness (22 % bad or very bad, see Fig. 9). As mentioned in chapter 2, this and the pleasantness-problem can partly be explained with several issues with tracking.

In Fig. 10 is illustrated the results for questions about movement with device. The using of the device and moving with it was found easy enough even though the museum grounds are in unpaved hillside with lots of obstacles.

Fig. 10.
figure 10

Easiness of use in navigation and movement.

Full size image

The storyline and gaming time (c.a. 45 min) was found to be fine, but problems arose within the museum pedagogic level of the app which according to the testees was not that visible. Moreover the hardship of separating fact from fiction caused 24 % of users some issues, as illustrated in Fig. 11.

Fig. 11.
figure 11

Opinions about content.

Full size image

The adventure length – as seen in Fig. 12 – was seen as a proper length but yet people wished both more and less scenes to the app. Most of the users were quite gratified on the amount of content presented. More historical background information was still requested by most of the users, as shown in Fig. 14. Thus, it seems that the most of the testees required more historically in-depth content within the same time amount while the requirement for the amount of scenes (and thus the length of one scene) was more heterogeneous in variance.

Fig. 12.
figure 12

Requests by the testees.

Full size image
Fig. 13.
figure 13

Eagerness to continue the adventure after the museum visit.

Full size image
Fig. 14.
figure 14

The value of the app and willingness to pay.

Full size image

Most of the testees would be enthusiastic to continue their adventure later on outside the museum area, as illustrated in Fig. 13. The age group 35–44 was the most eager in this. Yet again the question did not specify the means in continuing the adventure and thus more specific research on the subject is recommended to further develop the possibilities in connecting the on-site experience with off-site experiences.

In Fig. 14 it is shown that while it seems that the users were quite pleased towards the app and 78 % of them saw that the “application brought added value to the museum visit”, many of them saw the idea of paying for the application in a different light.

However, only one third (32,5 %) of them would have wanted to pay for the test application. The museum experience as a whole was seen more valuable when combined with the AR experience but more evidence is needed to validate and increase the demand ratio and the total amount of possible customers for the sole application.

When considering commercializing alternatives of Augmented Reality content for Cultural Heritage sites, the results show that user groups vary also regarding their favourite business model. In other words the testees were relatively evenly distributed in terms of how the AR content could be offered for a museum visitor.

Thus, there seems to be a requirement for extended adventure after the experience, our study shows no clear guidelines to what kind of extended adventure it should be. There are several possibilities – all taken from the gaming industry – by making payable content, by tying the use of extra content in social media or selling additional content or features afterwards [28].

There does not seem to be clear evidence that there would be a single one-fits-for-all commercialization method as the prizing and delivery preferences varied amongst the testees. Augmented reality technology is yet emerging and thus unknown but still quite fascinating media to most museum visitors.

5 Discussion

It must be kept in mind that the app tested was a prototype – and the testees were aware of that. Still, the understanding and vision of a prototype by a developer or IT professional varies from the understanding of a standard test person. Therefore the test data does not provide definite answers but a guideline for development and some understanding of the possibilities and market share.

The feedback the app received was generally very good: most of the feedback came back as either positive or very positive and thus, the general line must be derived from the fact that people were testing the product with positive mindset. The product got obviously a lot of credit and thus positive feedback on being free and in prototype state: similar functionality issues on a e.g. 8€ product would have gotten a lot of negative feedback. In our understanding based on the testing the robustness of the tracking functionality would be the primary key in promoting the whole usability and thus develop the user experience but to point out that correctly more research is required.

The AR-experience in the Luostarinmäki was found to be very pleasing and, combined with the perceived value added, it clearly indicates that there is a demand in enchanting the museum experience with augmented reality content. This indicates positive results for research assumption 1.

The testees were quite pleased with the idea of using a tablet for an augmented reality adventure in a museum-style heritage site even though the site itself was not as easy terrain and thus not as accessible for everyone as one could hope for, and there were challenges in some technical functionalities. Some elements from the other medias (e.g. products from gaming industry), especially for the males in age group 25 to 34, could be promoted to further develop the experience for this most demanding testee group. If the target audience is younger (7–12) or older than the aforementioned 25 to 34, there is not that great of demand for game-like appearance and functionality. The storyline was found good and the length of the adventure (45 min) was found proper. The amount of historical elements to the story were found a tad wanting and thus more information both to promote the historical information and specially so that the users can separate of fact from fiction require more work, but overall for a product that is first of its kind the reception was excellent. We argue that this answers to research assumption 1 & 2 showing that the AR experience adds value to the museum experience and that AR is indeed – when generated promoting the historical values – suitable for museum and culture travel atmosphere.

There seems to be no good basis for comparison since there are no similar products in the market, so the market analysis is rather hard. Although the application was known to be a prototype with limited features, the willingness to pay was low in comparison to the value added perceived. Still we argue that the rate of one third (32,5 %) in willingness to pay is rather adequate for a decent marketing potential. Yet it is harder to analyse the true market potential of AR in museums and cultural travel, but we argue that with the positive attitude received from the testees in this study, the amount of possible customers should rise when this technology comes more commonplace. Although we must admit that the results to back up the research assumption 3 are not enough and the question remains still unanswered. Thus researching this shall remain as one of our research focuses in the research yet undone.

6 Conclusions

Augmented reality adventures for museum and cultural travel seem to have demand potential and it seems to add value to the museum and cultural travel experience to be a meaningful addition worthy to create. It also seems to be – when created promoting the historical values – suitable for cultural heritage atmosphere.

As discussed before, there is no single one-fits-for-all commercialization method as the prizing and delivery preferences varied amongst the testees. As the technology is yet emerging and unknown to most museum visitors we suggest low-threshold approach in engaging the possible future customers with easy access to the content provided while promoting the possibilities for new and exciting experiences.

The potentiality for selling new additional content should increase when the audiences become familiar with the AR technology. Overall it seems these kind of apps have their place in the markets and there are several people who are willing to pay for them. The question remains is there enough and if, when? The inclusion of the app prize to the museum visit or charging a separate payment – or any other funding method – is yet another thing that should be researched more upon.

To further understand the negative feedback gotten in this project, we need to distinguish the frustration over functionality, usability issues and overall lack of interest from each other. Our next goal in future studies is to promote the separation of fact and fiction in the storytelling. We are doing more research in the field of cultural heritage AR applications and thus will continue studying also the UX aspects further. Moreover we aim to create guidelines on how the future apps for cultural and museum sites should be constructed.