Mid-air Gestures

The use of human gesturing to interact with devices such as computers or smartphones has presented several problems. This form of interaction relies on gesture interaction technology such as Leap Motion from Leap Motion, Inc, which enables humans to use hand gestures to interact with a computer. The technology has excellent hand detection performance, and even allows simple games to be played using gestures. Another example is the contactless use of a smartphone to take a photograph by simply folding and opening the palm. Research on interaction with other devices via hand gestures is in progress. Similarly, studies on the creation of a hologram display from objects that actually exist are also underway. We propose a hand gesture recognition system that can control the Tabletop holographic display based on an actual object. The depth image obtained using the latest Time-of-Flight based depth camera Azure Kinect is used to obtain information about the hand and hand joints by using th...

A presenca de dispositivos computacionais cresce a cada dia, tornando-os disponiveis nos mais diferentes cenarios de uso. A interacao com o sistema deve evoluir em conjunto com a tecnologia para prover aos usuarios uma melhor experiencia de uso nestes diferentes ambientes. Diversos estudos na area de interfaces multimodais defendem os beneficios das mesmas, pela disponibilidade de formas de interacao mais naturais, permitindo aos usuarios maior eficiencia e satisfacao na execucao da tarefa do sistema. Nos ultimos anos, dispositivos que possibilitam formas de interacao consideradas mais naturais, comecaram a estar amplamente disponiveis ao publico e ser utilizados com mais frequencia, demonstrando grande potencial para uso. Este trabalho apresenta uma pesquisa com objetivo de compreender e comparara preferencia e aceitacao de uso dessas tecnologias,a partir de sua implementacao e analise em um sistema de apresentacao, utilizando o dispositivo Kinect e um smartphone Android.O sistema ...

In this paper, we explore asymmetric bimanual interaction with mobile Virtual Reality (VR). We have developed a novel two handed interface for mobile VR which uses a 6 degree of freedom (DoF) controller input for the dominant hand and full-hand gesture input for the non-dominant hand. We evaluated our method in a pilot study by comparing it to three other asymmetric bimanual interfaces (1) 3D controller and 2D touch-pad, (2) 3D gesture and 2D controller, and (3) 3D gesture and 2D touchpad in a VR translation and rotation task. We observed that using our position aware handheld controller with gesture input provided an easy and natural experience. CCS Concepts •Human-centered computing → Virtual reality; Interface design prototyping;

If the interactions made on multi-touch surfaces can be moved to the air above them, a user will have access to a powerful tool. The possibility of tracking the hands' movement in mid-air, creates a world full of interaction possibilities. Despite having been created consolidated solutions for multi-touch interactions, these are limited to a bi-dimensional space, preventing the possibility of direct interactions on the visualization space. Some works were created to try and overcome this problem, however most of them use intrusive techniques, don't use stereoscopy or virtual reality or are lacking accuracy. This work had the objective to explore and evaluate which techniques allow to manipulate with precision 3D objects in mid-air, on a non intrusive semi-immersive/immersive environment. With this work it was possible to observe that the mid-air interactions in immersive systems are the most efficient and satisfying method for users, due to the direct manipulation of 6 degre...

If the interactions made on multi-touch surfaces can be moved to the air above them, a user will have access to a powerful tool. The possibility of tracking the hands' movement in mid-air, creates a world full of interaction possibilities. Despite having been created consolidated solutions for multi-touch interactions, these are limited to a bi-dimensional space, preventing the possibility of direct interactions on the visualization space. Some works were created to try and overcome this problem, however most of them use intrusive techniques, don't use stereoscopy or virtual reality or are lacking accuracy. This work had the objective to explore and evaluate which techniques allow to manipulate with precision 3D objects in mid-air, on a non intrusive semi-immersive/immersive environment. With this work it was possible to observe that the mid-air interactions in immersive systems are the most efficient and satisfying method for users, due to the direct manipulation of 6 degre...

On traditional tables, people frequently use the third dimension to pile, sort and store objects. However, while effective and informative for organization, this use of the third dimension does not usually extend far above the table. To enrich interaction with digital tables, we present the concept of shallow-depth 3D--3D interaction with limited depth. Within this shallow-depth 3D environment several common interaction methods need to be reconsidered. Starting from any of one, two and three touch points, we present interaction ...

Abstract—In this paper, we present the design and the development of a robust, low-cost and easy to implement system for a gesture-based interface that enables visitors to navigate in 3D in a virtual reality environment. By using a state-of-the-art 3D motion sensing device, Microsoft Kinect, and publicly available gesture recognition technologies, the interface allows visitors to navigate in three dimensions without introducing any kind of physical device like gloves or 3D pointers. In the paper, we describe the system, the tracking technology and the user interface through the case study of Rediscovering Vrouw Maria project. Rediscovering Vrouw Maria is a gesture-based, interactive, virtual reality simulation that enables visitors to explore an underwater cultural heritage site with an intuitive interface based on simple human gestures and actions. As a multidisciplinary design research project, it has been undertaken

The emerging capabilities of smartphones are fuelling a rise in the use of mobile phones as input devices. The complexity of 3D object manipulation has become a challenge for traditional input. Sensors and interaction methods available in a modern smartphone may turn it into an interactive input device for manipulation of 3D objects. This study investigates the usability of a smartphone as an input device for 3D, and also explores its potential as an optimal input method for manipulating 3D space. As such, usability study requirements are formulated, and prototype 3D software with defined tasks developed, to aid in the usability studies. User satisfaction is investigated for each defined task, with the aim to explore optimal interaction methods for different approaches in manipulating 3D objects.

Abstract—Interaction is a very important feature in a variety
of applications, especially when Virtual Reality (VR) is used. To
meet this need, alternative methods such as gestural interface
from 3D sensors can be a good proposition. Therefore, this work
aims to compare the Kinect and Leap Motion sensors related
to their usability in natural interfaces to VR environments. The
comparison will be made from assessments on the recognition of
gestures of these devices and the feasibility of their use in short
and long movements. The main goal is to guide the decision of
what is the best option to be used in a 3D Virtual Museum
application.
Resumo—A interação é uma caracterı́stica muito importante
em diversas aplicações, principalmente quando a Realidade
Virtual (RV) é usada. Para satisfazer essa necessidade, métodos
alternativos, como interface gestual a partir de sensores 3D pode
ser uma boa proposta. Por isso este trabalho visa comparar os
sensores Kinect e Leap Motion em frente à sua usabilidade em
interfaces naturais para ambientes RV. A comparação será feita
a partir de avaliações sobre o reconhecimento de gestos desses
dispositivos e a viabilidade de seu uso em movimentos curtos
e longos. O objetivo final é guiar a decisão de qual dispositivo
utilizar em uma aplicação de Museu Virtual 3D.

In diesem Beitrag wird ein Konzept für ein mobiles interaktives Personal-Fabrication Tool vorgeschlagen, das die Vor-Ort-Reparatur von beschädigten Alltagsgegenständen als Mixed Reality Anwendung realisiert. Das Toolkit (MixRepair) ermöglicht das Einscannen von beschädigten Gegenständen und rekonstruiert die fehlenden Teile als virtuelles Ersatzteil. Das Ersatzteil kann mit den Händen und menübasiert nachbearbeitet werden. Die Manipulation mit den Händen erfolgt entweder physikbasiert oder über interpretierte Gesten, die Menüs über Mid-Air-Touch. Das Einpassen des virtuellen Ersatzteils in den realen Alltagsgegenstand wird als virtuelles Overlay mit einer optischen See-Through-Brille möglich. Final soll das nachbearbeitete virtuelle Ersatzteil als STL exportiert und auf einem 3D-Drucker produziert werden.

Mid-air gesture interaction is often considered as a " natural " and " intuitive " input method, however, there has been little investigation of how older adults perceive and interact with these interfaces. We report the preliminary results of an experiment in which 20 younger and 20 older adults, without previous experience of gesturing in the air, are asked to perform 15 commonly used gestures and complete 5 computer tasks using mid-air gestures. We aim to learn how each age group engages with mid-air gestures regarding perceived easiness, confidence, intention of use and physical effort.

Este trabalho tem como objetivo analisar o conceito de interfaces naturais do usuário e
refletir sobre suas qualidades e limitações no contexto da interação humanocomputador,
explicitando o papel do design durante o projeto. Os principais autores que
guiaram a fundamentação teórica da pesquisa foram Wigdor & Wixon (2011) e Saffer
(2008), dando destaque para a interação gestual como um tipo de interface natural. Foi
desenvolvido um estudo de caso da vitrine baseada em interação gestual chamada
“Virtual Shoe Fitting” s partir dos elementos da experiência do usuário de Garrett.

This study aims to analyze the concept of natural user interfaces and clarify their
strengths and limitations in the field of human-computer interaction, explaining the role
of design throughout the project. The main authors who guided the theoretical
foundation of the research were Wigdor & Wixon (2011) and Saffer (2008),
highlighting the gestural interaction as a type of natural interface. A study case was
developed based on gestural interaction showcase called "Virtual Shoe Fitting" and it was based on Garrett’s elements of the user experience.

This paper aims to raise the question : "How much 3D bimanual interaction techniques can be useful to the design of collaborative interaction techniques in the field of Collaborative Virtual Environ-ment (CVE)?". Indeed, CVE involve the use of complex inter-action techniques based on specific collaborative metaphors. The design of these metaphors may be a difficult task because it has to deal with collaborative issues that came from sparse research areas (Human-Computer Interfaces, Human-Human Interactions, Networking, Physiology and Social Psychology). Metaphors for bimanual interactions have been developed for a while essentially because it is a widely spread area of interest for common tasks. Bi-manual interactions involve the simultaneous use of both hands of the user in order to achieve a goal with better performances com-pared to uni-manual interactions thanks to a natural skill that is pro-prioception. This collaborative aspect could certainly be a helpful entry poi...

Transparent Touch explores interaction with a display system with two spatially distinct layers. A transparent touch screen is overlaid in front of another information layer, which can be a 3-dimensional object or second screen. Both layers are optically aligned and offer the advantage to provide distinct semantic contexts while optically and cognitively integrating them. The system is explored in three use-case scenarios in which the transparent screen serves as an augmentation layer, as an annotation layer, and as a control layer. The concept is known from HUD displays in airplanes or cars and integrates features of augmented reality systems, mid-air interaction systems and touch screens. Our study collects an initial set of user responses.

This study aims to compare the use of specific mid-air gestures across platforms (Microsoft Kinect, Leap Motion and Myo Armband) in order to identify the most recurrent gestures and their functions within the interface. 250 applications were analysed and 15 common gestures were mapped within this study. Results will contribute to wider research on mid-air gesture-based interfaces and assistive technology.