Natural Gesture Based Interaction for Handheld Augmented Reality

Type of content
Theses / Dissertations
Publisher's DOI/URI
Thesis discipline
Computer Science
Degree name
Master of Science
University of Canterbury. Computer Science and Software Engineering
Journal Title
Journal ISSN
Volume Title
Gao, Lei

The goal of this research thesis is to explore and evaluate a novel interaction interface performing canonical manipulations in 3D space for Augmented Reality (AR) on handheld devices. Different from current handheld AR applications usually using touch-screen based interaction methods, we developed a 3D gesture based interaction approach for handheld AR using an attached RGB-Depth camera to provide intuitive 3D interaction experience in 3D space. By identifying fingertips and mapping their 3D positions into the coordinate system of AR virtual scene, our proposed method allows users to perform operations on virtual objects using their fingers in midair with six-degrees-of-freedom (6DOF). We applied our methods in two systems: (1) a client-server handheld AR system, and (2) a standalone handheld tablet AR system. In order to evaluate the usability of our gesture-based interface we conducted a user study in which we compared the performance to a 2D touch-based interface. From the results, we concluded that traditional 2D touch-based interface performed faster than our proposed 3D gesture-based interface. However, our method proved a high entertainment value, suggesting great possibilities for leisure applications.

Augmented Reality (AR), Natural gesture based interaction, Handheld device
Ngā upoko tukutuku/Māori subject headings
ANZSRC fields of research
Copyright Lei Gao