Gesture based interactions for augmented virtual mirrors

Abstract

In this research we explore the development of a gesture interface for an interactive virtual mirror display. We follow a user-centred approach and utilised interaction design methods. Through the course of designing and developing the interface, in order to investigate natural gesture interaction methods we conducted three user studies : a guessability study, a mapping study, and a target study. Twenty participants were recruited in a guessability study to discover user-defined gestures for common interaction tasks. For pointing tasks, 80% of users used a hovering gesture, with the rest using directing gestures. Swiping gestures were used in the majority (81%) of scrolling tasks. For selection tasks, a mixture of waiting, tapping and grabbing gestures were observed. When choosing between dichotomous options, a mix of thumbs up/down and directing gesture were used. Results suggests that for some tasks, designing a system to support multiple interaction methods may be beneficial for usability. Next, we recruited ten participants for a mapping study evaluating the use of a non-linear mapping of the interface cursor. Our interface showed potential as an interaction method for where a direct mapping method was unsuitable, such as when a target was out of reach. Finally, twenty participants took part in a target study where we evaluated a hovering, extension and pointing interface. The hovering interface was ranked as the easiest and most fun to use, followed by the extension interface, although task completion time between the hovering and extension interfaces showed no significant differences. The pointing interface had the longest task completion time and was ranked the lowest in ease of use and overall preference. In conclusion, the use of a user centred approach resulted in the development of three gesture interfaces utilising a hovering, extension and pointing interaction method that were felt to be natural and intuitive.