Physically interactive tabletop augmented reality using the Kinect

Type of content
Conference Contributions - Published
Thesis discipline
Degree name
Publisher
University of Canterbury. Computer Science and Software Engineering
University of Canterbury. Human Interface Technology Laboratory
Journal Title
Journal ISSN
Volume Title
Language
Date
2012
Authors
Corbett-Davies, S.
Green, R.
Clark, A.
Abstract

In this paper we present a method for allowing arbitrary objects to interact physically in an augmented reality (AR) environment. A Microsoft Kinect is used to track objects in 6 degrees of freedom, enabling realistic interaction between them and virtual content in an tabletop AR context. We propose a point cloud based method for achieving such interaction. An adaptive per-pixel depth threshold is used to extract foreground objects, which are grouped using connected-component analysis. Objects are tracked with a variant of the Iterative Closest Point algorithm, which uses randomised projective correspondences. Our algorithm tracks objects moving at typical tabletop speeds with median drifts of 8:5% (rotational) and 4:8% (translational). The point cloud representation of foreground objects is improved as additional views of the object are visible to the Kinect. Physics-based AR interaction is achieved by fitting a collection of spheres to the point cloud model and passing them to the Bullet physics engine as a physics proxy of the object. Our method is demonstrated in an AR application where the user can interact with a virtual tennis ball, illustrating our proposed method's potential for physics-based AR interaction.

Description
Citation
Corbett-Davies, S., Green, R., Clark, A. (2012) Physically interactive tabletop augmented reality using the Kinect. Dunedin, New Zealand: 27th International Conference on Image and Vision Computing New Zealand (IVCNZ '12), 26-28 Nov 2012. Proceedings:, 210-215.
Keywords
augmented reality, interaction, physical simulation
Ngā upoko tukutuku/Māori subject headings
ANZSRC fields of research
Field of Research::08 - Information and Computing Sciences::0801 - Artificial Intelligence and Image Processing
Field of Research::08 - Information and Computing Sciences::0803 - Computer Software
Field of Research::08 - Information and Computing Sciences::0806 - Information Systems::080602 - Computer-Human Interaction
Rights