Physically interactive tabletop augmented reality using the Kinect (2012)

View/ Open
Type of Content
Conference Contributions - PublishedPublisher
University of Canterbury. Computer Science and Software EngineeringUniversity of Canterbury. Human Interface Technology Laboratory
Collections
Abstract
In this paper we present a method for allowing arbitrary objects to interact physically in an augmented reality (AR) environment. A Microsoft Kinect is used to track objects in 6 degrees of freedom, enabling realistic interaction between them and virtual content in an tabletop AR context. We propose a point cloud based method for achieving such interaction. An adaptive per-pixel depth threshold is used to extract foreground objects, which are grouped using connected-component analysis. Objects are tracked with a variant of the Iterative Closest Point algorithm, which uses randomised projective correspondences. Our algorithm tracks objects moving at typical tabletop speeds with median drifts of 8:5% (rotational) and 4:8% (translational). The point cloud representation of foreground objects is improved as additional views of the object are visible to the Kinect. Physics-based AR interaction is achieved by fitting a collection of spheres to the point cloud model and passing them to the Bullet physics engine as a physics proxy of the object. Our method is demonstrated in an AR application where the user can interact with a virtual tennis ball, illustrating our proposed method's potential for physics-based AR interaction.
Citation
Corbett-Davies, S., Green, R., Clark, A. (2012) Physically interactive tabletop augmented reality using the Kinect. Dunedin, New Zealand: 27th International Conference on Image and Vision Computing New Zealand (IVCNZ '12), 26-28 Nov 2012. Proceedings:, 210-215.This citation is automatically generated and may be unreliable. Use as a guide only.
Keywords
augmented reality; interaction; physical simulationANZSRC Fields of Research
08 - Information and Computing Sciences::0801 - Artificial Intelligence and Image Processing08 - Information and Computing Sciences::0803 - Computer Software
08 - Information and Computing Sciences::0806 - Information Systems::080602 - Computer-Human Interaction
Related items
Showing items related by title, author, creator and subject.
-
Poster: Physically-Based Natural Hand and Tangible AR Interaction for Face-to-Face Collaboration on a Tabletop
Piumsomboon, T.; Clark, A.; Umakatsu, A.; Billinghurst, Mark (University of Canterbury. Human Interface Technology Laboratory, 2012)In this paper, we present an AR framework that allows natural hand and tangible AR interaction for physically-based interaction and environment awareness to support face-to-face collaboration using Microsoft Kinect. Our ... -
Grasp-Shell vs Gesture-Speech: A Comparison of Direct and Indirect Natural Interaction Techniques in Augmented Reality
Piumsomboon, T.; Altimira, D.; Kim, H.; Clark, A.; Lee, G.; Billinghurst, Mark (University of Canterbury. Human Interface Technology Laboratory, 2014)In order for natural interaction in Augmented Reality (AR) to become widely adopted, the techniques used need to be shown to support precise interaction, and the gestures used proven to be easy to understand and perform. ... -
A Comparison of Surface and Motion User-Defined Gestures for Mobile Augmented Reality.
Dong Z; Piumsomboon T; Zhang J; Clark AJ; Bai H; Lindeman RW (ACM, 2020)