Shape recognition and pose estimation for mobile augmented reality

• Published in: 2009 8th IEEE International Symposium on Mixed and Augmented Reality pp. 65 - 71
• Main Authors: Hagbi, Nate, Bergig, Oriel, El-Sana, Jihad, Billinghurst, Mark
• Format: Conference Proceeding
• Language: English
• Published: Washington, DC, USA IEEE Computer Society 19.10.2009; IEEE
• Series: ACM Other Conferences
• Subjects: 3D pose estimation; Active contours; Augmented reality; Cameras; Computing methodologies; Computing methodologies > Artificial intelligence; Computing methodologies > Artificial intelligence > Computer vision; Computing methodologies > Artificial intelligence > Computer vision > Computer vision problems; Computing methodologies > Artificial intelligence > Computer vision > Computer vision representations; Computing methodologies > Artificial intelligence > Computer vision > Computer vision representations > Shape representations; Computing methodologies > Computer graphics; Computing methodologies > Computer graphics > Graphics systems and interfaces; Computing methodologies > Computer graphics > Graphics systems and interfaces > Virtual reality; Computing methodologies > Computer graphics > Shape modeling; Feature extraction; free-hand sketching; geometric projective invariance; handheld AR; Human-centered computing; Human-centered computing > Human computer interaction (HCI); Human-centered computing > Human computer interaction (HCI) > Interaction paradigms; Human-centered computing > Human computer interaction (HCI) > Interaction paradigms > Mixed > augmented reality; Humans; In-Place Augmented Reality; Libraries; Performance analysis; Real time systems; Runtime; Shape; shape dual perception; shape recognition; vision-based tracking
• ISBN: 9781424453900; 1424453909
• DOI: 10.1109/ISMAR.2009.5336498

In this paper we present Nestor, a system for real-time recognition and camera pose estimation from planar shapes. The system allows shapes that carry contextual meanings for humans to be used as Augmented Reality (AR) tracking fiducials. The user can teach the system new shapes at runtime by showing them to the camera. The learned shapes are then maintained by the system in a shape library. Nestor performs shape recognition by analyzing contour structures and generating projective invariant signatures from their concavities. The concavities are further used to extract features for pose estimation and tracking. Pose refinement is carried out by minimizing the reprojection error between sample points on each image contour and its library counterpart. Sample points are matched by evolving an active contour in real time. Our experiments show that the system provides stable and accurate registration, and runs at interactive frame rates on a Nokia N95 mobile phone.