Occlusion in Mirror-Based Co-Located Augmented Reality Systems

This paper describes the incorporation of realistic occlusion effects into mirror-based, stereoscopic co-location augmented reality display systems. By adding a light-blocking device in the form of an LCD panel underneath the semitransparent mirror, the view of the physical world can be selectively blocked out such that virtual objects can fully occlude physical objects. Furthermore, by discarding pixels of the virtual objects rendered on the reflected display, physical objects seen through the semitransparent mirror and the transmissive LCD panel appear to occlude these virtual objects. The governing principles of the approach are described, and two algorithmic approaches (model based and vision based) for scene reconstruction and the generation of the occlusion masks are presented. Finally, a prototype implementation of the system is presented.