Real-Time 3D Tracking of Laparoscopy Training Instruments for Assessment and Feedback

Assessment of minimally invasive surgical skills is a non-trivial task, usually requiring the presence and time of expert observers, including subjectivity and requiring special and expensive equipment and software. Although there are virtual simulators that provide self-assessment features, they are limited as the trainee loses the immediate feedback from realistic physical interaction. The physical training boxes, on the other hand, preserve the immediate physical feedback, but lack the automated self-assessment facilities. This study develops an algorithm for real-time tracking of laparoscopy instruments in the video cues of a standard physical laparoscopy training box with a single fisheye camera. The developed visual tracking algorithm recovers the 3D positions of the laparoscopic instrument tips, to which simple colored tapes (markers) are attached. With such system, the extracted instrument trajectories can be digitally processed, and automated self-assessment feedback can be provided. In this way, both the physical interaction feedback would be preserved and the need for the observance of an expert would be overcome. Real-time instrument tracking with a suitable assessment criterion would constitute a significant step towards provision of real-time (immediate) feedback to correct trainee actions and show them how the action should be performed. This study is a step towards achieving this with a low cost, automated, and widely applicable laparoscopy training and assessment system using a standard physical training box equipped with a fisheye camera.

Localization System for Indoor Mobile Robot Using Large Square-Shaped Reflective Marker

A localization system using reflective markers and a fisheye camera with blinking infrared lights is useful and safe for mobile robot navigation in an environment with coexisting humans and robots; however, it has the problems of low robustness and a small measurable range for marker detection. A large, square-shaped reflective marker, with solid and dotted edges, is proposed for more reliable localization of indoor mobile robots. It can be easily detected using Hough transform and is robust for occlusion. The coordinates of the four corners of the square-shaped marker determine the robot’s localization. Infrared lighting with a new LED arrangement is designed for a wide measurable range via brightness simulation, including the effect of observation and reflection angles. A prototype system was developed, enabling the 2D position and orientation to be detected with an accuracy of 60 mm and 3◦, respectively, within a 4 m2 area.