Head Pose Estimation through Keypoints Matching between Reconstructed 3D Face Model and 2D Image

Mainstream methods treat head pose estimation as a supervised classification/regression problem, whose performance heavily depends on the accuracy of ground-truth labels of training data. However, it is rather difficult to obtain accurate head pose labels in practice, due to the lack of effective equipment and reasonable approaches for head pose labeling. In this paper, we propose a method which does not need to be trained with head pose labels, but matches the keypoints between a reconstructed 3D face model and the 2D input image, for head pose estimation. The proposed head pose estimation method consists of two components: the 3D face reconstruction and the 3D–2D matching keypoints. At the 3D face reconstruction phase, a personalized 3D face model is reconstructed from the input head image using convolutional neural networks, which are jointly optimized by an asymmetric Euclidean loss and a keypoint loss. At the 3D–2D keypoints matching phase, an iterative optimization algorithm is proposed to match the keypoints between the reconstructed 3D face model and the 2D input image efficiently under the constraint of perspective transformation. The proposed method is extensively evaluated on five widely used head pose estimation datasets, including Pointing’04, BIWI, AFLW2000, Multi-PIE, and Pandora. The experimental results demonstrate that the proposed method achieves excellent cross-dataset performance and surpasses most of the existing state-of-the-art approaches, with average MAEs of 4.78∘ on Pointing’04, 6.83∘ on BIWI, 7.05∘ on AFLW2000, 5.47∘ on Multi-PIE, and 5.06∘ on Pandora, although the model of the proposed method is not trained on any of these five datasets.

Pose Estimation of Automatic Battery-Replacement System Based on ORB and Improved Keypoints Matching Method

This paper presents an improved Oriented Feature from Accelerated Segment Test (FAST) and Rotated BRIEF (ORB) keypoints matching method for pose estimation of automatic battery-replacement systems. The key issue of the system is how to precisely estimate the pose of the camera in respect to the battery. In our system, the pose-estimation hardware module is mounted onto the robot manipulator, composed of double high brightness LED light source, one monocular camera, and two laser rangefinders. The camera is utilized to take an image of the battery, the laser rangefinders on both sides of the camera are utilized to detect the real-time distance between the battery and the pose-estimation system. The estimation result is significantly influenced by the matching result of the keypoints detected by the ORB technique. The modified matching procedure, based on spatial consistency nearest hamming distance searching method, is used to determine the correct correspondences. Meanwhile, the iterative reprojection error minimization algorithm is utilized to discard incorrect correspondences. Verified by the experiments, the results reveal that this method is highly reliable and able to achieve the required positioning accuracy. The positioning error is lower than 1 mm.

A New Scheme for Keypoint Detection and Description

The keypoint detection and its description are two critical aspects of local keypoints matching which is vital in some computer vision and pattern recognition applications. This paper presents a new scale-invariant and rotation-invariant detector and descriptor, coined, respectively, DDoG and FBRK. At first the Hilbert curve scanning is applied to converting a two-dimensional (2D) digital image into a one-dimensional (1D) gray-level sequence. Then, based on the 1D image sequence, an approximation of DoG detector using second-order difference-of-Gaussian function is proposed. Finally, a new fast binary ratio-based keypoint descriptor is proposed. That is achieved by using the ratio-relationships of the keypoint pixel value with other pixel of values around the keypoint in scale space. Experimental results show that the proposed methods can be computed much faster and approximate or even outperform the existing methods with respect to performance.