3D reconstruction of a highly reflective surface based on multi-view fusion and point cloud fitting with a structured light field.

In this paper, a three-dimensional (3D) reconstruction method based on multi-view fusion and point cloud fitting is proposed to accurately reconstruct the highly reflective surface, which can effectively restore the highlight regions with only a single shot. A multi-view image fusion method based on the characteristics of microlens array cameras is used to remove unsaturated highlights. A new adaptive method, to our knowledge, based on binary images is designed to quickly extract the highlight regions. Then, what we believe to be a novel B-spline point cloud repair algorithm is proposed to construct geometric features for the highlight regions, and the weighted least squares plane fitting method based on priority is utilized to remove the saturated highlight point by point. More importantly, the 3D reconstruction system based on a structured light field is set up independently to verify the feasibility and effectiveness of the proposed method. The experimental results show that the proposed method can effectively achieve 3D reconstruction with large reflectivity changes and large highlight areas, and its repaired rate can reach 98.44%. Consequently, the proposed method can fully restore all highlight regions and can uniformly, accurately, and smoothly recover different highly reflective surfaces while avoiding hardware complexity.