Large-area specular highlight removal based on disparity layering and multi-strategy hierarchical restoration with light field camera arrays

In this paper, what we believe to be a new highlight removal method based on disparity layering and multi-strategy hierarchical restoration is proposed to address the challenge of large-area specular highlights by using a light field camera array. Initially, the dichromatic reflection model is combined with unsupervised k-means clustering to precisely detect highlight regions and generate multi-view highlight masks. Subsequently, a strategy based on disparity estimation and energy minimization, which guided by the disparity reliability of non-highlight regions, is proposed to properly assign highlight pixels to their corresponding disparity layers. Finally, a multi-strategy hierarchical restoration framework based on multi-view observation availability is designed to adaptively recover highlight pixels within their assigned disparity layers. Furthermore, a composite evaluation metric, named PSSE, is designed to quantitatively assess the overall restoration performance. Experimental results on both public light field datasets and real captured scenes demonstrate that the method effectively removes large-area highlights while preserving geometric structure and photometric consistency, and it shows strong robustness and adaptability in complex textured environments. Compared with single-view methods that often lose structural details and multi-view methods that rely on sequential capture and unstable feature matching, the proposed approach achieves more accurate detection and robust restoration, and it also holds promising potential for industrial inspection and digital preservation of cultural heritage.