Thin functional barrier layers of aluminum oxide (Al2O3) that are used particularly in photovoltaic (PV) modules to prevent the possibility of water vapor ingress should be applied over the entire PV surface without any defects. However, for barrier layer thicknesses within the sub-micrometer range (up to 50 nm) produced through the atomic layer deposition (ALD) method, it is common for defects to occur during the production process. To avoid defective barriers from being incorporated in the final PV unit, defects need to be detected during the barrier production process. In this paper, the implementation of in process inspection system capable of detecting surface defects such as pinholes, scratches, or particles down to a lateral size of 3 μm and a vertical resolution of 10 nm over a 500 mm barrier width is presented. The system has a built-in environmental vibration compensation capability, and can monitor ALD-coated films manufactured using roll-to-roll (R2R) techniques. Ultimately, with the aid of this in process measurement system, it should be possible to monitor the coating surface process of large-area substrates, and if necessary, carry out remedial work on the process parameters.