Planetary Gearbox (PG) is widely used in many vital transmission systems with the advantages such as high power-weight ratio and high load capacity. To maintain the performance of these system, extensive attentions are paid in the vibration signal analysis for early fault detection and diagnosis of PGs. However, vibration signals are less accurate as they severely influenced by multi-interfaces energy dissipation and varying transmission paths. In contrast, Instantaneous Angular Speed (IAS) is directly related to the gear dynamics and free from the path varying effect. This paper evaluates the performance of an improved IAS measurement system to diagnose PG faults, in which a dual detector setup is adopted to suppress IAS errors caused by the setting parameters and encoder errors. Firstly, a PG dynamic model is established to simulate IAS responses from a PG with local faults on sun and planet gears. Secondly, the noise level of the demodulated IAS is studied under different system parameter configurations. Subsequently, a dual detector self-denoising method is developed to suppress the false IAS caused by encoder errors. Finally, the performance of proposed IAS measurement system is evaluated based on an industrial PG with sun and planet faults. Experimental results show PG faults can be effectively diagnosed from the simple IAS order spectra, indicating that the proposed IAS measurement system is a more promising and yet lower costing approach to monitor rotating machines.