Transient impulses caused by local defects are critical for the fault detection of rotating machines. However, they are extremely weak and overwhelmed in the strong noise and harmonic components, making the transient features are very difficult to be extracted. This paper proposes an adaptive multi-scale improved differential filter (AMIDIF) to enhance the identification of transient impulses for rotating machine fault diagnosis. In this scheme, firstly, the AMIDIF is performed to decompose the measured signal of rotating machine into a series of multi-scale improved differential filter (MIDIF) filtered signals. Subsequently, in view of the MIDIF filtered signals exhibit varying extents of validity in revealing fault features, a weighted reconstruction method using correlation analysis is proposed in which the weighted coefficients are counted and distributed to the corresponding MIDIF filtered signals to highlight the effective MIDIF filtered signals and weaken the invalid ones. Finally, the transient impulse components of rotating machinery are obtained by multiplying the weighted coefficients and the MIDIF filtered signals under different scales. Furthermore, the fault types of rotating machines are inferred from the fault defect frequencies in the envelope spectrum of the transient impulses. Simulation analysis and experimental studies are implemented to verify the performance of the AMIDIF compared with the state-of-the-art methods including spectral kurtosis (SK), multi-scale average combination different morphological filter (ACDIF) and multi-scale morphology gradient product operation (MGPO). The results prove that the AMIDIF has excellent performance in extracting transient features for rotating machines fault diagnosis.