Ballast mats are considered as an effective solution for reducing vehicle-induced vibrations. However, the research on the vibration characteristics of each part of the ballasted track with a ballast mat is limited. In this study, the ballast mat vibration reduction effects are evaluated by numerical and experimental analysis using wheelset drop tests. A three-dimensional model consisting of a wheel, track and the contact between them is built using a rigid–flexible coupling method. The accuracy of the numerical model is verified by comparison with the finite element model in terms of the track receptance and phase angle. Comparisons show that the proposed model is in good agreement with the finite element model, which allows validating the flexible-body model. Moreover, the track dynamic performance in the presence and absence of the ballast mat is studied with the wheelset drop tests in both time and frequency domains. The results from the wheelset drop excitation tests show that the use of the ballast mat decreases the mid- and high-frequency track vibration by 13–17 dB but increases the low-frequency track vibration by 5–15 dB.