To maintain desirable service quality and operational safety, a wireless monitoring unit integrated with the vibration energy harvesting technology becomes an available choice to achieve self-powered, maintenance-free, and real-time monitoring of the train. However, owing to the bulky size and split design, to collect the mechanical energy from the bogie frame movement is still a considerable challenge for conventional harvesters. Here, we proposed a compact all-in-one on-rotor electromagnetic energy harvester. The key novelty is that a counterweight acts as the friction pendulum to produce the desired relative motion between the coils and magnet and make the device more easily install on the wheelset. Besides, the layout of the magnetic materials and coils is optimized to improve the conversion efficiency. The output performance under broad train speeds of 420–820 rpm is systematically studied to verify the improvements of the power density (up to 1982 W m−3), and the converted electricity successfully powers the daily electric appliance and the commercial wireless Bluetooth sensors. Additionally, the harvester serves as a speed sensor to detect the motion state of the vehicle. This work makes significant progress towards potential applications in the embedded self-powered wireless condition monitoring units.