Internal radial clearance is a key factor influencing bearing fatigue life. Moreover, bearings inevitably suffer from various wears and tears, which result in gradual increase of clearance and shorten bearing life. Monitoring bearing clearance changes using vibration can effectively indicate the bearing wear and provide good leading time to perform maintenances. Previous studies show that vibration at ball pass frequency on outer race (BPFO) can be based for clearance monitoring. However, such clearance induced vibration has not been well understood, especially under complicated dynamic interactions such as in a gearbox system. To fill this gap, this paper presents a nonlinear gear-shaft-bearing-housing vibration model with fourteen degree of freedom (DOF) to investigate the vibration responses under the dynamic gear meshing force and progressively changed radial clearances at first. Then, the model was verified through a two-stage spur gearbox. Furthermore, bearing characteristics with different radial clearances under the influence of gear are revealed and indicator based on modulation signal bispectrum-sideband estimator (MSB-SE) was proposed. Finally, vibration data from a run-to-failure gearbox test rig was utilized to verify the effectiveness of the MSB-SE indicator for bearing clearances monitoring. Simulation results show that BPFO is modulated on gear meshing frequency (GMF) and BPFO amplitude from envelope spectrum increases with bearing clearances under the influence of gear meshing. Indicator based on MSB-SE, possessing the capability of purifying the interferences of gear meshing and strong noises, is effective to capture the variance of bearing clearances. The experiment based on a run-to-failure gearbox test rig provided evidence for the effectiveness of the proposed indicator, which is more accurate than BPFO amplitude from conventional envelope analysis and time-domain indicators, such as RMS and kurtosis. These findings are of significance for bearing fault diagnosis and maintenance.