A new technique, the instantaneous wavelet bicoherence (WB) is proposed and investigated. The use of the instantaneous and locally averaged WB from vibration measurements for local damage detection in gears is investigated for the first time; these bicoherences are better adapted than the classical Fourier bicoherence to the case of non-stationary signals. A new diagnostic feature based on the integrated modulus of the WB in a specific frequency range and a methodology for feature estimation are proposed. The WB techniques are applied to detection of a multiple like natural pitting on a back-to-back industrial spur gearbox system and natural pitting on a gear at test rig and show the possibility of early detection of local tooth faults. The detection effectiveness is evaluated by a local Fisher criterion estimated at each angular position of gear for the unpitted and pitted cases. The proposed WB-based diagnostic feature demonstrates robust experimental performance and superior detection capabilities (i.e., effective early damage detection differentially for teeth of the gear wheel) over the conventional detection methods based on the wavelet transform. The reason for this superior effectiveness is that the WB exploits the phase couplings of the wavelet transform at different frequencies, which contain useful additional information for detection of non-linear phenomena induced by local faults. The proposed approaches are compared with the two conventional approaches based on the wavelet transform.