The vibration signal of a faulty rolling bearing exhibits typical non-stationarity - often in the form of cyclostationarity. The spectrum tools often used to characterize cyclostationarity mainly include envelope spectrum, squared envelope spectrum and log-envelope spectrum. In this paper, new detection methods of cyclostationarity are developed for obtaining a larger family of envelope analysis and their effectiveness in rolling bearing fault diagnosis is evaluated rigorously. Firstly, based on the simplified Box-Cox transformation, the generalized envelope signals are constructed from the analytic signal for demodulation purposes, and then a spectrum family named generalized envelope spectra (GESs) is proposed to reveal cyclostationarity. Especially, GESs with different transformation parameters exhibit different performance advantages against the random impulse noise and Gaussian background noise which are commonly present in rolling bearing vibration signals. Subsequently, a novel spectrum tool that combines the performance advantages of different GESs, called product envelope spectrum (PES), is developed to strengthen the capability to detect cyclostationarity. Finally, an enhanced envelope analysis named Product Envelope Spectral Optimization-gram (PESOgram) is proposed to improve the accuracy and robustness of PES for rolling bearing fault diagnosis in the presence of different fault-unrelated interference noises. The performance of the PESOgram method is validated on numerically generated signal and experimental signals collected from two railway axle bearing test rigs and compared with several state-of-the-art envelope analysis methods. The results demonstrate the effectiveness of the proposed method for fault diagnosis of rolling bearings and its advantages over other state-of-the-art methods.