Quantitative detection of defects in structures is always a hot research topic in the field of guided wave inverse scattering. Research studies on how to effectively extract the defect-related information encompassed in the multi-frequency and multi-modes scattered wave signals for reconstructions of defects have been paid attention in recent decades. In this paper, a novel deep learning-based quantitative guided wave inverse scattering technique has been proposed to intelligently realize the end-to-end mapping of the multi-frequency, multi-modes scattered signals to defect profiles with high levels of accuracy and efficiency. Based on the manifold distribution principle, the data patterns of scattered SH-wave signals have been investigated, owing to leveraging the capability of the intelligent encoder-projection-decoder neural network. Following that, the manifold-learning-oriented network has been trained using the data generated by the modified boundary element method. Several numerical examples have been examined to demonstrate the correctness and efficiency of the proposed reconstruction approach. It has been concluded that this novel data-driven technique intelligently enables the high-quality solution to inverse scattering problems and provides valuable insight into the development of practical approaches to quantitative detection using multi-frequency and multi-modal acoustic data from scattered ultrasonic guided waves.