Fault diagnosis of rolling bearings has attracted extensive attention in industrial fields, which plays a vital role in guaranteeing the reliability, safety, and economical efficiency of mechanical systems. Traditional data-driven fault diagnosis methods require obtaining a dataset of full failure modes in advance as the training data. However, this kind of dataset is not always available in some critical industrial scenarios, which impairs the practicability of the data-driven fault diagnosis methods for various applications. A digital twin, which establishes a virtual representation of a physical entity to mirror its operating conditions, would make fault diagnosis of rolling bearings feasible when the fault data are insufficient. In this paper, we propose a novel digital twin-driven approach for implementing fault diagnosis of rolling bearings with insufficient training data. First, a dynamics-based virtual representation of rolling bearings is built to generate simulated data. Then, a Transformer-based network is developed to learn the knowledge of the simulated data for diagnostics. Meanwhile, a selective adversarial strategy is introduced to achieve cross-domain feature alignments in scenarios where the health conditions of the measured data are unknown. To this end, this study proposes a digital twin-driven fault diagnosis framework by using labeled simulated data and unlabeled measured data. The experimental results show that the proposed method can obtain high diagnostic performance when the real-world data is unlabeled and has unknown health conditions, proving that the proposed method has significant benefits for the health management of critical rolling bearings.