This paper investigates the reliability and security performance of the downlink non-orthogonal multiple access (NOMA) networks over Nakagami-m fading channels, where the base station (BS) aims to communicate with multi-antenna NOMA users in the presence of a multi-antenna eavesdropper. To be more practical, a detrimental factor at both transmitter and receiver is considered, namely in-phase and quadrature-phase imbalance (IQI). To further improve the reliability and security of the considered networks, the selection combining (SC) algorithm at the receiver is taken into account. More specifically, the exact analytical expressions for the outage probability (OP) and the intercept probability (IP) are derived in closed-form. To obtain a better understanding of the influence for the IQI parameters on the system performance, the asymptotic behaviors for the outage probabilities (OPs) in the high signal-to-noise ratio (SNR) region are analyzed. Based on the asymptotic results, the diversity order of the considered system are obtained and discussed. The numerical results are presented to verify the validity of the theoretical analysis.