In the phase measuring deflectometry, two groups of fringe patterns in orthogonal directions are usually applied to establish the correspondences between the pixel pairs on the screen and camera. Usually, 16 phase-shifting fringe patterns with different spatial frequencies are required in order to calculate the absolute phases in the conventional temporal phase unwrapping algorithms. This requirement makes the measurement inefficient and not robust against environmental noise. In this paper, an efficient phase retrieval strategy is developed, which requires only six fringe patterns. The modulating information in one-direction is obtained by first using four fringe patterns, and then it is applied to assist the phase calculation in the other direction, so that only two extra fringe patterns are needed. Subsequently the phases are unwrapped by using the geometric constraints of the software configurable optical test system without additional image acquisition. The measurement time is saved by 5/8, compared to the conventional methods. In this way, the influence of the low-frequency disturbances can be suppressed in the workshop condition. Experiments demonstrate that the proposed method can reliably retrieve the absolute phases, and it is of significance to improve the measuring efficiency and stability of in situ deflectometry.