With the rapid development of modern manufacturing processes, ultra-precision structured freeform surfaces are being widely explored for components with special surface functioning. Measurement of the 3D surface form of structured specular objects remains a challenge because of the complexity of the surface form. Benefiting from a high dynamic range and large measuring area, phase measurement deflectometry (PMD) exhibits great potential in the inspection of the specular surfaces. However, the PMD is insensitive to object height, which leads to the PMD only being used for smooth specular surface measurement. Direct phase measurement deflectometry (DPMD) has been introduced to measure structured specular surfaces, but the surface form measurement resolution and accuracy are limited. This paper presents a method named stereo-DPMD for measuring structured specular objects by introducing a stereo deflectometor into DPMD, so that it combines the advantages of slope integration of the stereo deflectometry and discontinuous height measurement from DPMD. The measured object is separated into individual continuous regions, so the surface form of each region can be recovered precisely by slope integration. Then, the relative positions between different regions are evaluated by DPMD system to reconstruct the final 3D shape of the object. Experimental results show that the structured specular surfaces can be measured accurately by the proposed stereo-DPMD method.