Polygyny and multiple marital unions are generally associated with overall happiness deficits, yet the rural-urban differentials of these associations are not thoroughly interrogated. This study uses a nationally representative dataset of 7785 married and in-union Ghanaian women aged 15–49 years to investigate the rural-urban differences in the relationship between polygyny, multiple marital unions, and happiness. Findings show that rural married and in-union women in polygynous relationships [APOR = 0.760, 95% CI:0.60, 0.96] and were married more than once [APOR = 0.79, 95% CI:0.66, 0.94] were less likely to be happy. Age, education, household wealth, and region of residence were significantly associated with happiness only among rural married women. Both rural and urban married and in-union women in the Upper West and Upper East Regions were more likely to be happy than married women in the Greater Accra Region. Policies should enable rural women in polygyny and those in their second or more marital unions with happiness-enabling resources.