A compact bandpass filter employing a novel ridged half-mode substrate integrated waveguide (RHMSIW) and the planar mixed electric and magnetic coupling is presented. The RHMSIW is proposed by integrating a ridge into the half-mode substrate integrated waveguide (HMSIW), where the ridge introduces a large capacitance in the cavity and hence leads to a lower cutoff frequency and a more compact size compared to the conventional HMSIW. Based on the RHMSIW, a fourth-order filter with the mixed coupling is designed and fabricated. The mixed coupling is realized by using an embedded open-ended strip line combined with an inductive window between adjacent RHMSIW cavities. Two transmission zeros are achieved in the upper stopband by using the mixed coupling. It is shown that the proposed RHMSIW filter is compact, low loss, easy to fabricate and integrate with other electronic systems.