G-quadruplexes (GQs) are four-stranded noncanonical DNA and RNA architectures that can be formed by guanine-rich sequences. The stability of GQs increases with the number of G-quartets, and three G-quartets generally form stable GQs. However, the stability of two-quartet GQs is an open issue. To understand the intrinsic stability of two-quartet GQ stems, we have carried out a series of unbiased molecular dynamics (MD) simulations (505 μs in total) of two- and four-quartet DNA and RNA GQs, with attention paid mainly to parallel-stranded arrangements. We used AMBER DNA parmOL15 and RNA parmOL3 force fields and tested different ion and water models. Two-quartet parallel-stranded DNA GQs unfolded in all the simulations, while the equivalent RNA GQ was stable in most of the simulations. GQs composed of two stacked units of two-quartet GQs were stable for both DNA and RNA. The simulations suggest that a minimum of three quartets are needed to form an intrinsically stable all-anti parallel-stranded DNA GQ. Parallel two-quartet DNA GQ may exist if substantially stabilized by another molecule or structural element, including multimerization. On the other hand, we predict that isolated RNA two-quartet parallel GQs may form, albeit being weakly stable. We also show that ionic parameters and water models should be chosen with caution because some parameter combinations can cause spurious instability of GQ stems. Some in-so-far unnoticed limitations of force-field description of multiple ions inside the GQs are discussed, which compromise the capability of simulations to fully capture the effect of increase in the number of quartets on the GQ stability.