Do variations in allocation concealment methods influence the effects found in intervention reviews?

Objective (i) To explore any discrepancies in intervention effects between a set of Cochrane reviews that includes trials with liberal criteria and a set with restrictive criteria in which trials with liberal design have been removed from the review, and (ii) to suggest ways to improve the quality of evidence. Methods A documentary analysis of three Cochrane reviews of intervention studies. The selection of the Cochrane reviews was based on a two-stage sampling. The stability of effect measures after removal of trials with liberal design was investigated. Results In two of the three reviews, we found changes in the original effect measure of the intervention after removing the studies without allocation concealment. One of these reported an 87% greater relative risk when randomized trials with liberal design were included. In the other, the risk was 19.5% lower when randomized trials with liberal design were included. Conclusions The instability of the effect measure indicates the importance of allocation concealment during recruitment for clinical trials. We recommend further research incorporating a large number of intervention reviews and factors other than allocation concealment.