Effects of psychological therapies in randomized trials and practice-based studies

Michael Barkham, William B. Stiles, Janice Connell, Elspeth Twigg, Chris Leach, Mike Lucock, John Mellor-Clark, Peter Bower, Michael King, David A. Shapiro, Gillian E. Hardy, Leslie Greenberg, Lynne Angus

Research output: Contribution to journalArticle

44 Citations (Scopus)

Abstract

Background: Randomized trials of the effects of psychological therapies seek internal validity via homogeneous samples and standardized treatment protocols. In contrast, practice-based studies aim for clinical realism and external validity via heterogeneous samples of clients treated under routine practice conditions. We compared indices of treatment effects in these two types of studies. Method: Using published transformation formulas, the Beck Depression Inventory (BDI) scores from five randomized trials of depression (N = 477 clients) were transformed into Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) scores and compared with CORE-OM data collected in four practice-based studies (N = 4,196 clients). Conversely, the practice-based studies' CORE-OM scores were transformed into BDI scores and compared with randomized trial data. Results: Randomized trials showed a modest advantage over practice-based studies in amount of pre-post improvement. This difference was compressed or exaggerated depending on the direction of the transformation but averaged about 12 There was a similarly sized advantage to randomized trials in rates of reliable and clinically significant improvement (RCSI). The largest difference was yielded by comparisons of effect sizes which suggested an advantage more than twice as large, reflecting narrower pre-treatment distributions in the randomized trials. Conclusions: Outcomes of completed treatments for depression in randomized trials appeared to be modestly greater than those in routine care settings. The size of the difference may be distorted depending on the method for calculating degree of change. Transforming BDI scores into CORE-OM scores and vice versa may be a preferable alternative to effect sizes for comparisons of studies using these measures.
LanguageEnglish
Pages397-415
Number of pages19
JournalBritish Journal of Clinical Psychology
Volume47
Issue number4
DOIs
Publication statusPublished - Nov 2008

Fingerprint

Outcome Assessment (Health Care)
Psychology
Equipment and Supplies
Therapeutics
Clinical Protocols
Clinical Studies

Cite this

Barkham, Michael ; Stiles, William B. ; Connell, Janice ; Twigg, Elspeth ; Leach, Chris ; Lucock, Mike ; Mellor-Clark, John ; Bower, Peter ; King, Michael ; Shapiro, David A. ; Hardy, Gillian E. ; Greenberg, Leslie ; Angus, Lynne. / Effects of psychological therapies in randomized trials and practice-based studies. In: British Journal of Clinical Psychology. 2008 ; Vol. 47, No. 4. pp. 397-415.
@article{de72e78c7f1a4fabb47e31c22acc7a15,
title = "Effects of psychological therapies in randomized trials and practice-based studies",
abstract = "Background: Randomized trials of the effects of psychological therapies seek internal validity via homogeneous samples and standardized treatment protocols. In contrast, practice-based studies aim for clinical realism and external validity via heterogeneous samples of clients treated under routine practice conditions. We compared indices of treatment effects in these two types of studies. Method: Using published transformation formulas, the Beck Depression Inventory (BDI) scores from five randomized trials of depression (N = 477 clients) were transformed into Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) scores and compared with CORE-OM data collected in four practice-based studies (N = 4,196 clients). Conversely, the practice-based studies' CORE-OM scores were transformed into BDI scores and compared with randomized trial data. Results: Randomized trials showed a modest advantage over practice-based studies in amount of pre-post improvement. This difference was compressed or exaggerated depending on the direction of the transformation but averaged about 12 There was a similarly sized advantage to randomized trials in rates of reliable and clinically significant improvement (RCSI). The largest difference was yielded by comparisons of effect sizes which suggested an advantage more than twice as large, reflecting narrower pre-treatment distributions in the randomized trials. Conclusions: Outcomes of completed treatments for depression in randomized trials appeared to be modestly greater than those in routine care settings. The size of the difference may be distorted depending on the method for calculating degree of change. Transforming BDI scores into CORE-OM scores and vice versa may be a preferable alternative to effect sizes for comparisons of studies using these measures.",
author = "Michael Barkham and Stiles, {William B.} and Janice Connell and Elspeth Twigg and Chris Leach and Mike Lucock and John Mellor-Clark and Peter Bower and Michael King and Shapiro, {David A.} and Hardy, {Gillian E.} and Leslie Greenberg and Lynne Angus",
year = "2008",
month = "11",
doi = "10.1348/014466508X311713",
language = "English",
volume = "47",
pages = "397--415",
journal = "British Journal of Clinical Psychology",
issn = "0144-6657",
publisher = "British Psychological Society",
number = "4",

}

Barkham, M, Stiles, WB, Connell, J, Twigg, E, Leach, C, Lucock, M, Mellor-Clark, J, Bower, P, King, M, Shapiro, DA, Hardy, GE, Greenberg, L & Angus, L 2008, 'Effects of psychological therapies in randomized trials and practice-based studies', British Journal of Clinical Psychology, vol. 47, no. 4, pp. 397-415. https://doi.org/10.1348/014466508X311713

Effects of psychological therapies in randomized trials and practice-based studies. / Barkham, Michael; Stiles, William B.; Connell, Janice; Twigg, Elspeth; Leach, Chris; Lucock, Mike; Mellor-Clark, John; Bower, Peter; King, Michael; Shapiro, David A.; Hardy, Gillian E.; Greenberg, Leslie; Angus, Lynne.

In: British Journal of Clinical Psychology, Vol. 47, No. 4, 11.2008, p. 397-415.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Effects of psychological therapies in randomized trials and practice-based studies

AU - Barkham, Michael

AU - Stiles, William B.

AU - Connell, Janice

AU - Twigg, Elspeth

AU - Leach, Chris

AU - Lucock, Mike

AU - Mellor-Clark, John

AU - Bower, Peter

AU - King, Michael

AU - Shapiro, David A.

AU - Hardy, Gillian E.

AU - Greenberg, Leslie

AU - Angus, Lynne

PY - 2008/11

Y1 - 2008/11

N2 - Background: Randomized trials of the effects of psychological therapies seek internal validity via homogeneous samples and standardized treatment protocols. In contrast, practice-based studies aim for clinical realism and external validity via heterogeneous samples of clients treated under routine practice conditions. We compared indices of treatment effects in these two types of studies. Method: Using published transformation formulas, the Beck Depression Inventory (BDI) scores from five randomized trials of depression (N = 477 clients) were transformed into Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) scores and compared with CORE-OM data collected in four practice-based studies (N = 4,196 clients). Conversely, the practice-based studies' CORE-OM scores were transformed into BDI scores and compared with randomized trial data. Results: Randomized trials showed a modest advantage over practice-based studies in amount of pre-post improvement. This difference was compressed or exaggerated depending on the direction of the transformation but averaged about 12 There was a similarly sized advantage to randomized trials in rates of reliable and clinically significant improvement (RCSI). The largest difference was yielded by comparisons of effect sizes which suggested an advantage more than twice as large, reflecting narrower pre-treatment distributions in the randomized trials. Conclusions: Outcomes of completed treatments for depression in randomized trials appeared to be modestly greater than those in routine care settings. The size of the difference may be distorted depending on the method for calculating degree of change. Transforming BDI scores into CORE-OM scores and vice versa may be a preferable alternative to effect sizes for comparisons of studies using these measures.

AB - Background: Randomized trials of the effects of psychological therapies seek internal validity via homogeneous samples and standardized treatment protocols. In contrast, practice-based studies aim for clinical realism and external validity via heterogeneous samples of clients treated under routine practice conditions. We compared indices of treatment effects in these two types of studies. Method: Using published transformation formulas, the Beck Depression Inventory (BDI) scores from five randomized trials of depression (N = 477 clients) were transformed into Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) scores and compared with CORE-OM data collected in four practice-based studies (N = 4,196 clients). Conversely, the practice-based studies' CORE-OM scores were transformed into BDI scores and compared with randomized trial data. Results: Randomized trials showed a modest advantage over practice-based studies in amount of pre-post improvement. This difference was compressed or exaggerated depending on the direction of the transformation but averaged about 12 There was a similarly sized advantage to randomized trials in rates of reliable and clinically significant improvement (RCSI). The largest difference was yielded by comparisons of effect sizes which suggested an advantage more than twice as large, reflecting narrower pre-treatment distributions in the randomized trials. Conclusions: Outcomes of completed treatments for depression in randomized trials appeared to be modestly greater than those in routine care settings. The size of the difference may be distorted depending on the method for calculating degree of change. Transforming BDI scores into CORE-OM scores and vice versa may be a preferable alternative to effect sizes for comparisons of studies using these measures.

U2 - 10.1348/014466508X311713

DO - 10.1348/014466508X311713

M3 - Article

VL - 47

SP - 397

EP - 415

JO - British Journal of Clinical Psychology

T2 - British Journal of Clinical Psychology

JF - British Journal of Clinical Psychology

SN - 0144-6657

IS - 4

ER -