Benchmarking Routine Psychological Services: A Discussion of Challenges and Methods

Jaime Delgadillo, Dean McMillan, Chris Leach, Michael Lucock, Simon Gilbody, Nick Wood

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

Background: Policy developments in recent years have led to important changes in the level of access to evidence-based psychological treatments. Several methods have been used to investigate the effectiveness of these treatments in routine care, with different approaches to outcome definition and data analysis. Aims: To present a review of challenges and methods for the evaluation of evidence-based treatments delivered in routine mental healthcare. This is followed by a case example of a benchmarking method applied in primary care. Method: High, average and poor performance benchmarks were calculated through a meta-analysis of published data from services working under the Improving Access to Psychological Therapies (IAPT) Programme in England. Pre-post treatment effect sizes (ES) and confidence intervals were estimated to illustrate a benchmarking method enabling services to evaluate routine clinical outcomes. Results: High, average and poor performance ES for routine IAPT services were estimated to be 0.91, 0.73 and 0.46 for depression (using PHQ-9) and 1.02, 0.78 and 0.52 for anxiety (using GAD-7). Data from one specific IAPT service exemplify how to evaluate and contextualize routine clinical performance against these benchmarks. Conclusions: The main contribution of this report is to summarize key recommendations for the selection of an adequate set of psychometric measures, the operational definition of outcomes, and the statistical evaluation of clinical performance. A benchmarking method is also presented, which may enable a robust evaluation of clinical performance against national benchmarks. Some limitations concerned significant heterogeneity among data sources, and wide variations in ES and data completeness.
LanguageEnglish
Pages16-30
Number of pages15
JournalBehavioural and Cognitive Psychotherapy
Volume42
Issue number1
Early online date24 Oct 2012
DOIs
Publication statusPublished - Jan 2014

Fingerprint

Benchmarking
Psychology
Policy Making
Information Storage and Retrieval
Psychometrics
England
Meta-Analysis
Primary Health Care
Therapeutics
Anxiety
Confidence Intervals
Depression
Delivery of Health Care

Cite this

Delgadillo, Jaime ; McMillan, Dean ; Leach, Chris ; Lucock, Michael ; Gilbody, Simon ; Wood, Nick. / Benchmarking Routine Psychological Services : A Discussion of Challenges and Methods. In: Behavioural and Cognitive Psychotherapy. 2014 ; Vol. 42, No. 1. pp. 16-30.
@article{bfc759aa8d934162ae40df21189536ee,
title = "Benchmarking Routine Psychological Services: A Discussion of Challenges and Methods",
abstract = "Background: Policy developments in recent years have led to important changes in the level of access to evidence-based psychological treatments. Several methods have been used to investigate the effectiveness of these treatments in routine care, with different approaches to outcome definition and data analysis. Aims: To present a review of challenges and methods for the evaluation of evidence-based treatments delivered in routine mental healthcare. This is followed by a case example of a benchmarking method applied in primary care. Method: High, average and poor performance benchmarks were calculated through a meta-analysis of published data from services working under the Improving Access to Psychological Therapies (IAPT) Programme in England. Pre-post treatment effect sizes (ES) and confidence intervals were estimated to illustrate a benchmarking method enabling services to evaluate routine clinical outcomes. Results: High, average and poor performance ES for routine IAPT services were estimated to be 0.91, 0.73 and 0.46 for depression (using PHQ-9) and 1.02, 0.78 and 0.52 for anxiety (using GAD-7). Data from one specific IAPT service exemplify how to evaluate and contextualize routine clinical performance against these benchmarks. Conclusions: The main contribution of this report is to summarize key recommendations for the selection of an adequate set of psychometric measures, the operational definition of outcomes, and the statistical evaluation of clinical performance. A benchmarking method is also presented, which may enable a robust evaluation of clinical performance against national benchmarks. Some limitations concerned significant heterogeneity among data sources, and wide variations in ES and data completeness.",
keywords = "Psychological interventions, Mental health, evidence-based treatment",
author = "Jaime Delgadillo and Dean McMillan and Chris Leach and Michael Lucock and Simon Gilbody and Nick Wood",
year = "2014",
month = "1",
doi = "10.1017/S135246581200080X",
language = "English",
volume = "42",
pages = "16--30",
journal = "Behavioural and Cognitive Psychotherapy",
issn = "1352-4658",
publisher = "Cambridge University Press",
number = "1",

}

Benchmarking Routine Psychological Services : A Discussion of Challenges and Methods. / Delgadillo, Jaime; McMillan, Dean; Leach, Chris; Lucock, Michael; Gilbody, Simon; Wood, Nick.

In: Behavioural and Cognitive Psychotherapy, Vol. 42, No. 1, 01.2014, p. 16-30.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Benchmarking Routine Psychological Services

T2 - Behavioural and Cognitive Psychotherapy

AU - Delgadillo, Jaime

AU - McMillan, Dean

AU - Leach, Chris

AU - Lucock, Michael

AU - Gilbody, Simon

AU - Wood, Nick

PY - 2014/1

Y1 - 2014/1

N2 - Background: Policy developments in recent years have led to important changes in the level of access to evidence-based psychological treatments. Several methods have been used to investigate the effectiveness of these treatments in routine care, with different approaches to outcome definition and data analysis. Aims: To present a review of challenges and methods for the evaluation of evidence-based treatments delivered in routine mental healthcare. This is followed by a case example of a benchmarking method applied in primary care. Method: High, average and poor performance benchmarks were calculated through a meta-analysis of published data from services working under the Improving Access to Psychological Therapies (IAPT) Programme in England. Pre-post treatment effect sizes (ES) and confidence intervals were estimated to illustrate a benchmarking method enabling services to evaluate routine clinical outcomes. Results: High, average and poor performance ES for routine IAPT services were estimated to be 0.91, 0.73 and 0.46 for depression (using PHQ-9) and 1.02, 0.78 and 0.52 for anxiety (using GAD-7). Data from one specific IAPT service exemplify how to evaluate and contextualize routine clinical performance against these benchmarks. Conclusions: The main contribution of this report is to summarize key recommendations for the selection of an adequate set of psychometric measures, the operational definition of outcomes, and the statistical evaluation of clinical performance. A benchmarking method is also presented, which may enable a robust evaluation of clinical performance against national benchmarks. Some limitations concerned significant heterogeneity among data sources, and wide variations in ES and data completeness.

AB - Background: Policy developments in recent years have led to important changes in the level of access to evidence-based psychological treatments. Several methods have been used to investigate the effectiveness of these treatments in routine care, with different approaches to outcome definition and data analysis. Aims: To present a review of challenges and methods for the evaluation of evidence-based treatments delivered in routine mental healthcare. This is followed by a case example of a benchmarking method applied in primary care. Method: High, average and poor performance benchmarks were calculated through a meta-analysis of published data from services working under the Improving Access to Psychological Therapies (IAPT) Programme in England. Pre-post treatment effect sizes (ES) and confidence intervals were estimated to illustrate a benchmarking method enabling services to evaluate routine clinical outcomes. Results: High, average and poor performance ES for routine IAPT services were estimated to be 0.91, 0.73 and 0.46 for depression (using PHQ-9) and 1.02, 0.78 and 0.52 for anxiety (using GAD-7). Data from one specific IAPT service exemplify how to evaluate and contextualize routine clinical performance against these benchmarks. Conclusions: The main contribution of this report is to summarize key recommendations for the selection of an adequate set of psychometric measures, the operational definition of outcomes, and the statistical evaluation of clinical performance. A benchmarking method is also presented, which may enable a robust evaluation of clinical performance against national benchmarks. Some limitations concerned significant heterogeneity among data sources, and wide variations in ES and data completeness.

KW - Psychological interventions

KW - Mental health

KW - evidence-based treatment

U2 - 10.1017/S135246581200080X

DO - 10.1017/S135246581200080X

M3 - Article

VL - 42

SP - 16

EP - 30

JO - Behavioural and Cognitive Psychotherapy

JF - Behavioural and Cognitive Psychotherapy

SN - 1352-4658

IS - 1

ER -