Implementation of blinded outcome assessment in the Effective Verruca Treatments trial (EverT) - lessons learned

Sarah Cockayne, Catherine Hewitt, Farina Hashmi, Kate Hicks, Michael Concannon, Caroline McIntosh, Kim Thomas, Jill Hall, Judith Watson, David Torgerson, Ian Watt

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Background: Trials using inadequate levels of blinding may report larger effect sizes than blinded studies. It has been suggested that blinded outcome assessment in open trials may in some cases be undertaken by assessments of photographs. The aim of this paper is to explore the effect of using different methods to assess the primary outcome in the EVerT (Effective Verruca Treatments) trial. It also aims to give an overview of the experiences of using digital photographs within the trial. Methods: We undertook a secondary analysis to explore the effect of using three different methods to assess the primary outcome in the EVerT trial: assessment of digital photographs by blinded healthcare professionals; blinded healthcare professional assessment at the recruiting site and patient self-report. The verruca clearance rates were calculated using the three different methods of assessment. A Cohen's kappa measure of inter-rater agreement was used to assess the agreement between the methods. We also investigated the experiences of healthcare professionals using digital photographs within the trial. Results: Digital photographs for 189 out of 240 (79 %) patients in the trial were received for outcome assessment. Of the 189 photographs, 30 (16 %) were uninterpretable. The overall verruca clearance rates were 21 % (43/202,) using the unblinded patient self-reported outcome, 6 % (9/159,) using blinded assessment of digital photographs and 14 % (30/210,) using blinded outcome assessment at the site. Conclusions: Despite differences in the clearance rates found using different methods of outcome assessment, this did not change the original conclusion of the trial, that there is no evidence of a difference in effectiveness between cryotherapy and salicylic acid. Future trials using digital photographs should consider individual training needs at sites and have a backup method of assessment agreed a priori. Trial registration: ISRCTN Registry ISRCTN18994246

Original languageEnglish
Article number21
JournalJournal of Foot and Ankle Research
Volume9
Issue number1
DOIs
Publication statusPublished - 12 Jul 2016

Fingerprint

Warts
Outcome Assessment (Health Care)
Delivery of Health Care
Therapeutics
Cryotherapy
Salicylic Acid
Self Report
Registries

Cite this

Cockayne, Sarah ; Hewitt, Catherine ; Hashmi, Farina ; Hicks, Kate ; Concannon, Michael ; McIntosh, Caroline ; Thomas, Kim ; Hall, Jill ; Watson, Judith ; Torgerson, David ; Watt, Ian. / Implementation of blinded outcome assessment in the Effective Verruca Treatments trial (EverT) - lessons learned. In: Journal of Foot and Ankle Research. 2016 ; Vol. 9, No. 1.
@article{58c5556792ba49b0a3d0ef202ab16093,
title = "Implementation of blinded outcome assessment in the Effective Verruca Treatments trial (EverT) - lessons learned",
abstract = "Background: Trials using inadequate levels of blinding may report larger effect sizes than blinded studies. It has been suggested that blinded outcome assessment in open trials may in some cases be undertaken by assessments of photographs. The aim of this paper is to explore the effect of using different methods to assess the primary outcome in the EVerT (Effective Verruca Treatments) trial. It also aims to give an overview of the experiences of using digital photographs within the trial. Methods: We undertook a secondary analysis to explore the effect of using three different methods to assess the primary outcome in the EVerT trial: assessment of digital photographs by blinded healthcare professionals; blinded healthcare professional assessment at the recruiting site and patient self-report. The verruca clearance rates were calculated using the three different methods of assessment. A Cohen's kappa measure of inter-rater agreement was used to assess the agreement between the methods. We also investigated the experiences of healthcare professionals using digital photographs within the trial. Results: Digital photographs for 189 out of 240 (79 {\%}) patients in the trial were received for outcome assessment. Of the 189 photographs, 30 (16 {\%}) were uninterpretable. The overall verruca clearance rates were 21 {\%} (43/202,) using the unblinded patient self-reported outcome, 6 {\%} (9/159,) using blinded assessment of digital photographs and 14 {\%} (30/210,) using blinded outcome assessment at the site. Conclusions: Despite differences in the clearance rates found using different methods of outcome assessment, this did not change the original conclusion of the trial, that there is no evidence of a difference in effectiveness between cryotherapy and salicylic acid. Future trials using digital photographs should consider individual training needs at sites and have a backup method of assessment agreed a priori. Trial registration: ISRCTN Registry ISRCTN18994246",
keywords = "Blinding, Digital photographs, Outcome assessment, Randomised controlled trial",
author = "Sarah Cockayne and Catherine Hewitt and Farina Hashmi and Kate Hicks and Michael Concannon and Caroline McIntosh and Kim Thomas and Jill Hall and Judith Watson and David Torgerson and Ian Watt",
year = "2016",
month = "7",
day = "12",
doi = "10.1186/s13047-016-0155-4",
language = "English",
volume = "9",
journal = "Journal of Foot and Ankle Research",
issn = "1757-1146",
publisher = "BioMed Central",
number = "1",

}

Cockayne, S, Hewitt, C, Hashmi, F, Hicks, K, Concannon, M, McIntosh, C, Thomas, K, Hall, J, Watson, J, Torgerson, D & Watt, I 2016, 'Implementation of blinded outcome assessment in the Effective Verruca Treatments trial (EverT) - lessons learned', Journal of Foot and Ankle Research, vol. 9, no. 1, 21. https://doi.org/10.1186/s13047-016-0155-4

Implementation of blinded outcome assessment in the Effective Verruca Treatments trial (EverT) - lessons learned. / Cockayne, Sarah; Hewitt, Catherine; Hashmi, Farina; Hicks, Kate; Concannon, Michael; McIntosh, Caroline; Thomas, Kim; Hall, Jill; Watson, Judith; Torgerson, David; Watt, Ian.

In: Journal of Foot and Ankle Research, Vol. 9, No. 1, 21, 12.07.2016.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Implementation of blinded outcome assessment in the Effective Verruca Treatments trial (EverT) - lessons learned

AU - Cockayne, Sarah

AU - Hewitt, Catherine

AU - Hashmi, Farina

AU - Hicks, Kate

AU - Concannon, Michael

AU - McIntosh, Caroline

AU - Thomas, Kim

AU - Hall, Jill

AU - Watson, Judith

AU - Torgerson, David

AU - Watt, Ian

PY - 2016/7/12

Y1 - 2016/7/12

N2 - Background: Trials using inadequate levels of blinding may report larger effect sizes than blinded studies. It has been suggested that blinded outcome assessment in open trials may in some cases be undertaken by assessments of photographs. The aim of this paper is to explore the effect of using different methods to assess the primary outcome in the EVerT (Effective Verruca Treatments) trial. It also aims to give an overview of the experiences of using digital photographs within the trial. Methods: We undertook a secondary analysis to explore the effect of using three different methods to assess the primary outcome in the EVerT trial: assessment of digital photographs by blinded healthcare professionals; blinded healthcare professional assessment at the recruiting site and patient self-report. The verruca clearance rates were calculated using the three different methods of assessment. A Cohen's kappa measure of inter-rater agreement was used to assess the agreement between the methods. We also investigated the experiences of healthcare professionals using digital photographs within the trial. Results: Digital photographs for 189 out of 240 (79 %) patients in the trial were received for outcome assessment. Of the 189 photographs, 30 (16 %) were uninterpretable. The overall verruca clearance rates were 21 % (43/202,) using the unblinded patient self-reported outcome, 6 % (9/159,) using blinded assessment of digital photographs and 14 % (30/210,) using blinded outcome assessment at the site. Conclusions: Despite differences in the clearance rates found using different methods of outcome assessment, this did not change the original conclusion of the trial, that there is no evidence of a difference in effectiveness between cryotherapy and salicylic acid. Future trials using digital photographs should consider individual training needs at sites and have a backup method of assessment agreed a priori. Trial registration: ISRCTN Registry ISRCTN18994246

AB - Background: Trials using inadequate levels of blinding may report larger effect sizes than blinded studies. It has been suggested that blinded outcome assessment in open trials may in some cases be undertaken by assessments of photographs. The aim of this paper is to explore the effect of using different methods to assess the primary outcome in the EVerT (Effective Verruca Treatments) trial. It also aims to give an overview of the experiences of using digital photographs within the trial. Methods: We undertook a secondary analysis to explore the effect of using three different methods to assess the primary outcome in the EVerT trial: assessment of digital photographs by blinded healthcare professionals; blinded healthcare professional assessment at the recruiting site and patient self-report. The verruca clearance rates were calculated using the three different methods of assessment. A Cohen's kappa measure of inter-rater agreement was used to assess the agreement between the methods. We also investigated the experiences of healthcare professionals using digital photographs within the trial. Results: Digital photographs for 189 out of 240 (79 %) patients in the trial were received for outcome assessment. Of the 189 photographs, 30 (16 %) were uninterpretable. The overall verruca clearance rates were 21 % (43/202,) using the unblinded patient self-reported outcome, 6 % (9/159,) using blinded assessment of digital photographs and 14 % (30/210,) using blinded outcome assessment at the site. Conclusions: Despite differences in the clearance rates found using different methods of outcome assessment, this did not change the original conclusion of the trial, that there is no evidence of a difference in effectiveness between cryotherapy and salicylic acid. Future trials using digital photographs should consider individual training needs at sites and have a backup method of assessment agreed a priori. Trial registration: ISRCTN Registry ISRCTN18994246

KW - Blinding

KW - Digital photographs

KW - Outcome assessment

KW - Randomised controlled trial

UR - http://www.scopus.com/inward/record.url?scp=84978119309&partnerID=8YFLogxK

U2 - 10.1186/s13047-016-0155-4

DO - 10.1186/s13047-016-0155-4

M3 - Article

VL - 9

JO - Journal of Foot and Ankle Research

JF - Journal of Foot and Ankle Research

SN - 1757-1146

IS - 1

M1 - 21

ER -