Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms

P. Jonathon Phillips, Amy N. Yates, Ying Hu, Carina A. Hahn, Eilidh Noyes, Kelsey Jackson, Jacqueline G. Cavazos, Géraldine Jeckeln, Rajeev Ranjan, Swami Sankaranarayanan, Jun-Cheng Chen, Carlos D. Castillo, Rama Chellappa, David White, Alice J. O’Toole

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

This study measures face identification accuracy for an international group of professional forensic facial examiners working under circumstances that apply in real world casework. Examiners and other human face “specialists,” including forensically trained facial reviewers and untrained superrecognizers, were more accurate than the control groups on a challenging test of face identification. Therefore, specialists are the best available human solution to the problem of face identification. We present data comparing state-of-the-art face recognition technology with the best human face identifiers. The best machine performed in the range of the best humans: professional facial examiners. However, optimal face identification was achieved only when humans and machines worked in collaboration. Achieving the upper limits of face identification accuracy in forensic applications can minimize errors that have profound social and personal consequences. Although forensic examiners identify faces in these applications, systematic tests of their accuracy are rare. How can we achieve the most accurate face identification: using people and/or machines working alone or in collaboration? In a comprehensive comparison of face identification by humans and computers, we found that forensic facial examiners, facial reviewers, and superrecognizers were more accurate than fingerprint examiners and students on a challenging face identification test. Individual performance on the test varied widely. On the same test, four deep convolutional neural networks (DCNNs), developed between 2015 and 2017, identified faces within the range of human accuracy. Accuracy of the algorithms increased steadily over time, with the most recent DCNN scoring above the median of the forensic facial examiners. Using crowd-sourcing methods, we fused the judgments of multiple forensic facial examiners by averaging their rating-based identity judgments. Accuracy was substantially better for fused judgments than for individuals working alone. Fusion also served to stabilize performance, boosting the scores of lower-performing individuals and decreasing variability. Single forensic facial examiners fused with the best algorithm were more accurate than the combination of two examiners. Therefore, collaboration among humans and between humans and machines offers tangible benefits to face identification accuracy in important applications. These results offer an evidence-based roadmap for achieving the most accurate face identification possible.
LanguageEnglish
Pages6171-6176
Number of pages6
JournalProceedings of the National Academy of Sciences of the United States of America
Volume115
Issue number24
DOIs
Publication statusPublished - 12 Jun 2018
Externally publishedYes

Fingerprint

Crowdsourcing
Facial Recognition
Forensic Anthropology
Dermatoglyphics
Students
Technology
Control Groups

Cite this

Phillips, P. Jonathon ; Yates, Amy N. ; Hu, Ying ; Hahn, Carina A. ; Noyes, Eilidh ; Jackson, Kelsey ; Cavazos, Jacqueline G. ; Jeckeln, Géraldine ; Ranjan, Rajeev ; Sankaranarayanan, Swami ; Chen, Jun-Cheng ; Castillo, Carlos D. ; Chellappa, Rama ; White, David ; O’Toole, Alice J. / Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms. In: Proceedings of the National Academy of Sciences of the United States of America. 2018 ; Vol. 115, No. 24. pp. 6171-6176.
@article{a0495d64a9e44f93bc0f76c335afacf3,
title = "Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms",
abstract = "This study measures face identification accuracy for an international group of professional forensic facial examiners working under circumstances that apply in real world casework. Examiners and other human face “specialists,” including forensically trained facial reviewers and untrained superrecognizers, were more accurate than the control groups on a challenging test of face identification. Therefore, specialists are the best available human solution to the problem of face identification. We present data comparing state-of-the-art face recognition technology with the best human face identifiers. The best machine performed in the range of the best humans: professional facial examiners. However, optimal face identification was achieved only when humans and machines worked in collaboration. Achieving the upper limits of face identification accuracy in forensic applications can minimize errors that have profound social and personal consequences. Although forensic examiners identify faces in these applications, systematic tests of their accuracy are rare. How can we achieve the most accurate face identification: using people and/or machines working alone or in collaboration? In a comprehensive comparison of face identification by humans and computers, we found that forensic facial examiners, facial reviewers, and superrecognizers were more accurate than fingerprint examiners and students on a challenging face identification test. Individual performance on the test varied widely. On the same test, four deep convolutional neural networks (DCNNs), developed between 2015 and 2017, identified faces within the range of human accuracy. Accuracy of the algorithms increased steadily over time, with the most recent DCNN scoring above the median of the forensic facial examiners. Using crowd-sourcing methods, we fused the judgments of multiple forensic facial examiners by averaging their rating-based identity judgments. Accuracy was substantially better for fused judgments than for individuals working alone. Fusion also served to stabilize performance, boosting the scores of lower-performing individuals and decreasing variability. Single forensic facial examiners fused with the best algorithm were more accurate than the combination of two examiners. Therefore, collaboration among humans and between humans and machines offers tangible benefits to face identification accuracy in important applications. These results offer an evidence-based roadmap for achieving the most accurate face identification possible.",
keywords = "Face identification, Face recognition algorithm, Forensic science, Machine learning technology, Wisdom-of-crowds",
author = "Phillips, {P. Jonathon} and Yates, {Amy N.} and Ying Hu and Hahn, {Carina A.} and Eilidh Noyes and Kelsey Jackson and Cavazos, {Jacqueline G.} and G{\'e}raldine Jeckeln and Rajeev Ranjan and Swami Sankaranarayanan and Jun-Cheng Chen and Castillo, {Carlos D.} and Rama Chellappa and David White and O’Toole, {Alice J.}",
year = "2018",
month = "6",
day = "12",
doi = "10.1073/pnas.1721355115",
language = "English",
volume = "115",
pages = "6171--6176",
journal = "Proceedings of the National Academy of Sciences of the United States of America",
issn = "0027-8424",
publisher = "National Academy of Sciences",
number = "24",

}

Phillips, PJ, Yates, AN, Hu, Y, Hahn, CA, Noyes, E, Jackson, K, Cavazos, JG, Jeckeln, G, Ranjan, R, Sankaranarayanan, S, Chen, J-C, Castillo, CD, Chellappa, R, White, D & O’Toole, AJ 2018, 'Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms', Proceedings of the National Academy of Sciences of the United States of America, vol. 115, no. 24, pp. 6171-6176. https://doi.org/10.1073/pnas.1721355115

Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms. / Phillips, P. Jonathon; Yates, Amy N.; Hu, Ying; Hahn, Carina A.; Noyes, Eilidh; Jackson, Kelsey; Cavazos, Jacqueline G.; Jeckeln, Géraldine; Ranjan, Rajeev; Sankaranarayanan, Swami; Chen, Jun-Cheng; Castillo, Carlos D.; Chellappa, Rama; White, David; O’Toole, Alice J.

In: Proceedings of the National Academy of Sciences of the United States of America, Vol. 115, No. 24, 12.06.2018, p. 6171-6176.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms

AU - Phillips, P. Jonathon

AU - Yates, Amy N.

AU - Hu, Ying

AU - Hahn, Carina A.

AU - Noyes, Eilidh

AU - Jackson, Kelsey

AU - Cavazos, Jacqueline G.

AU - Jeckeln, Géraldine

AU - Ranjan, Rajeev

AU - Sankaranarayanan, Swami

AU - Chen, Jun-Cheng

AU - Castillo, Carlos D.

AU - Chellappa, Rama

AU - White, David

AU - O’Toole, Alice J.

PY - 2018/6/12

Y1 - 2018/6/12

N2 - This study measures face identification accuracy for an international group of professional forensic facial examiners working under circumstances that apply in real world casework. Examiners and other human face “specialists,” including forensically trained facial reviewers and untrained superrecognizers, were more accurate than the control groups on a challenging test of face identification. Therefore, specialists are the best available human solution to the problem of face identification. We present data comparing state-of-the-art face recognition technology with the best human face identifiers. The best machine performed in the range of the best humans: professional facial examiners. However, optimal face identification was achieved only when humans and machines worked in collaboration. Achieving the upper limits of face identification accuracy in forensic applications can minimize errors that have profound social and personal consequences. Although forensic examiners identify faces in these applications, systematic tests of their accuracy are rare. How can we achieve the most accurate face identification: using people and/or machines working alone or in collaboration? In a comprehensive comparison of face identification by humans and computers, we found that forensic facial examiners, facial reviewers, and superrecognizers were more accurate than fingerprint examiners and students on a challenging face identification test. Individual performance on the test varied widely. On the same test, four deep convolutional neural networks (DCNNs), developed between 2015 and 2017, identified faces within the range of human accuracy. Accuracy of the algorithms increased steadily over time, with the most recent DCNN scoring above the median of the forensic facial examiners. Using crowd-sourcing methods, we fused the judgments of multiple forensic facial examiners by averaging their rating-based identity judgments. Accuracy was substantially better for fused judgments than for individuals working alone. Fusion also served to stabilize performance, boosting the scores of lower-performing individuals and decreasing variability. Single forensic facial examiners fused with the best algorithm were more accurate than the combination of two examiners. Therefore, collaboration among humans and between humans and machines offers tangible benefits to face identification accuracy in important applications. These results offer an evidence-based roadmap for achieving the most accurate face identification possible.

AB - This study measures face identification accuracy for an international group of professional forensic facial examiners working under circumstances that apply in real world casework. Examiners and other human face “specialists,” including forensically trained facial reviewers and untrained superrecognizers, were more accurate than the control groups on a challenging test of face identification. Therefore, specialists are the best available human solution to the problem of face identification. We present data comparing state-of-the-art face recognition technology with the best human face identifiers. The best machine performed in the range of the best humans: professional facial examiners. However, optimal face identification was achieved only when humans and machines worked in collaboration. Achieving the upper limits of face identification accuracy in forensic applications can minimize errors that have profound social and personal consequences. Although forensic examiners identify faces in these applications, systematic tests of their accuracy are rare. How can we achieve the most accurate face identification: using people and/or machines working alone or in collaboration? In a comprehensive comparison of face identification by humans and computers, we found that forensic facial examiners, facial reviewers, and superrecognizers were more accurate than fingerprint examiners and students on a challenging face identification test. Individual performance on the test varied widely. On the same test, four deep convolutional neural networks (DCNNs), developed between 2015 and 2017, identified faces within the range of human accuracy. Accuracy of the algorithms increased steadily over time, with the most recent DCNN scoring above the median of the forensic facial examiners. Using crowd-sourcing methods, we fused the judgments of multiple forensic facial examiners by averaging their rating-based identity judgments. Accuracy was substantially better for fused judgments than for individuals working alone. Fusion also served to stabilize performance, boosting the scores of lower-performing individuals and decreasing variability. Single forensic facial examiners fused with the best algorithm were more accurate than the combination of two examiners. Therefore, collaboration among humans and between humans and machines offers tangible benefits to face identification accuracy in important applications. These results offer an evidence-based roadmap for achieving the most accurate face identification possible.

KW - Face identification

KW - Face recognition algorithm

KW - Forensic science

KW - Machine learning technology

KW - Wisdom-of-crowds

UR - http://www.scopus.com/inward/record.url?scp=85048558031&partnerID=8YFLogxK

U2 - 10.1073/pnas.1721355115

DO - 10.1073/pnas.1721355115

M3 - Article

VL - 115

SP - 6171

EP - 6176

JO - Proceedings of the National Academy of Sciences of the United States of America

T2 - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 24

ER -