CoChat
Sign Up
Cited by 21

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Likelihood Ratios for Deep Neural Networks in Face Comparison

Macarulla Rodriguez, Andrea, Geradts, Zeno, Worring, Marcel·2020·Journal of Forensic Sciences
DOI 10.1111/1556-4029.14324PMC PMC7383913
Open this paper in CoChat — free account

Abstract

In this study, we aim to compare the performance of systems and forensic facial comparison experts in terms of likelihood ratio computation to assess the potential of the machine to support the human expert in the courtroom. In forensics, transparency in the methods is essential. Consequently, state‐of‐the‐art free software was preferred over commercial software. Three different open‐source automated systems chosen for their availability and clarity were as follows: OpenFace, SeetaFace, and FaceNet; all three based on convolutional neural networks that return a distance (OpenFace, FaceNet) or similarity (SeetaFace). The returned distance or similarity is converted to a likelihood ratio using three different distribution fits: parametric fit Weibull distribution, nonparametric fit kernel density estimation, and isotonic regression with pool adjacent violators algorithm. The results show that with low‐quality frontal images, automated systems have better performance to detect nonmatches than investigators: 100% of precision and specificity in confusion matrix against 89% and 86% obtained by investigators, but with good quality images forensic experts have better results. The rank correlation between investigators and software is around 80%. We conclude that the software can assist in reporting officers as it can do faster and more reliable comparisons with full‐frontal images, which can help the forensic expert in casework.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/1556-4029.14324

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In facial image comparison, studies did not take into account how different calibration characteristics such as features affect the results.

    “Although the process of calibration has been studied and analysed in speaker comparison works such as [28] or [31], [3] and in facial image comparison in [40] those studies in facial image comparison did not take into account how different calibration characteristics such as features affect the results.”

    From Calibration of score based likelihood ratio estimation in automated forensic facial image comparison (2022) · cites this paper as [40]

    The Results (ENFSI Test 2011) compare three calibration methods and report differences in their results. The Discussion also tests calibration datasets with different sizes and image characteristics and reports an effect on results. The paper therefore does examine how differing calibration choices affect outcomes, although these passages do not establish that it tested feature selection specifically. Page numbers are unavailable in our copy. (LLM, conf=0.86)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Facial image comparison has proven to have potential to help the forensic expert if the likelihood ratio estimation method is properly standardized and validated.

    “And facial image comparison has proven to have potential to help the forensic expert if the likelihood ratio estimation method is properly standardized and validated [40].”

    From Calibration of score based likelihood ratio estimation in automated forensic facial image comparison (2022) · cites this paper as [40]

    The Conclusion (page not available) says the software can help forensic experts, particularly with full-frontal images. The paper also describes validating its likelihood-ratio estimates against expert results and discusses guidelines that standardize forensic reasoning. It does not establish that proper standardization and validation of the estimation method are the condition under which facial image comparison generally helps experts; its demonstrated benefit is narrower. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Automated systems for facial image comparison, especially deep-learning systems, combined with score-based likelihood ratio estimation have great potential to help the forensic expert in the evaluative process.

    “As explored in [40] and [3], automated systems for facial image comparison (especially when based on deep learning) combined with score-based likelihood ratio estimation have a great potential to help the forensic expert in the evaluative process [16].”

    From Calibration of score based likelihood ratio estimation in automated forensic facial image comparison (2022) · cites this paper as [40]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 40

  1. Celentino JC, 2016, Face‐to‐face with facial recognition evidence: admissibility under the post‐Crawford confrontation clause, Mich L Rev, 114, 1317
  2. Meuwly D, 2018, Forensic face recognition as a means to determine strength of evidence: a survey, Forensic Sci Rev, 30, 21
  3. Ali T, 2012, Face recognition: methods, applications and technology, 9
  4. 10.1007/978-0-85729-932-1_26 · DOI
  5. McKenna L, 2015, ENFSI Guideline for Evaluative Reporting in Forensic Science: Strengthening the evaluation of forensic results across Europe (STEOFRAE), 30
  6. Lund SP, 2017, Likelihood ratio as weight of forensic evidence: a closer look, J Res Nat Inst Stand Technol, 122, 1
  7. 10.1073/pnas.91.13.6007 · DOI
  8. 10.1111/j.1556-4029.2006.00266.x · DOI
  9. Ali T, 2017, Biometric score calibration for forensic face recognition [PhD thesis], 11
  10. 10.1007/978-3-319-50673-9_14 · DOI · Paper page
  11. TurkMA PentlandAP.Face recognition using eigenfaces. In: Proceedings of the 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition; 1991 June 3‐6; Maui HI.Piscataway NJ:IEEE 1991;586–91.
  12. 10.5244/C.29.41 · DOI · Paper page
  13. MasiI WuY HassnerT NatarajanP.Deep face recognition: a survey. In: Proceedings of the 31st Conference on Graphics Patterns and Images (SIBGRAPI); 2018 Oct 29 – Nov 1; Foz Do Iguacu Brazil.Piscataway NJ:IEEE 2018;471–8. · DOI
  14. 10.1007/978-3-319-67639-5_2 · DOI
  15. 10.1073/pnas.1721355115 · DOI · Paper page
  16. 10.1016/j.dib.2016.11.008 · DOI · Paper page
  17. 10.1049/iet-bmt.2015.0059 · DOI
  18. 10.1007/s11704-016-6076-3 · DOI
  19. SchroffF KalenichenkoD PhilbinJ.FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the 28th IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2015); 2015 June 7–12; Boston MA.Piscataway NJ:IEEE 2015;815–23. · DOI · Paper page
  20. 10.1007/978-1-84628-288-1_3 · DOI
  21. 10.1080/24709360.2017.1396742 · DOI
  22. 10.18637/jss.v032.i05 · DOI
  23. 10.1016/0005-2795(75)90109-9 · DOI · Paper page
  24. Aitken CCC, 2004, Statistics and the evaluation of evidence for forensic scientists, 69, 10.1002/0470011238.ch3 · DOI
  25. 10.1177/1365712716674796 · DOI
  26. PerlinMW.Explaining the likelihood ratio in DNA mixture interpretation. In: Proceedings of Promega's Twenty First International Symposium on Human Identification; 2010 Oct 11‐14; San Antonio TX.Fitchburg WI:Promega Corporation 2010;1–32.
  27. Haraksim R, 2016, Proceedings of the 15th International Conference of the Biometrics Special Interest Group (BIOSIG 2016); 2016 Sep 21–23; Darmstadt, Germany, 225
  28. 10.1111/1556-4029.13339 · DOI
  29. 10.1093/lpr/mgv009 · DOI
  30. 10.1016/j.scijus.2017.03.002 · DOI
  31. 10.1016/j.scijus.2014.04.008 · DOI
  32. 10.1007/11550822_86 · DOI
  33. TaigmanY YangM RanzatoM WolfL.DeepFace: closing the gap to human‐level performance in face verification. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition; 2014 June 23–28; Columbus OH.Piscataway NJ:IEEE 2014;1701–8. · DOI · Paper page
  34. Sharif M, 2010, Illumination normalization preprocessing for face recognition, 44
  35. Kumar R, 2019, A review paper on face recognition techniques, Int J Eng Res Appl, 7, 223
  36. 10.1109/LSP.2016.2603342 · DOI · Paper page
  37. 10.1016/j.neucom.2016.09.072 · DOI
  38. 10.1007/978-3-319-10605-2_1 · DOI
  39. Ali T, 2012, Proceedings of the 33rd WIC Symposium on Information Theory in the Benelux and the 2nd Joint WIC/IEEE Symposium on Information Theory and Signal Processing in the Benelux; 2012 May 24‐25; Boekelo, the Netherlands, 126
  40. 10.1111/j.1467-9868.2008.00677.x · DOI

Cited by

Sorted by citations

Metadata

Authors
Macarulla Rodriguez, A., Geradts, Z., & Worring, M.
Affiliations
  • Netherlands Forensic Institute Laan van Ypenburg 6 2497 GB Den Haag The Netherlands
  • University of Amsterdam Science Park 904 1098 XH Amsterdam The Netherlands
ORCID
Venue
Journal of Forensic Sciences
Publisher
Wiley
Published
2020-05-12
Volume
65
Issue
4
Pages
1169-1183
ISSN
  • 0022-1198
  • 1556-4029
DOI
10.1111/1556-4029.14324
PMC
PMC7383913
Cited by
21
References
1
Language
en
Licence
creativecommons.org/licenses/by-nc/4.0/ (vor)
Funders
ASGARD (ID : 700381)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review