CoChat

Discovering and Validating AI Errors With Crowdsourced Failure Reports

Open access
Cabrera, Ángel Alexander, Druck, Abraham J., Hong, Jason I., Perer, Adam · 2021 · Proceedings of the ACM on Human-Computer Interaction
Cited by 60

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Abstract

AI systems can fail to learn important behaviors, leading to real-world issues like safety concerns and biases. Discovering these systematic failures often requires significant developer attention, from hypothesizing potential edge cases to collecting evidence and validating patterns. To scale and streamline this process, we introduce crowdsourced failure reports, end-user descriptions of how or why a model failed, and show how developers can use them to detect AI errors. We also design and implement Deblinder, a visual analytics system for synthesizing failure reports that developers can use to discover and validate systematic failures. In semi-structured interviews and think-aloud studies with 10 AI practitioners, we explore the affordances of the Deblinder system and the applicability of failure reports in real-world settings. Lastly, we show how collecting additional data from the groups identified by developers can improve model performance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3479569

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 11 references

Research works printed but not linked: 57

  1. 10.1109/TVCG.2019.2934262 · DOI · Paper page
  2. 10.1109/ICSE-SEIP.2019.00042 · DOI · Paper page
  3. 10.1145/2531602.2531653 · DOI · Paper page
  4. Josh M Attenberg, Pagagiotis G Ipeirotis, and Foster Provost. 2011. Beat the machine: Challenging workers to find the unknown unknowns. In Workshops at the Twenty-Fifth AAAI Conference on Artificial Intelligence.
  5. Gagan Bansal and Daniel S. Weld. 2018. A coverage-based utility model for identifying unknown unknowns. 32nd AAAI Conference on Artificial Intelligence, AAAI 2018 (2018), 1463--1470.
  6. 10.1016/b978--155860915-0/50016-0 · doi:10.1016/b978--155860915-0/50016-0
  7. 10.1145/1453101.1453146 · DOI · Paper page
  8. Nicolas Bettenburg, Rahul Premraj, Thomas Zimmermann, and Sunghun Kim. [n.d.]. Duplicate bug reports considered harmful? really?. In 2008 IEEE International Conference on Software Maintenance. IEEE, 337--345.
  9. Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77--91.
  10. 10.1109/VAST47406.2019.8986948 · DOI · Paper page
  11. Ángel Alexander Cabrera Fred Hohman Jason Lin and Duen Horng Chau. 2018. Interactive Classification for Deep Learning Interpretation. (2018) 1--5. http://arxiv.org/abs/1806.05660 · arXiv
  12. 10.1145/2858036.2858411 · DOI
  13. 10.1145/1978942.1978967 · DOI · Paper page
  14. 10.1145/3172944.3172950 · DOI · Paper page
  15. Vincent S. Chen, Sen Wu, Zhenzhen Weng, Alexander Ratner, and Christopher Ré. 2019. Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices. NeurIPS (2019). http://arxiv.org/abs/1909.06349 · arXiv
  16. 10.1145/2675133.2675214 · DOI · Paper page
  17. 10.1109/TVCG.2017.2746018 · DOI
  18. 10.1145/2207676.2207711 · DOI · Paper page
  19. 10.1145/3134680 · DOI
  20. 10.1109/TVCG.2012.324 · DOI
  21. Nitesh Goyal. 2015. Designing for Collaborative Sensemaking: Using Expert & Non-Expert Crowd. (2015). http://arxiv.org/abs/1511.06053 · arXiv
  22. 10.14778/3421424.3421429 · DOI
  23. 10.1109/TR.2018.2873427 · DOI
  24. 10.1109/ACCESS.2020.2985222 · DOI
  25. 10.1145/2939502.2939503 · DOI · Paper page
  26. 10.1145/2531602.2531644 · DOI
  27. 10.1145/2470654.2481415 · DOI · Paper page
  28. 10.1109/PACIFICVIS.2015.7156366 · DOI · Paper page
  29. 10.1007/s11263-020-01316-z · DOI · Paper page
  30. 10.1609/aaai.v31i1.10821 · DOI · Paper page
  31. 10.1145/3366423.3380306 · DOI · Paper page
  32. 10.1109/ICCV.2015.425 · DOI · Paper page
  33. 10.1145/3415168 · DOI
  34. 10.1016/j.patcog.2011.06.019 · DOI · Paper page
  35. 10.3115/v1/D14-1162 · DOI · Paper page
  36. 10.1037/0033--295X.106.4.643 · doi:10.1037/0033--295x.106.4.643
  37. 10.1007/s13398-014-0173--7.2 · doi:10.1007/s13398-014-0173--7.2
  38. 10.1007/s11263-016-0965--7 · doi:10.1007/s11263-016-0965--7
  39. Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings (2016), 1--16.
  40. 10.1038/s41586-019--1138-y · doi:10.1038/s41586-019--1138-y
  41. 10.1609/aaai.v33i01.33016137 · DOI
  42. 10.1109/TSE.2013.2297712 · DOI · Paper page
  43. Alexander Ratner, Christopher De Sa, Sen Wu, Daniel Selsam, and Christopher Ré. 2016. Data Programming: Creating Large Training Sets, Quickly. In Proceedings of the 30th International Conference on Neural Information Processing Systems (Barcelona, Spain) (NIPS'16). Curran Associates Inc., Red Hook, NY, USA, 3574--3582.
  44. 10.1109/TVCG.2016.2598828 · DOI · Paper page
  45. 10.1002/9780470689646.ch1 · DOI · Paper page
  46. 10.1109/MSR.2010.5463280 · doi:10.1109/msr.2010.5463280
  47. 10.1145/1357054.1357247 · DOI · Paper page
  48. Tom Simonite. 2018. When it comes to gorillas, google photos remains blind. Wired, January, Vol. 13 (2018).
  49. 10.1109/ASE.2011.6100061 · DOI · Paper page
  50. 10.1145/1806799.1806811 · DOI · Paper page
  51. 10.2139/ssrn.3063289 · DOI · Paper page
  52. Daisuke Wakabayashi. 2018. Self-driving Uber car kills pedestrian in Arizona, where robots roam. The New York Times, Vol. 3 (2018), 19.
  53. Karl E Weick. 1995. Sensemaking in Organizations (Foundations for Organizational Science). Star (1995).
  54. 10.18653/v1/P19-1073 · DOI · Paper page
  55. 10.1109/ICSME.2014.66 · doi:10.1109/icsme.2014.66
  56. 10.1007/s11432-014--5241--2 · doi:10.1007/s11432-014--5241--2
  57. 10.1109/TSE.2010.63 · DOI · Paper page

Cited by

See all 60

Metadata

Authors
Cabrera, Á. A., Druck, A. J., Hong, J. I., & Perer, A.
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2021-10-13
Volume
5
Issue
CSCW2
Pages
1-22
ISSN
2573-0142 (ISSN-L)
OpenAlex
W3204393347
Cited by
60
References
11
FWCI
16.50
Type
journal article
Language
en
Open access
bronze
Funders
National Science Foundation (IIS-2040942)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Discovering and Validating AI Errors With Crowdsourced Failure Reports. https://app.cochat.ai/papers/10.1145/3479569
The paper
Cabrera, Á. A., Druck, A. J., Hong, J. I., & Perer, A. (2021). Discovering and Validating AI Errors With Crowdsourced Failure Reports. Proceedings of the ACM on Human-Computer Interaction. https://doi.org/10.1145/3479569

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.