The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
“Believe it or not: Designing a human-ai partnership for mixed-initiative fact-checking,”
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For instance, manipulations of the inputs result in new AI's outcome computations for exploratory purposes (Liu et al., 2021;Gu et al., 2023;Nguyen et al., 2018;Zytek et al., 2022;Suresh et al., 2022).”
From Human-AI collaboration is not very collaborative yet: a taxonomy of interaction patterns in AI-assisted decision making from a systematic review (2025) · cites this paper as Nguyen et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
lmerTest Package: Tests in Linear Mixed Effects Models
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017) · Journal of Statistical Software · doi:10.18637/jss.v082.i13 · Cited by 24,346
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
What makes Web sites credible?: a report on a large quantitative study
Fogg, B. J., Marshall, J., Laraki, O., Osipovich, A., Varma, C., Fang, N., Paul, J., Rangnekar, A., Shon, J., Swani, P., et al. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365037 · Cited by 685
Fails, J. A., & Olsen, D. R. (2003) · Proceedings of the 8th international conference on Intelligent user interfaces · doi:10.1145/604045.604056 · Cited by 633
Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257
Leveraging the Crowd to Detect and Reduce the Spread of Fake News and Misinformation
Kim, J., Tabibian, B., Oh, A., Schölkopf, B., & Gomez-Rodriguez, M. (2018) · Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining · doi:10.1145/3159652.3159734 · Cited by 230
Fake News Detection in Social Networks via Crowd Signals
Tschiatschek, S., Singla, A., Gomez Rodriguez, M., Merchant, A., & Krause, A. (2018) · Companion of the The Web Conference 2018 on The Web Conference 2018 - WWW '18 · doi:10.1145/3184558.3188722 · Cited by 229
Tell me more?: the effects of mental model soundness on personalizing an intelligent agent
Kulesza, T., Stumpf, S., Burnett, M., & Kwan, I. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207678 · Cited by 225
Research works printed but not linked: 38
- 10.1145/2702123.2702509 · DOI · Paper page
- Douglas Bates , Martin Maechler , Ben Bolker , Steven Walker , and others. 2014. lme4: Linear mixed-effects models using Eigen and S4. R package version 1, 7 ( 2014 ), 1--23. Douglas Bates, Martin Maechler, Ben Bolker, Steven Walker, and others. 2014. lme4: Linear mixed-effects models using Eigen and S4. R package version 1, 7 (2014), 1--23.
- 10.1145/3122803 · DOI
- 10.1145/3159652.3159661 · DOI · Paper page
- Finale Doshi-Velez and Been Kim . 2017. Towards A Rigorous Science of Interpretable Machine Learning. arXiv ( 2017 ). https://arxiv.org/abs/1702.08608 Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. arXiv (2017). https://arxiv.org/abs/1702.08608 · arXiv
- 10.1145/3025453.3025739 · DOI · Paper page
- 10.1145/1772690.1772726 · DOI
- Rong-En Fan , Kai-Wei Chang , Cho-Jui Hsieh , Xiang-Rui Wang , and Chih-Jen Lin . 2008 . LIBLINEAR: A library for large linear classification . Journal of machine learning research 9 , Aug (2008), 1871 -- 1874 . Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. Journal of machine learning research 9, Aug (2008), 1871--1874.
- William Ferreira and Andreas Vlachos . 2016. Emergent: a novel data-set for stance classification . In North American Chapter of the Association for Computational Linguistics. ACL. William Ferreira and Andreas Vlachos. 2016. Emergent: a novel data-set for stance classification. In North American Chapter of the Association for Computational Linguistics. ACL.
- 10.1145/1357054.1357061 · DOI
- 10.1145/2851581.2856492 · DOI
- Aurora Harley . 2016. Trustworthiness in Web Design: 4 Credibility Factors. Utg. av Nielsen Norman group. url: https://www. nngroup. com/articles/trustworthy-design ( 2016 ). Aurora Harley. 2016. Trustworthiness in Web Design: 4 Credibility Factors. Utg. av Nielsen Norman group. url: https://www. nngroup. com/articles/trustworthy-design (2016).
- 10.1145/302979.303030 · DOI · Paper page
- Panagiotis G Ipeirotis. 2010. Demographics of mechanical turk. (2010). NYU Working Paper No. CEDER-10-01. https://ssrn.com/abstract=1585030. Panagiotis G Ipeirotis. 2010. Demographics of mechanical turk. (2010). NYU Working Paper No. CEDER-10-01. https://ssrn.com/abstract=1585030.
- Michael I Jordan and Tom M Mitchell . 2015. Machine learning: Trends, perspectives, and prospects. Science 349, 6245 ( 2015 ), 255--260. Michael I Jordan and Tom M Mitchell. 2015. Machine learning: Trends, perspectives, and prospects. Science 349, 6245 (2015), 255--260.
- Alireza Karduni , Ryan Wesslen , Sashank Santhanam , Isaac Cho , Svitlana Volkova , Dustin Arendt , Samira Shaikh , and Wenwen Dou . 2018 . Can You Verifi This? Studying Uncertainty and Decision-Making About Misinformation using Visual Analytics . In 12th International AAAI Conference on Web and Social Media (ICWSM 2018). Alireza Karduni, Ryan Wesslen, Sashank Santhanam, Isaac Cho, Svitlana Volkova, Dustin Arendt, Samira Shaikh, and Wenwen Dou. 2018. Can You Verifi This? Studying Uncertainty and Decision-Making About Misinformation using Visual Analytics. In 12th International AAAI Conference on Web and Social Media (ICWSM 2018).
- 10.1145/3173225.3173302 · DOI
- 10.1145/2858036.2858529 · DOI · Paper page
- Carol Collier Kuhlthau. 1987. Information Skills for an Information Society: A Review of Research. An ERIC Information Analysis Product. ERIC. Carol Collier Kuhlthau. 1987. Information Skills for an Information Society: A Review of Research. An ERIC Information Analysis Product. ERIC.
- 10.1145/2030365.2030367 · DOI
- 10.1145/2464464.2464485 · DOI
- Ndapandula Nakashole and Tom M Mitchell. 2014. Language-Aware Truth Assessment of Fact Candidates.. In ACL (1). 1009--1019. Ndapandula Nakashole and Tom M Mitchell. 2014. Language-Aware Truth Assessment of Fact Candidates.. In ACL (1). 1009--1019. · DOI
- An T. Nguyen , Aditya Kharosekar , Matthew Lease , and Byron C . Wallace . 2018 . An Interpretable Joint Graphical Model for Fact-Checking from Crowds. In AAAI. An T. Nguyen, Aditya Kharosekar, Matthew Lease, and Byron C. Wallace. 2018. An Interpretable Joint Graphical Model for Fact-Checking from Crowds. In AAAI.
- Fabian Pedregosa , Gaël Varoquaux , Alexandre Gramfort , Vincent Michel , Bertrand Thirion , Olivier Grisel , Mathieu Blondel , Peter Prettenhofer , Ron Weiss , Vincent Dubourg , and others. 2011 . Scikit-learn: Machine learning in Python . Journal of machine learning research 12 , Oct (2011), 2825 -- 2830 . Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, and others. 2011. Scikit-learn: Machine learning in Python. Journal of machine learning research 12, Oct (2011), 2825--2830.
- 10.1145/3041021.3055133 · DOI
- 10.1145/3180308.3180329 · DOI · Paper page
- 10.1145/2984511.2984544 · DOI
- Jon Roozenbeek and Sander van der Linden . 2018. The fake news game: actively inoculating against the risk of misinformation. Journal of Risk Research ( 2018 ), 1--11. Jon Roozenbeek and Sander van der Linden. 2018. The fake news game: actively inoculating against the risk of misinformation. Journal of Risk Research (2018), 1--11.
- 10.1145/1753846.1753873 · DOI · Paper page
- Dominik Sacha , Michael Sedlmair , Leishi Zhang , John Aldo Lee , Daniel Weiskopf , Stephen North , and Daniel Keim . 2016 . Human-centered machine learning through interactive visualization . European Symposium on Artificial Neural Networks (ESANN). Dominik Sacha, Michael Sedlmair, Leishi Zhang, John Aldo Lee, Daniel Weiskopf, Stephen North, and Daniel Keim. 2016. Human-centered machine learning through interactive visualization. European Symposium on Artificial Neural Networks (ESANN).
- Mehdi Samadi , Partha Talukdar , Manuela Veloso , and Manuel Blum . 2016 . ClaimEval: Integrated and Flexible Framework for Claim Evaluation Using Credibility of Sources . In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. 222--228 . Mehdi Samadi, Partha Talukdar, Manuela Veloso, and Manuel Blum. 2016. ClaimEval: Integrated and Flexible Framework for Claim Evaluation Using Credibility of Sources. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. 222--228.
- 10.1145/3078714.3078746 · DOI
- 10.1080/13658816.2011.557379 · DOI
- 10.1145/3172944.3172965 · DOI
- 10.1145/2984511.2984573 · DOI
- 10.1145/3209978.3210037 · DOI
- 10.18653/v1/P17-2102 · DOI · Paper page
- 10.18653/v1/P17-2067 · DOI · Paper page
Cited by
Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581001 · Cited by 233
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
Gomez, C., Cho, S. M., Ke, S., Huang, C.-M., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 209
Herrmann, T., & Pfeiffer, S. (2023) · AI & SOCIETY · doi:10.1007/s00146-022-01391-5 · Cited by 173
“• allowing individuals the option to respond to incorrect predictions of an AI system (Nguyen et al. 2018), • giving prompts that help counter the human tendency to place too much trust in automated systems (Okamura and Yamada 2020), • and integrating knowledge management with learning (Li and Herd 2017).”
BackgroundHuman-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Lai, V., Carton, S., Bhatnagar, R., Liao, Q. V., Zhang, Y., & Tan, C. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501999 · Cited by 145
Metadata
- Authors
- Nguyen, A. T., Kharosekar, A., Krishnan, S., Krishnan, S., Tate, E., Wallace, B. C., & Lease, M.
- Affiliations
- University of Texas at Austin, Austin, TX, USA
- Northeastern University, Boston, MA, USA
- Venue
- Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology
- Publisher
- ACM
- Published
- 2018-10-11
- Pages
- 189-199
- Cited by
- 95
- References
- 8
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: “Believe it or not: Designing a human-ai partnership for mixed-initiative fact-checking,”. https://app.cochat.ai/papers/10.1145/3242587.3242666
- The paper
- Nguyen, A. T., Kharosekar, A., Krishnan, S., Krishnan, S., Tate, E., Wallace, B. C., & Lease, M. (2018). “Believe it or not: Designing a human-ai partnership for mixed-initiative fact-checking,”. Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. https://doi.org/10.1145/3242587.3242666
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.