On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
Open accessThe DOI matches its Crossref record; 87 of the 87 references it cites were found in a registry or an index.
Abstract
Humans are the final decision makers in critical tasks that involve ethical and legal concerns, ranging from recidivism prediction, to medical diagnosis, to fighting against fake news. Although machine learning models can sometimes achieve impressive performance in these tasks, these tasks are not amenable to full automation. To realize the potential of machine learning for improving human decisions, it is important to understand how assistance from machine learning models affects human performance and human agency.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“this remedy is unlikely to be effective: substantial evidence demonstrates that humans tend to override algorithms in detrimental rather than beneficial ways”
From The flaws of policies requiring human oversight of government algorithms (2022) · cites this paper as Lai & Tan, 2019
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“They found that all these features increased people's trust, measured as acceptance of the AI's recommendation as the final decision, and also the decision accuracy.”
From Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making (2020) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“observing the results reported by correct versus incorrect model decisions, all these features increased participants willingness to accept the AI's prediction regardless of its correctness, which is evidence that they are ineffective in calibrating trust.”
From Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making (2020) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is in contrast to [17] where the humans performed substantially worse than the AI (by 37%).”
From Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making (2020) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“while research repeatedly demonstrates that providing high-performance indicators of an AI system, such as showing high accuracy scores, could enhance people's trust and acceptance of the system [17, 30, 32]”
From Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making (2020) · cites this paper as [17]
Experimental — accuracy not yet measured
45 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 3×
“We build on recent developments in interpretable machine learning that provide useful frameworks for generating explanations of machine predictions [34, 35, 45, 50, 64, 65].”
BackgroundMethodDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“9%) [29]; Kleinberg et al.”
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 2×
“nternet. For instance, Ott et al. [58] find that deceptive reviews are a growing problem on multiple platforms such as TripAdvisor and Yelp. Fake news has also received significant attention recently [43, 74] and might have influenced the outcome of the U.S. presidential election in 2016 [3]. Enhancing humans’ ability…”
BackgroundStatistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722 · cited 1×
“knowledge databases and representations to reason about complex decisions [5, 31, 33, 55, 67].”
BackgroundAn introduction to cybernetics
Ashby, W. R. (1956) · doi:10.5962/bhl.title.5851 · Cited by 7,578 · cited 1×
“Our work also resonates with the seminal work on mixed-initiative user interfaces [31] and intelligence augmentation [4].”
ResultAxten, N., Newell, A., & Simon, H. A. (1973) · Contemporary Sociology · doi:10.2307/2063712 · Cited by 6,834 · cited 1×
“knowledge databases and representations to reason about complex decisions [5, 31, 33, 55, 67].”
BackgroundSocial Media and Fake News in the 2016 Election
Allcott, H., & Gentzkow, M. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.211 · Cited by 6,767 · cited 1×
“Fake news has also received significant attention recently [43, 74] and might have influenced the outcome of the U.S. presidential election in 2016 [3].”
Background“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 3×
“nteract with machine learning models, it makes a significant difference whether predicted labels are shown. However, this observation also echoes with concerns about humans overly relying on machines [44]. To further understand human performance with predicted labels, we examine all experimental setups with predicted…”
MethodThe Effect of Word of Mouth on Sales: Online Book Reviews
Chevalier, J. A., & Mayzlin, D. (2006) · Journal of Marketing Research · doi:10.1509/jmkr.43.3.345 · Cited by 6,003 · cited 1×
“Since people are increasingly relying on online reviews to make purchase decisions [11, 72, 78, 81], machine learning methods have been used to detect deception in online reviews [22, 24, 32, 60,”
MethodHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 1×
“In the case of general automation, researchers have also studied issues of appropriate reliance and trust [8, 18, 44, 61, 76].”
BackgroundLazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160 · cited 2×
“nternet. For instance, Ott et al. [58] find that deceptive reviews are a growing problem on multiple platforms such as TripAdvisor and Yelp. Fake news has also received significant attention recently [43, 74] and might have influenced the outcome of the U.S. presidential election in 2016 [3]. Enhancing humans’ ability…”
BackgroundAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 1×
“However, humans may not trust the machine due to algorithm aversion [15].”
BackgroundWhen Corrections Fail: The Persistence of Political Misperceptions
Nyhan, B., & Reifler, J. (2010) · Political Behavior · doi:10.1007/s11109-010-9112-2 · Cited by 3,173 · cited 2×
“In addition, Nyhan and Reifler [57] demonstrate the “backfire” effect, which suggests that corrections of misperceptions may enhance people’s false beliefs, and Vosoughi et al. [74] show that fake news is more innovative and spreads faster than real news.”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“Human decision makers with assistance from machines further complicate the current discussions on the issue of fairness in algorithmic decision making [12, 28, 38].”
BackgroundTrusov, M., Bucklin, R. E., & Pauwels, K. (2009) · Journal of Marketing · doi:10.1509/jmkg.73.5.90 · Cited by 2,451
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 2×
“We build on recent developments in interpretable machine learning that provide useful frameworks for generating explanations of machine predictions [34, 35, 45, 50, 64, 65].”
BackgroundMethodHow to improve Bayesian reasoning without instruction: Frequency formats.
Gigerenzer, G., & Hoffrage, U. (1995) · Psychological Review · doi:10.1037//0033-295x.102.4.684 · Cited by 2,127 · cited 1×
“n is almost the same as not showing machine accuracy. Our frequency explanations are also knownasfrequentformatandhavebeenshowntobemoreeffective for conveying uncertainty than stating the probability [25, 26, 66]. 6 CONCLUDING DISCUSSION In this paper, we conduct the first empirical study to investigate whether machin…”
BackgroundAccuracy of Deception Judgments
Bond, C. F., & DePaulo, B. M. (2006) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr1003_2 · Cited by 1,953 · cited 1×
“formance (Figure 3a). As Figure 3a shows, human performance in control is no better than chance (51.1%). This finding is consistent with Ott et al. [60] and decades of research on deception detection [7]. Explanations alone slightly improve human performance over control, and the differences are statistically signific…”
ResultInfluentialExplaining collaborative filtering recommendations
Herlocker, J. L., Konstan, J. A., & Riedl, J. (2000) · Proceedings of the 2000 ACM conference on Computer supported cooperative work · doi:10.1145/358916.358995 · Cited by 1,731 · cited 1×
“Therefore, improving their transparency and interpretability has attracted broad interest [34, 35, 45, 50, 64, 65], dating back to early work on recommendation systems [13, 30].”
BackgroundIntelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · cited 1×
“earning models remain as black boxes despite wide adoption. Blindly following machine predictions may lead to dire repercussions, especially in scenarios such as medical diagnosis and justice systems [9, 36, 73]. Therefore, improving their transparency and interpretability has attracted broad interest [34, 35, 45, 50,…”
BackgroundDecision support systems : an organizational perspective
Keen, P., & Morton, M. S. S. (1978) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 1,673
Jindal, N., & Liu, B. (2008) · Proceedings of the international conference on Web search and web data mining - WSDM '08 · doi:10.1145/1341531.1341560 · Cited by 1,542 · cited 1×
“Since people are increasingly relying on online reviews to make purchase decisions [11, 72, 78, 81], machine learning methods have been used to detect deception in online reviews [22, 24, 32, 60, 77, 79].”
BackgroundPrinciples of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · cited 2×
“Our work also resonates with the seminal work on mixed-initiative user interfaces [31] and intelligence augmentation [4].”
BackgroundResultSupport vector machines for spam categorization
Drucker, H., Wu, D., & Vapnik, V. N. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.788645 · Cited by 1,471 · cited 1×
“Deceptive reviews can also be seen as an instance of spamming and online fraud [2, 16, 27, 56].”
BackgroundSlovic, P., & Peters, E. (2006) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2006.00461.x · Cited by 1,443 · cited 2×
“This observation echoes prior work on numeracy that suggests that average humans and even doctors struggle with interpreting and acting on numbers [6, 62, 63, 69].”
BackgroundPast, present, and future of decision support technology
Shim, J. P., Warkentin, M., Courtney, J. F., Power, D. J., Sharda, R., & Carlsson, C. (2002) · Decision Support Systems · doi:10.1016/s0167-9236(01)00139-7 · Cited by 1,425 · cited 1×
“knowledge databases and representations to reason about complex decisions [5, 31, 33, 55, 67].”
BackgroundThe role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1×
“erest [34, 35, 45, 50, 64, 65], dating back to early work on recommendation systems [13, 30]. In the case of general automation, researchers have also studied issues of appropriate reliance and trust [8, 18, 44, 61, 76]. There are two major approaches to providing explanations of machine learning models: example-based…”
Peters, E., Västfjäll, D., Slovic, P., Mertz, C. K., Mazzocco, K., & Dickert, S. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01720.x · Cited by 1,180
Ye, Q., Law, R., Gu, B., & Chen, W. (2011) · Computers in Human Behavior · doi:10.1016/j.chb.2010.04.014 · Cited by 1,163 · cited 1×
“Since people are increasingly relying on online reviews to make purchase decisions [11, 72, 78, 81], machine learning methods have been used to detect deception in online reviews [22, 24, 32, 60, 77, 79].”
BackgroundAlgorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Combating Web Spam with TrustRank
Gyöngyi, Z., Garcia-Molina, H., & Pedersen, J. (2004) · Proceedings 2004 VLDB Conference · doi:10.1016/b978-012088469-8.50052-8 · Cited by 1,085 · cited 1×
“Deceptive reviews can also be seen as an instance of spamming and online fraud [2, 16, 27, 56].”
BackgroundHuman Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Finding Deceptive Opinion Spam by Any Stretch of the Imagination
Ott, M., Choi, Y., Cardie, C., & Hancock, J. T. (2011) · arXiv · doi:10.48550/arxiv.1107.4557 · Cited by 1,008 · cited 4×
“Since people are increasingly relying on online reviews to make purchase decisions [11, 72, 78, 81], machine learning methods have been used to detect deception in online reviews [22, 24, 32, 60, 77, 79].”
BackgroundInfluentialInherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 1×
“Human decision makers with assistance from machines further complicate the current discussions on the issue of fairness in algorithmic decision making [12, 28, 38].”
BackgroundInherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941 · cited 1×
“Human decision makers with assistance from machines further complicate the current discussions on the issue of fairness in algorithmic decision making [12, 28, 38].”
BackgroundFacial signs of emotional experience.
Ekman, P., Freisen, W. V., & Ancoli, S. (1980) · Journal of Personality and Social Psychology · doi:10.1037/h0077722 · Cited by 886 · cited 1×
“To detect deception, researchers have examined the role of behavioral, emotional, and linguistic cues [17, 19, 39, 42, 54, 75].”
BackgroundRationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · cited 2×
“We build on recent developments in interpretable machine learning that provide useful frameworks for generating explanations of machine predictions [34, 35, 45, 50, 64, 65].”
BackgroundMethodZhang, Z., Ye, Q., Law, R., & Li, Y. (2010) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2010.02.002 · Cited by 760 · cited 1×
“Since people are increasingly relying on online reviews to make purchase decisions [11, 72, 78, 81], machine learning methods have been used to detect deception in online reviews [22, 24, 32, 60, 77, 79].”
BackgroundThe structure and function of explanations
Lombrozo, T. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.08.004 · Cited by 639 · cited 1×
“itiative user interfaces [31] and intelligence augmentation [4]. In addition, our work is connected to cognitive studies on understanding effective explanations beyond the context of machine learning [48, 49]. Deception and misinformation. Deception is a widely studied phenomenon in many disciplines [75]. In psycholog…”
BackgroundDetecting Lies and Deceit: The Psychology of Lying and the Implications for Professional Practice
Vrij, A. (2000) · Medical Entomology and Zoology · Cited by 625
Detecting spam web pages through content analysis
Ntoulas, A., Najork, M., Manasse, M., & Fetterly, D. (2006) · Proceedings of the 15th international conference on World Wide Web · doi:10.1145/1135777.1135794 · Cited by 608 · cited 1×
“Deceptive reviews can also be seen as an instance of spamming and online fraud [2, 16, 27, 56].”
BackgroundKleinberg, J., Ludwig, J., Mullainathan, S., & Obermeyer, Z. (2015) · American Economic Review · doi:10.1257/aer.p20151023 · Cited by 584 · cited 1×
“Moreover, since many policy decisions can be formulated as prediction problems [37], understanding human predictions with assistance from machine learning models”
Background(2016) · neural information processing systems · Cited by 556
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017) · doi:10.3386/w23180 · Cited by 545
Detecting lies and deceit: the psychology of lying and the implications for professional practice
Clifford, B. R. (2001) · Applied Cognitive Psychology · doi:10.1002/acp.743 · Cited by 537
Syntactic Stylometry for Deception Detection
Feng, S., Banerjee, R., & Choi, Y. (2012) · Cited by 499
Numeracy, Ratio Bias, and Denominator Neglect in Judgments of Risk and Probability
Reyna, V. F., & Brainerd, C. J. (2008) · Learning and Individual Differences · doi:10.1016/j.lindif.2007.03.011 · Cited by 481 · cited 2×
“This observation echoes prior work on numeracy that suggests that average humans and even doctors struggle with interpreting and acting on numbers [6, 62, 63, 69].”
BackgroundIs seeing believing?: how recommender system interfaces affect users' opinions
Cosley, D., Lam, S. K., Albert, I., Konstan, J. A., & Riedl, J. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642713 · Cited by 467 · cited 1×
“Therefore, improving their transparency and interpretability has attracted broad interest [34, 35, 45, 50, 64, 65], dating back to early work on recommendation systems [13, 30].”
BackgroundThe Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · cited 1×
“erest [34, 35, 45, 50, 64, 65], dating back to early work on recommendation systems [13, 30]. In the case of general automation, researchers have also studied issues of appropriate reliance and trust [8, 18, 44, 61, 76]. There are two major approaches to providing explanations of machine learning models: example-based…”
Opinion Fraud Detection in Online Reviews by Network Effects
Akoglu, L., Chandy, R., & Faloutsos, C. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14380 · Cited by 391 · cited 2×
“create the first sizable dataset in deception detection by asking Amazon Mechanical Turkers to write deceptive reviews. Deceptive reviews can also be seen as an instance of spamming and online fraud [2, 16, 27, 56]. More recently, the issue of misinformation and fake news has drawn much attention from both the public…”
BackgroundSimplicity and probability in causal explanation
LOMBROZO, T. (2007) · Cognitive Psychology · doi:10.1016/j.cogpsych.2006.09.006 · Cited by 384 · cited 1×
“itiative user interfaces [31] and intelligence augmentation [4]. In addition, our work is connected to cognitive studies on understanding effective explanations beyond the context of machine learning [48, 49]. Deception and misinformation. Deception is a widely studied phenomenon in many disciplines [75]. In psycholog…”
Detecting True Lies: Police Officers' Ability to Detect Suspects' Lies.
Mann, S., Vrij, A., & Bull, R. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.1.137 · Cited by 383 · cited 1×
“Our results contribute to mixed observations regarding gender differences in deception detection [14, 46, 52, 53].”
ResultTeaching Bayesian reasoning in less than two hours.
Sedlmeier, P., & Gigerenzer, G. (2001) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.130.3.380 · Cited by 366 · cited 1×
“Our frequency explanations are also known as frequent format and have been shown to bemore effective for conveying uncertainty than stating the probability [25, 26, 66].”
Background(2013) · Cited by 353
The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
Kim, B., Rudin, C. D., & Shah, J. (2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 340
AN EXPLORATION OF DECEPTION AS A COMMUNICATION CONSTRUCT
KNAPP, M. L., HART, R. P., & DENNIS, H. S. (1974) · Human Communication Research · doi:10.1111/j.1468-2958.1974.tb00250.x · Cited by 327 · cited 1×
“ed to foster in another person a belief or understanding which the deceiver considers false [41]. To detect deception, researchers have examined the role of behavioral, emotional, and linguistic cues [17, 19, 39, 42, 54, 75]. Since people are increasingly relying on online reviews to make purchase decisions [11, 72, 7…”
MethodEstimating the prevalence of deception in online review communities
Ott, M., Cardie, C., & Hancock, J. (2012) · Proceedings of the 21st international conference on World Wide Web · doi:10.1145/2187836.2187864 · Cited by 282 · cited 1×
“[58] find that deceptive reviews are a growing problem on multiple platforms such as TripAdvisor and Yelp.”
BackgroundThe Psychology of Good Judgment: Frequency Formats and Simple Algorithms
Gigerenzer, G. (1996) · Medical Decision Making · doi:10.1177/0272989x9601600312 · Cited by 219 · cited 1×
“n is almost the same as not showing machine accuracy. Our frequency explanations are also knownasfrequentformatandhavebeenshowntobemoreeffective for conveying uncertainty than stating the probability [25, 26, 66]. 6 CONCLUDING DISCUSSION In this paper, we conduct the first empirical study to investigate whether machin…”
BackgroundComparison of Deceptive and Truthful Travel Reviews
Yoo, K.-H., & Gretzel, U. (2009) · Information and Communication Technologies in Tourism 2009 · doi:10.1007/978-3-211-93971-0_4 · Cited by 219 · cited 1×
“Since people are increasingly relying on online reviews to make purchase decisions [11, 72, 78, 81], machine learning methods have been used to detect deception in online reviews [22, 24, 32, 60, 77, 79].”
BackgroundMachine Teaching: An Inverse Problem to Machine Learning and an Approach Toward Optimal Education
Zhu, X. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9761 · Cited by 216 · cited 2×
“This direction also connects to the area of machine teaching [51, 68, 82].”
BackgroundTELLING IT LIKE IT ISN'T: A REVIEW OF THEORY AND RESEARCH ON DECEPTIVE COMMUNICATIONS
L.KNAPP, M., & COMADEN, M. E. (1979) · Human Communication Research · doi:10.1111/j.1468-2958.1979.tb00640.x · Cited by 213 · cited 1×
“ed to foster in another person a belief or understanding which the deceiver considers false [41]. To detect deception, researchers have examined the role of behavioral, emotional, and linguistic cues [17, 19, 39, 42, 54, 75]. Since people are increasingly relying on online reviews to make purchase decisions [11, 72, 7…”
MethodBerwick, D. M., Fineberg, H. V., & Weinstein, M. C. (1981) · The American Journal of Medicine · doi:10.1016/0002-9343(81)90325-9 · Cited by 180 · cited 2×
“hout, even if the accuracy statement suggests poor machine performance. These observations echo with prior work on numeracy and suggest that it is difficult for humans to interpret and act on numbers [6, 62, 63, 69]. We also find that frequency explanations (e.g., 5 out of 10 for explaining 50%) can help humans calibr…”
BackgroundCombating Web Spam with TrustRank
Gyöngyi, Z., GARCIAMOLINA, H., & PEDERSEN, J. (2004) · Elsevier eBooks · doi:10.1016/b978-012088469-8/50052-8 · Cited by 160
Distortion as a validation criterion in the identification of suspicious reviews
Wu, G., Greene, D., Smyth, B., & Cunningham, P. (2010) · Proceedings of the First Workshop on Social Media Analytics · doi:10.1145/1964858.1964860 · Cited by 156 · cited 1×
“Since people are increasingly relying on online reviews to make purchase decisions [11, 72, 78, 81], machine learning methods have been used to detect deception in online reviews [22, 24, 32, 60, 77, 79].”
BackgroundZhang, A. X., Robbins, M., Bice, E., Hawke, S., Karger, D., Mina, A. X., Ranganathan, A., Metz, S. E., Appling, S., Sehat, C. M., et al. (2018) · Companion of the The Web Conference 2018 on The Web Conference 2018 - WWW '18 · doi:10.1145/3184558.3188731 · Cited by 120 · cited 1×
“[80], which explores varying types of credibility annotations specifically designed for news articles.”
BackgroundEngineering safety in machine learning
Varshney, K. R. (2016) · 2016 Information Theory and Applications Workshop (ITA) · doi:10.1109/ita.2016.7888195 · Cited by 118 · cited 1×
“Blindly following machine predictions may lead to dire repercussions, especially in scenarios such as medical diagnosis and justice systems [9, 36, 73].”
BackgroundThe Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
Kim, B., Rudin, C. D., & Shah, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.01161 · Cited by 117 · cited 2×
“ead to dire repercussions, especially in scenarios such as medical diagnosis and justice systems [9, 36, 73]. Therefore, improving their transparency and interpretability has attracted broad interest [34, 35, 45, 50, 64, 65], dating back to early work on recommendation systems [13, 30]. In the case of general automati…”
BackgroundDetecting lies and deceit: the psychology of lying and the implications for professional practice
Clifford, B. R. (2001) · Applied Cognitive Psychology · doi:10.1002/acp.743.abs · Cited by 107
Elaad, E. (2003) · Applied Cognitive Psychology · doi:10.1002/acp.871 · Cited by 105 · cited 1×
“Our results are not exactly aligned with the previous finding that humans tend to overestimate their capacity of detecting lying [20].”
ResultThe Supreme Court of the United States
Hughes, C. E. (1928) · Columbia University Press eBooks · doi:10.7312/hugh94232 · Cited by 104
Detecting Deceptive Opinions with Profile Compatibility
Feng, V. W., & Hirst, G. (2013) · Cited by 98
Peters, E. M., Västfjäll, D., Slovic, P., Mazzocco, K., & Dickert, S. (2005) · PsycEXTRA Dataset · doi:10.1037/e640112011-034 · Cited by 78
Near-Optimally Teaching the Crowd to Classify
Singla, A. K., Bogunovic, I., Bartók, G., Karbasi, A., & Krause, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1402.2092 · Cited by 67 · cited 2×
“One possible direction is to develop automatic tutorials to teach the intuitions behind important features, which is related to machine teaching [51, 68, 82].”
BackgroundDeception detection using a multimodal approach
Abouelenien, M., Pérez‐Rosas, V., Mihalcea, R. F., & Burzo, M. G. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2663229 · Cited by 67 · cited 1×
“Second, deception detection is a challenging task for humans and has been extensively studied [1, 2, 22, 24, 60].”
BackgroundCHANGES IN LANGUAGE BEHAVIOR AS A FUNCTION OF VERACITY
DULANEY, E. F. (1982) · Human Communication Research · doi:10.1111/j.1468-2958.1982.tb00684.x · Cited by 60 · cited 1×
“To detect deception, researchers have examined the role of behavioral, emotional, and linguistic cues [17, 19, 39, 42, 54, 75].”
BackgroundDo explanations make VQA models more predictable to a human?
Chandrasekaran, A., Prabhu, V., Yadav, D., Chattopadhyay, P., & Parikh, D. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1128 · Cited by 55 · cited 1×
“Concurrently, several recent studies have also examined how explanations relate to human performance [10, 23].”
BackgroundTeaching Categories to Human Learners with Visual Explanations
Aodha, O. M., Su, S., Chen, Y., Perona, P., & Yue, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00402 · Cited by 52
Rowbotham, M. (2021) · Productivity Press eBooks · doi:10.4324/9781003165682-1 · Cited by 47
What Women Know that Men don't: Sex Differences in Determining the Truth Behind Deceptive Messages
McCornack, S. A., & Parks, M. R. (1990) · Journal of Social and Personal Relationships · doi:10.1177/0265407590071006 · Cited by 44 · cited 1×
“Our results contribute to mixed observations regarding gender differences in deception detection [14, 46, 52, 53].”
ResultImproving Human-Machine Cooperative Visual Search With Soft Highlighting
Kneusel, R. T., & Mozer, M. C. (2017) · ACM Transactions on Applied Perception · doi:10.1145/3129669 · Cited by 31 · cited 1×
“Soft-highlighting (heatmap) has been shown to improve visual search on targeted areas for humans [40].”
BackgroundWhat can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play
Feng, S., & Boyd-Graber, J. L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.09648 · Cited by 23 · cited 1×
“Concurrently, several recent studies have also examined how explanations relate to human performance [10, 23].”
Background(1981) · The American Journal of Medicine · doi:10.1016/0002-9343(81)90327-2 · Cited by 11
Sex Differences in Deception Detection
Li, L. (2011) · Cited by 5
Daubert v. Merrell Dow Pharmaceuticals (1993)
Heilbronner, R. L. (2017) · Springer eBooks · doi:10.1007/978-3-319-56782-2_962-2 · Cited by 3
Research works printed but not linked: 5
- [21] Diane Farsetta and Daniel Price. 2006. Fake TV news: Widespread and undisclosed. Center for Media and Democracy 6 (2006). -
- [41] Robert M Krauss, Valerie Geller, and Christopher Olson. 1976. Modalities and cues in the detection of deception. In Meeting of the American Psychological Association, Washington, DC . -
- [54] Albert Mehrabian. 1971. Silent messages . Vol. 8. Wadsworth Belmont, CA. -
- [65] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: HighPrecision Model-Agnostic Explanations. In Proceedings of AAAI . -
- [76] Christopher D Wickens, Justin G Hollands, Simon Banbury, and Raja Parasuraman. 2015. Engineering psychology & human performance . Psychology Press. -
Cited by
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
“Consistent with prior research [7, 9, 26, 41], our results demonstrate that people aided by simple explainable AI interfaces performed better overall than people who completed the task without any AI support.”
BackgroundResultInfluentialDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
“ge-scale study provides informative insights that will inform future work on these domains. Finally, we only explored a small fraction of possible ways to present explanations: highlighting key words [18, 36, 42, 62] and natural language arguments. It may be that alternative approaches provide more benefit to team per…”
BackgroundInfluentialZhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
“ge-scale study provides informative insights that will inform future work on these domains. Finally, we only explored a small fraction of possible ways to present explanations: highlighting key words [18, 36, 42, 62] and natural language arguments. It may be that alternative approaches provide more benefit to team per…”
BackgroundInfluentialWill Humans-in-the-Loop Become Borgs? Merits and Pitfalls of Working with AI
Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16553 · Cited by 469
“In a deception detection experiment, where genuine and deceptive hotel reviews have to be classified, Lai and Tan (2019) find that adding explanatory information to AI suggestions significantly increases performance.”
Background
Metadata
- Authors
- Lai, V., & Tan, C.
- Affiliations
- University of Colorado Boulder (US)
- ORCID
- Tan, Chenhao
- Venue
- Proceedings of the Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2019-01-29
- Pages
- 29-38
- OpenAlex
- W2901895173
- Semantic Scholar
- 127c1cb96b73399f429de553d315561504cc7cd4
- Cited by
- 377
- References
- 85
- Citations by year
- 2026: 55 · 2025: 53 · 2024: 73 · 2023: 65 · 2022: 54 · 2021: 41 · 2020: 26 · 2019: 9
- FWCI
- 12.43
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- human-machine decision-making, explainable machine learning, human prediction, machine learning prediction, human agency, human performance, recidivism prediction, diagnosis, fake news detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.