A Review on Human–Machine Trust Evaluation: Human-Centric and Machine-Centric Perspectives
Open accessThe DOI matches its Crossref record; 125 of the 125 references it cites were found in a registry or an index.
Abstract
As complex autonomous systems become increasingly ubiquitous, their deployment and integration into our daily lives will become a significant endeavor. Human–machine trust relationship is now acknowledged as one of the primary aspects that characterize a successful integration. In the context of human–machine interaction (HMI), proper use of machines and autonomous systems depends both on the human and machine counterparts. On one hand, it depends on how well the human relies on the machine regarding the situation or task at hand based on willingness and experience. On the other hand, it depends on how well the machine carries out the task and how well it conveys important information on how the job is done. Furthermore, proper calibration of trust for effective HMI requires the factors affecting trust to be properly accounted for and their relative importance to be rightly quantified. In this article, the functional understanding of human–machine trust is viewed from two perspectives—human-centric and machine- centric. The human aspect of the discussion outlines factors, scales, and approaches, which are available to measure and calibrate human trust. The discussion on the machine aspect spans trustworthy artificial intelligence, built-in machine assurances, and ethical frameworks of trustworthy machines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“At present, the academic community is focused on the human-centered development of interpretable technology [100,101,102].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [102]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321 · cited 1×
“with inherent interpretability are classified under transparent models, and those with aided interpretability are known to exhibit post-hoc explainability [83].”
BackgroundBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 10×
“Concepts of calibration, resolution, and specificity are provided in [5] to describe the mismatch between user trust level and automation capabilities.”
BackgroundInfluentialThe global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“These frameworks are also relevant to trust, accentuated by references to trustworthy AI design and development principles for entailing customers’ trust [74].”
Supervised Machine Learning: A Review of Classification Techniques
Kotsiantis, S. B. (2007) · Informatica · Cited by 6,007
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 2×
“Overtrust and lack of trust on the users’ behalf lead to disuse and misuse of autonomous systems, respectively [1].”
BackgroundRempel, J. K., Holmes, J. G., & Zanna, M. P. (1985) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.49.1.95 · Cited by 3,146 · cited 1×
“Researchers modeled human trust based on social interpersonal interactions [27], [28] to represent the psychological state or attitude toward delegation of tasks under implied risk.”
BackgroundTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · cited 1×
“This definition of trust is by far the most widely used in subsequent studies of trust in automation [25].”
BackgroundRobust physical-world attacks on deep learning visual classification
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00175 · Cited by 2,173 · cited 2×
“driving deep neural networks [73], demonstrating how vulnerable these systems can be.”
BackgroundA Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction
Hancock, P. A., Billings, D. R., Schaefer, K. E., Chen, J. Y. C., de Visser, E. J., & Parasuraman, R. (2011) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720811417254 · Cited by 2,051
Foundations for an Empirically Determined Scale of Trust in Automated Systems
Jian, J.-Y., Bisantz, A. M., & Drury, C. G. (2000) · International Journal of Cognitive Ergonomics · doi:10.1207/s15327566ijce0401_04 · Cited by 1,799
Review on solving the inverse problem in EEG source analysis
Grech, R., Cassar, T., Muscat, J., Camilleri, K. P., Fabri, S. G., Zervakis, M., Xanthopoulos, P., Sakkalis, V., & Vanrumste, B. (2008) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-5-25 · Cited by 1,193
Explainable artificial intelligence: A survey
Dosilovic, F. K., Brcic, M., & Hlupic, N. (2018) · 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2018.8400040 · Cited by 1,147 · cited 1×
“and the system’s interpretability quotient [89].”
Trust, self-confidence, and operators' adaptation to automation
Lee, J. D., & Moray, N. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1007 · Cited by 932 · cited 1×
“automation’s capabilities” [112], [113].”
Rivest, R. L. (1987) · Machine Learning · doi:10.1007/bf00058680 · Cited by 886 · cited 1×
“Acomputationalexplanation approachisusedin[88]wherefeaturecoefficientsareevaluated tounderstandtheireffectontheexplainabilityofinterpretable modelsusingdecisionlistsordecisiontrees.”
MUIR, B. M. (1994) · Ergonomics · doi:10.1080/00140139408964957 · Cited by 873 · cited 6×
“trust scale in [4] and the trust evolution and reliance scale in [5].”
BackgroundMethodInfluentialAI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation
Gehr, T., Mirman, M., Drachsler-Cohen, D., Tsankov, P., Chaudhuri, S., & Vechev, M. (2018) · 2018 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2018.00058 · Cited by 799 · cited 2×
“, AI2, was developed in [105] to certify large neural networks’ robustness.”
BackgroundShneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768
Trust between humans and machines, and the design of decision aids
Muir, B. M. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80013-5 · Cited by 762 · cited 1×
“Calibration is defined as “the correspondence between a person’s trust in automation and the automation’s capabilities” [112], [113].”
BackgroundTrust and Distrust Definitions: One Bite at a Time
Harrison McKnight, D., & Chervany, N. L. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-45547-7_3 · Cited by 747
A Survey on Trust Management for Mobile Ad Hoc Networks
Cho, J.-H., Swami, A., & Chen, I.-R. (2011) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2011.092110.00088 · Cited by 664
Inferring psychological significance from physiological signals.
Cacioppo, J. T., & Tassinary, L. G. (1990) · American Psychologist · doi:10.1037//0003-066x.45.1.16 · Cited by 635
Humans and Automation: System Design and Research Issues
Sheridan, T. B. (2002) · Medical Entomology and Zoology · Cited by 558
Merritt, S. M., & Ilgen, D. R. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288574 · Cited by 527
Principles of trust for MAS: cognitive anatomy, social importance, and quantification
Castelfranchi, C., & Falcone, R. (2002) · Proceedings International Conference on Multi Agent Systems (Cat. No.98EX160) · doi:10.1109/icmas.1998.699034 · Cited by 491 · cited 1×
“Researchers modeled human trust based on social interpersonal interactions [27], [28] to represent the psychological state or attitude toward delegation of tasks under implied risk.”
BackgroundOvertrust of robots in emergency evacuation scenarios
Robinette, P., Li, W., Allen, R., Howard, A. M., & Wagner, A. R. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451740 · Cited by 468
Online Verification of Automated Road Vehicles Using Reachability Analysis
Althoff, M., & Dolan, J. M. (2014) · IEEE Transactions on Robotics · doi:10.1109/tro.2014.2312453 · Cited by 412 · cited 1×
“of autonomous vehicles is formally verified online [100]–[102] by continually monitoring vehicle maneuvers using reachability analysis of all possible behaviors given some knowledge of”
MethodIntelligent agent transparency in human–agent teaming for multi-UxV management
Mercado, J. E., Rupp, M. A., Chen, J. Y. C., Barnes, M. J., Barber, D., & Procci, K. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815621206 · Cited by 403 · cited 1×
“For instance, studies [4], [124] uphold the significance of system transparency in instigating regulated trust levels.”
The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions
Whittlestone, J., Nyrup, R., Alexandrova, A., & Cave, S. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314289 · Cited by 399
The EU Approach to Ethics Guidelines for Trustworthy Artificial Intelligence
Smuha, N. A. (2019) · Computer Law Review International · doi:10.9785/cri-2019-200402 · Cited by 356 · cited 3×
“As such, recent guidelines [75], [130] proposed recommendations on performing the translation.”
BackgroundPsychosocial factors associated with intended use of automated vehicles: A simulated driving study
Buckley, L., Kaye, S.-A., & Pradhan, A. K. (2018) · Accident Analysis & Prevention · doi:10.1016/j.aap.2018.03.021 · Cited by 345 · cited 2×
“Researchershave usedawidevarietyofapproachesinmeasuringtrustwithself-reportedaccountsfromhumansubjectsunderstudy[34]–[38].”
Cho, J.-H., Chan, K., & Adali, S. (2015) · ACM Computing Surveys · doi:10.1145/2815595 · Cited by 342 · cited 1×
“Furthermore,severalsurveypapershavediscussed human–machinetrustwithregardtomultidisciplinarydefini-tionsoftrust,trustframeworks,andfactorsaffectingtrust[13].”
Trust and etiquette in high-criticality automated systems
Parasuraman, R., & Miller, C. A. (2004) · Communications of the ACM · doi:10.1145/975817.975844 · Cited by 311 · cited 1×
“Other topicsrangingfromtrustdynamicsinautonomousvehiclesto trustetiquetteandtrust-distrustdefinitionswerediscussedin [7]–[12].”
The relationship between trust in AI and trustworthy machine learning technologies
Toreini, E., Aitken, M., Coopamootoo, K., Elliott, K., Zelaya, C. G., & van Moorsel, A. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372834 · Cited by 296 · cited 2×
“People’s perceptions of trustworthiness can be affected by the environment, inclination toward technology, complexity of system (uncorroborated with explanations) and different interactions [70].”
BackgroundA review of the use of psychophysiological methods in game research
Kivikangas, J. M., Chanel, G., Cowley, B., Ekman, I., Salminen, M., Järvelä, S., & Ravaja, N. (2011) · Journal of Gaming & Virtual Worlds · doi:10.1386/jgvw.3.3.181_1 · Cited by 288 · cited 2×
“Subsequently, the interpretation of the relevant signals to draw a causal relationship with trust is still an elusive task [68], [69].”
BackgroundMeasuring Trust in Human Robot Interactions: Development of the “Trust Perception Scale-HRI”
Schaefer, K. E. (2016) · Robust Intelligence and Trust in Autonomous Systems · doi:10.1007/978-1-4899-7668-0_10 · Cited by 270 · cited 2×
“Thescales based on self-report measurements in [37], [48], [49], and [51]–[53] provided the most widely used scales in trust measurement.”
BackgroundExplainable AI for Designers: A Human-Centered Perspective on Mixed-Initiative Co-Creation
Zhu, J., Liapis, A., Risi, S., Bidarra, R., & Youngblood, G. M. (2018) · 2018 IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490433 · Cited by 255 · cited 1×
“pouse techniques that are not interpretable and trustworthy [82].”
BackgroundScenic: a language for scenario specification and scene generation
Fremont, D. J., Dreossi, T., Ghosh, S., Yue, X., Sangiovanni-Vincentelli, A. L., & Seshia, S. A. (2019) · Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/3314221.3314633 · Cited by 241 · cited 1×
“[92] and data-driven [93], [94] approaches to formally model uncertainties in human behavior and physical environments.”
BackgroundAdaptive trust calibration for human-AI collaboration
Okamura, K., & Yamada, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229132 · Cited by 239 · cited 1×
“Okamura and Yamada [125] assert that transparency of these systems alone is not enough to recover from such states of trust.”
BackgroundYou Want Me to Trust a ROBOT? The Development of a Human–Robot Interaction Trust Scale
Yagoda, R. E., & Gillan, D. J. (2012) · International Journal of Social Robotics · doi:10.1007/s12369-012-0144-0 · Cited by 238 · cited 2×
“Researchershave usedawidevarietyofapproachesinmeasuringtrustwithself-reportedaccountsfromhumansubjectsunderstudy[34]–[38].”
Fully Automated Driving: Impact of Trust and Practice on Manual Control Recovery
Payre, W., Cestac, J., & Delhomme, P. (2016) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815612319 · Cited by 229
DeepRED – Rule Extraction from Deep Neural Networks
Zilke, J. R., Loza Mencía, E., & Janssen, F. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46307-0_29 · Cited by 228
Trust in Robots: Challenges and Opportunities
Kok, B. C., & Soh, H. (2020) · Current Robotics Reports · doi:10.1007/s43154-020-00029-y · Cited by 221 · cited 4×
“Most recently, Kok and Soh [19] addressed how trust in robots can be gained, maintained, and calibrated.”
BackgroundInfluentialThe Seven Deadly Myths of "Autonomous Systems"
Bradshaw, J. M., Hoffman, R. R., Johnson, M., & Woods, D. D. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.70 · Cited by 216 · cited 1×
“In explaining this idea, Bradshaw et al. [21] emphasized that autonomy entails at least two dimensions: 1) self-directedness, which describes independence of an agent from its physical environment and social context; and 2) self-sufficiency, which describes self-generation of goals by the agent.”
MethodToward verified artificial intelligence
Seshia, S. A., Sadigh, D., & Sastry, S. S. (2022) · Communications of the ACM · doi:10.1145/3503914 · Cited by 215
The Perception And Measurement Of Human-robot Trust
Schaefer, K. E. (2013) · STARS (University of Central Florida) · Cited by 192
Overview of Reliability Testing
Elsayed, E. A. (2012) · IEEE Transactions on Reliability · doi:10.1109/tr.2012.2194190 · Cited by 175 · cited 1×
“Reliabilityreferstotheconsistencyofmeasurement[47].”
Using Formal Verification to Evaluate Human-Automation Interaction: A Review
Bolton, M. L., Bass, E. J., & Siminiceanu, R. I. (2013) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmca.2012.2210406 · Cited by 172 · cited 2×
“State-space explosion, expressiveness power of modeling formalisms, and model validity all determine the effectiveness of a formal verification method [108].”
BackgroundCreating Appropriate Trust in Automated Vehicle Systems: A Framework for HMI Design
Ekman, F., Johansson, M., & Sochor, J. (2018) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2017.2776209 · Cited by 169 · cited 2×
“Other widely used models of trust include those introduced in [9], [16], [30], and [31].”
BackgroundMethodInformation gathering actions over human internal state
Sadigh, D., Sastry, S. S., Seshia, S. A., & Dragan, A. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759036 · Cited by 160 · cited 1×
“[92] and data-driven [93], [94] approaches to formally model uncertainties in human behavior and physical environments.”
BackgroundA Classification Model for Sensing Human Trust in Machines Using EEG and GSR
Akash, K., Hu, W.-L., Jain, N., & Reid, T. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3132743 · Cited by 157 · cited 7×
“Theshiftawayfromself-reportingreliancefortrustmea-surement has paved a way toward using objective measurement approaches of trust correlates based on psycho-physiological sensors such as EEG, fMRI, and GSR.”
MethodInfluentialBlockchain for explainable and trustworthy artificial intelligence
Nassar, M., Salah, K., ur Rehman, M. H., & Svetinovic, D. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1340 · Cited by 149 · cited 1×
“as neural networks adopt model- simplification techniques, like the DeepRED algorithm [86] and block-chain solutions [87], to ascribe transparency to the model.”
MethodWing, J. M. (2021) · Communications of the ACM · doi:10.1145/3448248 · Cited by 148
Survey on Computational Trust and Reputation Models
Braga, D. D. S., Niemann, M., Hellingrath, B., & Neto, F. B. D. L. (2018) · ACM Computing Surveys · doi:10.1145/3236008 · Cited by 146 · cited 1×
“Othersurveypapersincludestudiesofalgorithmicassurances inhuman–autonomytrustrelationships[14]andcomputational trustandreputationmodels[15]thatfocusedonmodelsap-pliedtononengineeringfields.”
Towards Verified Artificial Intelligence
Seshia, S. A., Sadigh, D., & Sastry, S. S. (2016) · arXiv · doi:10.48550/arxiv.1606.08514 · Cited by 142 · cited 2×
“formal verification methods in autonomous systems [91].”
BackgroundFisher, M., Dennis, L., & Webster, M. (2013) · Communications of the ACM · doi:10.1145/2494558 · Cited by 137 · cited 1×
“Although it is quite difficult to provide a governing definition of autonomy without the situational context of the application, Fisher et al. [20] defined autonomous systems as those systems that decide for themselves what to do and when to do it.”
BackgroundCompositional Falsification of Cyber-Physical Systems with Machine Learning Components
Dreossi, T., Donzé, A., & Seshia, S. A. (2019) · Journal of Automated Reasoning · doi:10.1007/s10817-018-09509-5 · Cited by 124 · cited 2×
“generation [95] are some formal method-based research efforts used to optimize data specifications.”
BackgroundMethodMeasuring Individual Differences in the Perfect Automation Schema
Merritt, S. M., Unnerstall, J. L., Lee, D., & Huber, K. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815581247 · Cited by 123 · cited 1×
“Other trust scales include the perfect automation schema scale in [52], the psychometric instrument designed to measure cognitive and affective components of human–computer trust in [53], and the early scales include operators’ trust and prediction of trust scale in [4] and the trust evolution and…”
BackgroundReliability and Validity A Quick Review
Fitzner, K. (2007) · The Diabetes Educator · doi:10.1177/0145721707308172 · Cited by 120
Dismukes, R. K., Berman, B. A., & Loukopoulos, L. D. (2017) · Routledge eBooks · doi:10.4324/9781315238654 · Cited by 118 · cited 1×
“Anotherexampleisan incidentwithSouthwestAirlinesFlight1455thattraveledfrom Las Vegas to Burbank, California, in March 2000 [3].”
Backgroundde Visser, E. J., Monfort, S. S., Goodyear, K., Lu, L., O’Hara, M., Lee, M. R., Parasuraman, R., & Krueger, F. (2017) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816687205 · Cited by 113 · cited 1×
“psycho-physiological sensors to measure neural and physical correlates of trust [10], [39]–[43].”
BackgroundMeasuring Human Trust in a Virtual Assistant using Physiological Sensing in Virtual Reality
Gupta, K., Hajika, R., Pai, Y. S., Duenser, A., Lochner, M., & Billinghurst, M. (2020) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr46266.2020.00099 · Cited by 109
Kim, W., Kim, N., Lyons, J. B., & Nam, C. S. (2020) · Applied Ergonomics · doi:10.1016/j.apergo.2020.103056 · Cited by 108 · cited 1×
“TheStructuralEqua-tionModelmethod[120]providesonesuchrigorousfactor analysismethodinaccountingforhowalatentbehaviorlike trustisaffectedbytheidentifiedpredictors.”
A theory of formal synthesis via inductive learning
Jha, S., & Seshia, S. A. (2017) · Acta Informatica · doi:10.1007/s00236-017-0294-5 · Cited by 99 · cited 1×
“Finally, the theory of formal inductive synthesis [98] is an emerging solution used to address the challenge of synthesizing “correct-by-construction” design of models for a learning system.”
MethodInitial validation of the trust of automated systems test (TOAST)
Wojton, H. M., Porter, D., T. Lane, S., Bieber, C., & Madhavan, P. (2020) · The Journal of Social Psychology · doi:10.1080/00224545.2020.1749020 · Cited by 98 · cited 3×
“The Trust of Automated Systems Test (TOAST) measuring trust based on the theory of trust formation was proposed in [48].”
BackgroundAutomatic Generation of Natural Language Explanations
Costa, F., Ouyang, S., Dolog, P., & Lawlor, A. (2018) · Companion Proceedings of the 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3180308.3180366 · Cited by 97 · cited 1×
“Visual [84] and natural language [85] explanations can be used to rationalize and elaborate the decisions .”
BackgroundWhat Does it Mean to Trust a Robot?: Steps Toward a Multidimensional Measure of Trust
Ullman, D., & Malle, B. F. (2018) · Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3173386.3176991 · Cited by 95 · cited 2×
“Thescales based on self-report measurements in [37], [48], [49], and [51]–[53] provided the most widely used scales in trust measurement.”
BackgroundMethodEstablishing Appropriate Trust via Critical States
Huang, S. H., Bhatia, K., Abbeel, P., & Dragan, A. D. (2018) · 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593649 · Cited by 94
FairSquare: probabilistic verification of program fairness
Albarghouthi, A., D'Antoni, L., Drews, S., & Nori, A. V. (2017) · Proceedings of the ACM on Programming Languages · doi:10.1145/3133904 · Cited by 94 · cited 1×
“Furthermore, the work in [107] formulated ethical”
User Trust Dynamics: An Investigation Driven by Differences in System Performance
Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., & Chen, F. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025219 · Cited by 90
The Measurement of the Propensity to Trust Automation
Jessup, S. A., Schneider, T. R., Alarcon, G. M., Ryan, T. J., & Capiola, A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-21565-1_32 · Cited by 90 · cited 1×
“As different typesoftrustareinvestigated:trustpropensity[32],dispositional trust[16],history-basedtrust[33],andaffectiveandcognitive trust[5],whatismeasuredcontinuestovarywidelywithin theexistingliterature.”
LiDAR Data Integrity Verification for Autonomous Vehicle
Changalvala, R., & Malik, H. M. A. (2019) · IEEE Access · doi:10.1109/access.2019.2943207 · Cited by 83 · cited 2×
“by verifying the integrity of sensor data is proposed in terms of LIDAR [79] and Radar [80] sensors, which use the quantization index modulation based data hiding technique.”
MethodCompositional Reasoning in Model Checking
Berezin, S., Campos, S., & Clarke, E. M. (1998) · Lecture notes in computer science · doi:10.1007/3-540-49213-5_4 · Cited by 81 · cited 1×
“is designing scalable and efficient computational engines, was maneuvered using modular reasoning [97].”
MethodA Survey on Trust in Autonomous Systems
Shahrdar, S., Menezes, L., & Nojoumian, M. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-01177-2_27 · Cited by 75 · cited 2×
“Another review [17] summarized the research concerning autonomous systems by considering four categories of such systems, namely decision support systems, robots, self-driving cars, and autopilot systems.”
BackgroundMeasuring Trust with Psychophysiological Signals: A Systematic Mapping Study of Approaches Used
Ajenaghughrure, I. B., Sousa, S., & Lamas, D. (2020) · Multimodal Technologies and Interaction · doi:10.3390/mti4030063 · Cited by 75 · cited 1×
“Anotherapproachthathasgrowninrecentyearsistheuseof psycho-physiologicalsensorstomeasureneuralandphysical correlatesoftrust[10],[39]–[43].”
Israelsen, B. W., & Ahmed, N. R. (2019) · ACM Computing Surveys · doi:10.1145/3267338 · Cited by 74 · cited 1×
“Other survey papers include studies of algorithmic assurances in human–autonomy trust relationships [14] and computational trust and reputation models [15] that focused on models ap-”
BackgroundThe Handbook of Formal Methods in Human-Computer Interaction
Weyers, B., Bowen, J., Dix, A. J., & Palanque, P. A. (2017) · Human-computer interaction series · doi:10.1007/978-3-319-51838-1 · Cited by 73 · cited 2×
“Theextendedoperatorfunctionmodel[109]wasalsousedto modelhumanoperatorbehavior.”
MethodDrnec, K., Marathe, A. R., Lukos, J. R., & Metcalfe, J. S. (2016) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2016.00290 · Cited by 72 · cited 1×
“Measurements using physiological traits, such as employing heart-rate variability (HRV) measurement [55], evaluation of brain activity using fMRI [56], [57], and fNIRS [58], and electroencephalogram (EEG) [59]–[63], have been used to study and evaluate trust via its psychophysiological correlates.”
MethodAkash, K., McMahon, G., Reid, T., & Jain, N. (2020) · IEEE Control Systems · doi:10.1109/mcs.2020.3019151 · Cited by 72 · cited 2×
“They further expanded their experiments [66] to dynamically vary and calibrate automation”
BackgroundTrustworthy AI Development Guidelines for Human System Interaction
Wickramasinghe, C. S., Marino, D. L., Grandio, J., & Manic, M. (2020) · 2020 13th International Conference on Human System Interaction (HSI) · doi:10.1109/hsi49210.2020.9142644 · Cited by 71 · cited 1×
“” TAS cements human trust in AI-based systems so that societies can assuredly design, develop, and reap the benefits of these technologies [71].”
BackgroundSafety verification of autonomous vehicles for coordinated evasive maneuvers
Althoff, M., Althoff, D., Wollherr, D., & BUSS, M. J. (2010) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2010.5548121 · Cited by 68
Fisher, M., Dennis, L. A., & Webster, M. (2013) · Communications of the ACM · doi:10.1145/2500468.2494558 · Cited by 67
Wintersberger, P., Nicklas, H., Martlbauer, T., Hammer, S., & Riener, A. (2020) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3409120.3410659 · Cited by 66 · cited 1×
“The composition of feedback relayed to users is investigated and automatically generated using XAI principles to influence desired trust effects in automated vehicles [128].”
MethodData-Driven Probabilistic Modeling and Verification of Human Driver Behavior
Sadigh, D., Driggs-Campbell, K., Puggelli, A., Li, W., Shia, V., Bajcsy, R., Sangiovanni-Vincentelli, A. L., Sastry, S., & Seshia, S. A. (2014) · eScholarship (California Digital Library) · Cited by 63
Probabilistic verification of fairness properties via concentration
Bastani, O., Zhang, X., & Solar-Lezama, A. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360544 · Cited by 57 · cited 1×
“(FairSquare) in [104] and [105], respectively.”
Predictive model to assess user trust: a psycho-physiological approach
Ajenaghughrure, I. B., Sousa, S., Kosunen, I., & Lamas, D. (2019) · Indian Conference on Human-Computer Interaction (IndiaHCI) · doi:10.1145/3364183.3364195 · Cited by 56 · cited 3×
“a one-to-one correspondence between the signals from these sensors and trust states [54].”
BackgroundInfluentialHuman Trust Measurement Using an Immersive Virtual Reality Autonomous Vehicle Simulator
Shahrdar, S., Park, C., & Nojoumian, M. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314264 · Cited by 53 · cited 1×
“Also in [60], Shahrdar et al. combined the use of immersive virtual reality experimentation with self-reporting software to gauge and measure the subjects’ trust level in a self-driving autonomous vehicle.”
BackgroundTowards trustworthy automation: User interfaces that convey internal and external awareness
Rezvani, T., Driggs-Campbell, K., Sadigh, D., Sastry, S., Seshia, S. A., & Bajcsy, R. K. (2016) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2016.7795627 · Cited by 52 · cited 1×
“In [119], the investigation was carried out on how user interfaces that communicate both internal and external system awareness may increase the driver’s perceived and measured awareness, as well as their trust in the system.”
BackgroundEEG-Based Neural Correlates of Trust in Human-Autonomy Interaction
Wang, M., Hussein, A., Rojas, R. F., Shafi, K., & Abbass, H. A. (2018) · 2018 IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2018.8628649 · Cited by 50
Formalizing and guaranteeing human-robot interaction
Kress‐Gazit, H., Eder, K., Hoffman, G., Admoni, H., Argall, B., Ehlers, R., Heckman, C., Jansen, N., Knepper, R. A., Křetínský, J., et al. (2021) · Communications of the ACM · doi:10.1145/3433637 · Cited by 48 · cited 1×
“Anewelementoflimi-tationisintroducedin Anewelementoflimi-tationisintroducedin[110],whichalludestothedifficultyof formalizingamodelthatevolvesandadaptstochangeinhuman behavior.”
EEG-Based Classification of Emotional State Using an Autonomous Vehicle Simulator
Park, C., Shahrdar, S., & Nojoumian, M. (2018) · IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM) · doi:10.1109/sam.2018.8448945 · Cited by 45
Set-based computation of vehicle behaviors for the online verification of autonomous vehicles
Althoff, M., & Dolan, J. M. (2011) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2011.6083052 · Cited by 44 · cited 1×
“of autonomous vehicles is formally verified online [100]–[102] by continually monitoring vehicle maneuvers using reachability analysis of all possible behaviors given some knowledge of”
MethodSchaefer, K. E., Billings, D. R., Szalma, J. L., Adams, J. K., Sanders, T. L., Chen, J. Y., & Hancock, P. A. (2014) · doi:10.21236/ada607926 · Cited by 43
Modeling Trust Context in Networks
Adalı, S. (2013) · SpringerBriefs in computer science · doi:10.1007/978-1-4614-7031-1 · Cited by 43
Exploring Psycho-Physiological Correlates to Trust: Implications for Human-Robot-Human Interaction
Khalid, H. M., Liew, W. S., Nooralishahi, P., Rasool, Z., Helander, M. G., Kiong, L. C., & Ai-vyrn, C. (2016) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601160 · Cited by 43 · cited 1×
“heart-rate variability (HRV) measurement [55], evaluation of brain activity using fMRI [56], [57], and fNIRS [58], and electroencephalogram (EEG) [59]–[63], have been used to study and”
MethodTaxonomy of Pathways to Dangerous Artificial Intelligence
Yampolskiy, R. V. (2016) · ThinkIR: The University of Louisville's Institutional Repository (University of Louisville) · Cited by 41
Trust Measurement in Human–Automation Interaction: A Systematic Review
Brzowski, M., & Nathan-Roberts, D. (2019) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181319631462 · Cited by 40 · cited 1×
“Also, Brzowski and Nathan-Roberts [45] iterated the need to validate trust scales.”
BackgroundSocial secret sharing in cloud computing using a new trust function
Nojoumian, M., & Stinson, D. R. (2012) · Conference on Privacy, Security and Trust · doi:10.1109/pst.2012.6297936 · Cited by 37 · cited 1×
“provided a quantitative definition of trust that appeared in the authors’ prior work [18].”
BackgroundGoodyear, K., Parasuraman, R., Chernyak, S., de Visser, E., Madhavan, P., Deshpande, G., & Krueger, F. (2017) · Social Neuroscience · doi:10.1080/17470919.2016.1205131 · Cited by 37 · cited 1×
“Measurements using physiological traits, such as employing heart-rate variability (HRV) measurement [55], evaluation of brain activity using fMRI [56], [57], and fNIRS [58], and electroencephalogram (EEG) [59]–[63], have been used to study and evaluate trust via its psychophysiological correlates.”
MethodAdaptive stress testing of airborne collision avoidance systems
Lee, R., Kochenderfer, M. J., Mengshoel, O. J., Brat, G. P., & Owen, M. P. (2015) · IEEE/AIAA Digital Avionics Systems Conference (DASC) · doi:10.1109/dasc.2015.7311613 · Cited by 37
Incorporating behavioral trust theory into system development for ubiquitous applications
Hoffmann, H., & Söllner, M. (2012) · Personal and Ubiquitous Computing · doi:10.1007/s00779-012-0631-1 · Cited by 34 · cited 1×
“Other widely used models of trust include those introduced in [9], [16], [30], and [31].”
MethodTrust-based decision-making for smart and adaptive environments
Hammer, S., Wißner, M., & André, E. (2015) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-015-9160-8 · Cited by 32 · cited 2×
“Despite the effort to computationally relate system properties to trust, the system under investigation [131] is not a learning system, and the system properties reflect user perceptions of them instead of actual measurements.”
BackgroundMethodAdversarial Learning-Based On-Line Anomaly Monitoring for Assured Autonomy
Patel, N., Saridena, A. N., Choromanska, A., Krishnamurthy, P., & Khorrami, F. (2018) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2018.8593375 · Cited by 32
A Preliminary Study on Human Trust Measurements by EEG for Human-Machine Interactions
Dong, S.-Y., Kim, B.-K., Lee, K., & Lee, S. Y. (2015) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/2814940.2814993 · Cited by 31 · cited 1×
“Human-like cues were also demonstrated to play an important part in the neural response to an agent’s technical capability [65].”
BackgroundTrust Dynamics in Human Autonomous Vehicle Interaction: A Review of Trust Models
Basu, C., & Singhal, M. (2016) · National Conference on Artificial Intelligence · Cited by 30
An Empirical Model of Cultural Factors on Trust in Automation
Chien, S.-Y. J., Lewis, M., Semnani‐Azad, Z., & Sycara, K. P. (2014) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931214581181 · Cited by 26
Measuring Human Trust Behavior in Human-Machine Teams
Bindewald, J. M., Rusnock, C. F., & Miller, M. E. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-60591-3_5 · Cited by 25
Trust Game Database: Behavioral and EEG Data From Two Trust Games
Fu, C., Yao, X., Yang, X., Zheng, L., Li, J., & Wang, Y. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02656 · Cited by 23 · cited 1×
“Measurements using physiological traits, such as employing heart-rate variability (HRV) measurement [55], evaluation of brain activity using fMRI [56], [57], and fNIRS [58], and electroencephalogram (EEG) [59]–[63], have been used to study and evaluate trust via its psychophysiological correlates.”
MethodSemantic Adversarial Deep Learning
Seshia, S. A., Jha, S., & Dreossi, T. (2020) · IEEE Design and Test · doi:10.1109/mdat.2020.2968274 · Cited by 21 · cited 1×
“The robustness of machine learning models is formally specified in [106], which can be used as the basis for verification.”
BackgroundThe level of measurement of trust in automation
Wei, J., Bolton, M. L., & Humphrey, L. R. (2020) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2020.1766596 · Cited by 20 · cited 2×
“In[44],Wei et al. pointed out that, despite the availability of a large number of research in trust measurement, it is not clear what psychometric level of measurement is most appropriate for trust in automation.”
BackgroundMeasuring Trust of Autonomous Vehicles: A Development and Validation Study
García, D. Á., Kreutzer, C. P., Badillo-Urquiola, K. A., & Mouloua, M. (2015) · Communications in computer and information science · doi:10.1007/978-3-319-21383-5_102 · Cited by 19
The Impact of Autonomous Vehicles’ Active Feedback on Trust
Mackay, A., Fortes, I., Santos, C. L., Machado, D., Barbosa, P., Boas, V. V., Ferreira, J. P., Costa, N., Silva, C., & de Sousa, E. A. F. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-20497-6_32 · Cited by 19
Engineering Issues in Physiological Computing
Novak, D. (2014) · Human-computer interaction series · doi:10.1007/978-1-4471-6392-3_2 · Cited by 18 · cited 2×
“Additionally, there is difficulty in establishing a one-to-one relationship between such psycho-physiological correlates and trust states [67]–[69], [121].”
BackgroundSecurity in Autonomous Systems
Katzenbeisser, S., Polian, I., Regazzoni, F., & Stöttinger, M. (2019) · IEEE European Test Symposium (ETS) · doi:10.1109/ets.2019.8791552 · Cited by 12 · cited 1×
“Asurveyofsecurityinautonomoussystemsin[78]highlights majorsecuritythreatsinautonomoussystemsandprovides insightsonavailablecyber-securitysolutionsalongwiththeir constraints.”
Supporting operator reliance on automation through continuous feedback
Seppelt, B. D. (2018) · doi:10.17077/etd.90t9uqnh · Cited by 11 · cited 1×
“Seppelt [126] demonstrated that continuous system feedback promotes trust calibration in automation.”
BackgroundControl Improvisation with Probabilistic Temporal Specifications
Akkaya, I., Fremont, D. J., Valle, R., Donzé, A., Lee, E. A., & Seshia, S. A. (2016) · IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI) · doi:10.1109/iotdi.2015.33 · Cited by 8 · cited 1×
“Algorithmic improvisations [96] and counter example-guided training data”
BackgroundChecking formal verification models for human-automation interaction
van Paassen, M. M., Bolton, M. L., & Jimenez, N. (2014) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2014.6974507 · Cited by 7 · cited 1×
“The Symbolic Analysis Laboratory model checker is used in [111] to formally verify the interaction between a human operator and a single unmanned aerial vehicle (UAV).”
MethodRadar Data Integrity Verification Using 2D QIM-Based Data Hiding
Changalvala, R., Fedoruk, B., & Malik, H. M. A. (2020) · Sensors · doi:10.3390/s20195530 · Cited by 7 · cited 2×
“by verifying the integrity of sensor data is proposed in terms of LIDAR [79] and Radar [80] sensors, which use the quantization index modulation based data hiding technique.”
MethodControl Improvisation with Probabilistic Temporal Specifications
Akkaya, I., Fremont, D., Valle, R., Donzé, A., Lee, E., & Seshia, S. (2016) · eScholarship (California Digital Library) · Cited by 7
Trust, Influence and Reputation Management Based on Human Reasoning.
Nojoumian, M. (2015) · National Conference on Artificial Intelligence · Cited by 7
Case Study: Safety Verification of an Unmanned Underwater Vehicle
Lopez, D. M., Musau, P., Hamilton, N. P., Tran, H.-D., & Jonhson, T. T. (2020) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw50608.2020.00047 · Cited by 6 · cited 1×
“object is provided in [103] using an efficient overapproximate reachability scheme.”
MethodAkash, K., Reid, T., & Jain, N. (2018) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.2018.8431132 · Cited by 4 · cited 1×
“Furthermore, the signals from these sensors are typically non-stationary [122].”
BackgroundHuman-Agent Teaming - an Evolving Interaction Paradigm: An Innovative Measure of Trust
Palmer, S., Richards, D., Shelton-Rayner, G. K., Inch, D., & İzzetoğlu, K. (2019) · Research Repository (Delft University of Technology) · Cited by 4
Algorithmic Assurances and Self-Assessment of Competency Boundaries In Autonomous Systems
Israelsen, B. W. (2019) · CU Scholar (University of Colorado Boulder) · Cited by 3
Research works printed but not linked: 7
- Harris, 2019, NTSB investigation into deadly Uber self-driving car crash reveals lax attitude toward safety, IEEE Spectr.
- French, 2018, Trust in automation, Intell. Syst., 28, 84
- 2015, Unmanned Systems Integrated Roadmap FY20112036
- 10.1518/001872008X288574 · DOI · Paper page
- Madsen, Measuring human-computer trust, 6
- 10.1037/0003-066X.45.1.16 · doi:10.1037/0003-066x.45.1.16 · Paper page
- Cohen, Trust in decision aids: A model and its training implications
Cited by
Survey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
A Systematic Literature Review of User Trust in AI-Enabled Systems: An HCI Perspective
Bach, T. A., Khan, A., Hallock, H., Beltrão, G., & Sousa, S. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2138826 · Cited by 377
“This is likely due to trust being an abstract concept that can be relatively difficult to define or generalize (Gebru et al., 2022; Gulati et al., 2019; Sousa et al., 2016), with dynamic characteristics, meaning that trust can change over time and in different contexts and situations (Elkins &…”
BackgroundMethodHuman-machine Collaborative Decision-making: An Evolutionary Roadmap Based on Cognitive Intelligence
Ren, M., Chen, N., & Qiu, H. (2023) · International Journal of Social Robotics · doi:10.1007/s12369-023-01020-1 · Cited by 177
“(2) Discussing the mechanism and principle of human-machine intelligence integration ranging from fundamental information fusion technologies [30, 31] to high-level human-machine cognitive integration and mutual trust of humans and machines [32, 33].”
BackgroundHow do we assess the trustworthiness of AI? Introducing the trustworthiness assessment model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108671 · Cited by 76
How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79
Metadata
- Authors
- Gebru, B., Zeleke, L., Blankson, D., Nabil, M., Nateghi, S., Homaifar, A., & Tunstel, E.
- Affiliations
- North Carolina Agricultural and Technical State University (US)
- University of the People (US)
- ORCID
- Venue
- IEEE Transactions on Human-Machine Systems
- Published
- 2022-10
- Volume
- 52
- Issue
- 5
- Pages
- 952-962
- ISSN
- 2168-2291 (ISSN-L)
- 2168-2305
- OpenAlex
- W4213440380
- Cited by
- 101
- References
- 122
- Citations by year
- 2026: 23 · 2025: 39 · 2024: 22 · 2023: 13 · 2022: 4
- FWCI
- 18.42
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Human-Machine Systems (publishedVersion, cc-by)
- Funders
- Air Force Research Laboratory (FA8750-15-2-0116)
- NASA University Leadership Initiative (80NSSC20M0161)
- National Institute of Aerospace's Langley Distinguished Professor Program (C16-2B00-NCAT)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Computer Networks and Communications › Access Control and Trust
- Keywords
- human-machine trust, trust evaluation, human-machine interaction, autonomous systems, trust calibration, trustworthy AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.