Establishing and maintaining long-term human-computer relationships
The DOI matches its Crossref record; 100 of the 103 references it cites were found in a registry or an index.
Abstract
This research investigates the meaning of “human-computer relationship” and presents techniques for constructing, maintaining, and evaluating such relationships, based on research in social psychology, sociolinguistics, communication and other social sciences. Contexts in which relationships are particularly important are described, together with specific benefits (like trust) and task outcomes (like improved learning) known to be associated with relationship quality. We especially consider the problem of designing for long-term interaction, and define relational agents as computational artifacts designed to establish and maintain long-term social-emotional relationships with their users. We construct the first such agent, and evaluate it in a controlled experiment with 101 users who were asked to interact daily with an exercise adoption system for a month. Compared to an equivalent task-oriented agent without any deliberate social-emotional or relationship-building skills, the relational agent was respected more, liked more, and trusted more, even after four weeks of interaction. Additionally, users expressed a significantly greater desire to continue working with the relational agent after the termination of the study. We conclude by discussing future directions for this research together with ethical and other ramifications of this work for HCI designers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, research on relational agents (e.g.Bickmore and Picard, 2005) suggests that relationships often arise because of conversational behaviours, either verbal or non-verbal.”
From My Chatbot Companion - a Study of Human-Chatbot Relationships (2021) · cites this paper as Bickmore and Picard, 2005
In the citing paper
“By means of small talk, the actors can develop a common ground for the conversation (Bickmore and Picard 2005; Cassell and Bickmore 2003).”
From AI-based chatbots in customer service and their effects on user compliance (2021) · cites this paper as Bickmore and Picard 2005
In the citing paper
“Results are promising, as ECA interventions have been found to be easier to use [5] and used more frequently [5,11-13] than interventions without an ECA.”
In the citing paper
“Research has demonstrated that users respond and connect to conversational agents in social ways, and they can encourage honest disclosure [14,15].”
From Conversational agents in the treatment of mental health problems: mixed-method systematic review (2019) · cites this paper as [15]
In the citing paper
“therapeutic alliance [16,17]”
Citations
Cites
Politeness : Some Universals in Language Usage
Brown, P. (1987) · Cited by 12,119
Politeness: Some Universals in Language Usage
Brown, P., Levinson, S. C., & Gumperz, J. J. (1987) · Cambridge University Press eBooks · doi:10.1017/cbo9780511813085 · Cited by 10,438
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
Self-citationLife on the Screen: Identity in the Age of the Internet
Turkle, S. (1997) · Cumulative Index of Computer Aided Architectural Design · Cited by 6,607
Winnicott, D. W. (2012) · Routledge eBooks · doi:10.4324/9780203441022 · Cited by 5,685
Hand and mind: what gestures reveal about thought
(1993) · Choice Reviews Online · doi:10.5860/choice.30-2492 · Cited by 5,094
Development and validation of the working alliance inventory
Horvath, A. O., & Greenberg, L. S. (1989) · Journal of Counseling Psychology · doi:10.1037//0022-0167.36.2.223 · Cited by 3,520
Development and validation of the State Hope Scale.
Snyder, C. R. R., Sympson, S. C., Ybasco, F. C., Borders, T. F., Babyak, M. A., & Higgins, R. L. (1996) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.70.2.321 · Cited by 3,520
(1973) · Medical Entomology and Zoology · Cited by 2,836
Relation between working alliance and outcome in psychotherapy: A meta-analysis
Horvath, A. O., & Symonds, B. D. (1991) · Journal of Counseling Psychology · doi:10.1037/0022-0167.38.2.139 · Cited by 2,673
The Meaning of Things: Domestic Symbols and the Self
Csikszentmihalyi, M., & Halton, E. (1981) · Cambridge University Press eBooks · doi:10.1017/cbo9781139167611 · Cited by 2,482
The American Heritage Dictionary of the English Language
Rinehart, E. C. (1970) · The Journal of Higher Education · doi:10.1080/00221546.1970.11773921 · Cited by 2,466
Embodied Conversational Agents
(2000) · The MIT Press eBooks · doi:10.7551/mitpress/2697.001.0001 · Cited by 1,799
THE PRONOUNS OF POWER AND SOLIDARITY
Brown, R., & Gilman, A. (1968) · Readings in the Sociology of Language · doi:10.1515/9783110805376.252 · Cited by 1,782
Breazeal, C. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/2376.001.0001 · Cited by 1,553
Transition network grammars for natural language analysis
Woods, W. A. (1970) · Communications of the ACM · doi:10.1145/355598.362773 · Cited by 1,323
The role of the therapeutic alliance in psychotherapy.
Horvath, A. O., & Luborsky, L. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.4.561 · Cited by 1,279
Attitude change: Multiple roles for persuasion variables.
Petty, R. E., & Wegener, D. T. (1998) · Cited by 1,271
The problem of meaning in primitive languages
Malinowski, B. (1967) · Medical Entomology and Zoology · Cited by 1,135
(1981) · Poetry of Grammar and Grammar of Poetry · doi:10.1515/9783110802122.18 · Cited by 1,078
The Working Alliance: Theory, Research, and Practice
Bogart, G. (1995) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1995.49.2.306 · Cited by 994
McNeill, D. (2011) · Advances in Visual Semiotics · doi:10.1515/9783110874259.351 · Cited by 927
Maintenance strategies and romantic relationship type, gender and relational characteristics
Stafford, L., & Canary, D. J. (1991) · Journal of Social and Personal Relationships · doi:10.1177/0265407591082004 · Cited by 860
The elements of computer credibility
Fogg, B. J., & Tseng, H. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303001 · Cited by 802
The persona effect: affective impact of animated pedagogical agents
Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., & Bhogal, R. S. (1997) · Proceedings of the ACM SIGCHI Conference on Human factors in computing systems · doi:10.1145/258549.258797 · Cited by 801
BEAT: the Behavior Expression Animation Toolkit
Cassell, J., Vilhjálmsson, H. H., & Bickmore, T. (2001) · Proceedings of the 28th annual conference on Computer graphics and interactive techniques · doi:10.1145/383259.383315 · Cited by 739 · cited 2×
“This behavior was determined for each utterance using the BEAT text-to-embodiedspeech system [Cassell et al. 2001b], with several enhancements to support the exercise dialogues.”
BackgroundMethodSelf-citationCredibility and computing technology
Tseng, S., & Fogg, B. J. (1999) · Communications of the ACM · doi:10.1145/301353.301402 · Cited by 583
Embodiment in conversational interfaces: Rea
Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H., & Yan, H. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303150 · Cited by 566
Self-citationThis computer responds to user frustration:
Klein, J., Moon, Y., & Picard, R. W. (2002) · Interacting with Computers · doi:10.1016/s0953-5438(01)00053-4 · Cited by 558
Self-citationTudor-Locke, C. E., & Myers, A. M. (2001) · Research Quarterly for Exercise and Sport · doi:10.1080/02701367.2001.10608926 · Cited by 518
The transtheoretical model : Applications to exercise
Prochaska, J. O., & Marcus, B. H. (1994) · Cited by 511
The fundamental topoi of relational communication
Burgoon, J. K., & Hale, J. L. (1984) · Communication Monographs · doi:10.1080/03637758409390195 · Cited by 504
MAES, P. (1989) · Connection Science · doi:10.1080/09540098908915643 · Cited by 486
Relational agents: a model and implementation of building user trust
Bickmore, T., & Cassell, J. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365304 · Cited by 479
Self-citationThe concept of the alliance and its role in psychotherapy: Theoretical and empirical considerations.
Gaston, L. (1990) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/0033-3204.27.2.143 · Cited by 410
Andrew, A. M. (2003) · Kybernetes · doi:10.1108/k.2003.06732iae.003 · Cited by 406
Embodied conversational interface agents
Cassell, J. (2000) · Communications of the ACM · doi:10.1145/332051.332075 · Cited by 393
Computers that recognise and respond to user emotion: theoretical and practical implications
Picard, R. W., & Klein, J. (2002) · Interacting with Computers · doi:10.1016/s0953-5438(01)00055-8 · Cited by 392
Self-citation&NA; (1996) · Journal of Occupational & Environmental Medicine · doi:10.1097/00043764-199606000-00005 · Cited by 365
Perceived dimensions of interpersonal relations.
Wish, M., Deutsch, M., & Kaplan, S. J. (1976) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.33.4.409 · Cited by 364
(2013) · Handbook of Educational Psychology · doi:10.4324/9780203053874-11 · Cited by 351
University Press, C. (2021) · doi:10.26226/morressier.60dd99435d86378f03b413f9 · Cited by 351
The therapeutic alliance and its relationship to alcoholism treatment participation and outcome.
Connors, G. J., Carroll, K. M., DiClemente, C. C., Longabaugh, R., & Donovan, D. M. (1997) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.65.4.588 · Cited by 337
Stafford, L., Dainton, M., & Haas, S. (2000) · Communication Monographs · doi:10.1080/03637750009376512 · Cited by 330
Readings in natural language processing
(1989) · Journal of Pragmatics · doi:10.1016/0378-2166(89)90051-9 · Cited by 309
Collagen: applying collaborative discourse theory to human-computer interaction
Rich, C., Sidner, C. L., & Lesh, N. B. (2001) · doi:10.1609/aimag.v22i4.1589 · Cited by 287
Dainton, M., & Stafford, L. (1993) · Journal of Social and Personal Relationships · doi:10.1177/026540759301000206 · Cited by 277
Interaction Rhythms: Periodicity in Communicative Behavior
Davis, M. (1982) · Medical Entomology and Zoology · Cited by 271
Hardware companions?: what online AIBO discussion forums reveal about the human-robotic relationship
Friedman, B., Kahn, P. H., & Hagman, J. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642660 · Cited by 270
External manifestations of trustworthiness in the interface
Cassell, J., & Bickmore, T. (2000) · Communications of the ACM · doi:10.1145/355112.355123 · Cited by 255
Self-citationThe Development of Working Relationships
(2014) · Intellectual Teamwork · doi:10.4324/9781315807645-11 · Cited by 254
Getting Acquainted in Conversation: A study of initial interactions
Svennevig, J. (2000) · Pragmatics & Beyond New Series · doi:10.1075/pbns.64 · Cited by 237
Reconsidering power and distance
Spencer-Oatey, H. (1996) · Journal of Pragmatics · doi:10.1016/0378-2166(95)00047-x · Cited by 236
Proceedings of the Association for Computational Linguistics 2014
Heafield, K., Kayser, M., & Manning, C. D. (2014) · Cited by 228
The Psychotherapy Relationship: Theory, Research, and Practice
Gelso, C. J., & Hayes, J. A. (1998) · Medical Entomology and Zoology · Cited by 226
The therapeutic alliance as interpersonal process.
Henry, W. P., & Strupp, H. H. (1994) · Cited by 225
Duck, S. (2007) · SAGE Publications Ltd eBooks · doi:10.4135/9781446214336 · Cited by 221
Keijsers, G. P. J., Schaap, C. P. D. R., & Hoogduin, C. A. L. (2000) · Behavior Modification · doi:10.1177/0145445500242006 · Cited by 202
Luborsky, L. B. (1994) · Cited by 190
Non-verbal cues for discourse structure
Cassell, J., Nakano, Y. I., Bickmore, T. W., Sidner, C. L., & Rich, C. (2001) · Proceedings of the 39th Annual Meeting on Association for Computational Linguistics - ACL '01 · doi:10.3115/1073012.1073028 · Cited by 181
Self-citationLinguistic Routines and Politeness in Greeting and Parting
LAVER, J. (1981) · Rasmus Rask Studies in Pragmatic Linguistics, Volume 2, Conversational Routine · doi:10.1515/9783110809145.289 · Cited by 169
An expert system intervention for smoking cessation
Velicer, W. F., & Prochaska, J. O. (1999) · Patient Education and Counseling · doi:10.1016/s0738-3991(98)00129-3 · Cited by 162
Argyle, M. (2013) · Routledge eBooks · doi:10.4324/9780203753835 · Cited by 160
What makes Internet users visit cyber stores again? key design factors for customer loyalty
Lee, J., Kim, J., & Moon, J. Y. (2000) · Proceedings of the SIGCHI conference on Human Factors in Computing Systems · doi:10.1145/332040.332448 · Cited by 158
The role of vision in face-to-face and mediated communication.
Whittaker, S., & O’Conaill, B. (1997) · Medical Entomology and Zoology · Cited by 158
Horvath, A. O. (1994) · Cited by 151
The therapeutic alliance in cognitive-behavioral therapy.
Castonguay, L. G., Constantino, M. J., McAleavey, A. A., & Goldfried, M. R. (1994) · Cited by 145
Media inequality in conversation
Shechtman, N., & Horowitz, L. M. (2003) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642661 · Cited by 139
Effective Helping: Interviewing and Counseling Techniques
Okun, B. F. (1976) · Medical Entomology and Zoology · Cited by 134
Bachelor, A. (1991) · Psychotherapy · doi:10.1037/0033-3204.28.4.534 · Cited by 133
Improvising linguistic style: social and affective bases for agent personality
Walker, M. A., Cahn, J. E., & Whittaker, S. (1997) · doi:10.1145/267658.267680 · Cited by 128
Features of Service Relationships and Encounters
Gutek, B. A., Cherry, B., Bhappu, A. D., Schneider, S. K., & WOOLF, L. (2000) · Work and Occupations · doi:10.1177/0730888400027003004 · Cited by 125
Small talk : analysing phatic discourse
Schneider, K. P. (1988) · Hitzeroth eBooks · Cited by 124
More than just a pretty face: affordances of embodiment
Cassell, J., Bickmore, T., Vilhjálmsson, H. H., & Yan, H. (2000) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/325737.325781 · Cited by 102
Self-citationStrategic uses of peer learning in children's education.
Damon, W., & Phelps, E. (1989) · Cited by 100
Cooperation, close relationships, and cognitive development.
Hartup, W. W. (1996) · Cited by 96
Session impact, working alliance, and treatment outcome in brief counseling.
Mallinckrodt, B. S. (1993) · Journal of Counseling Psychology · doi:10.1037/0022-0167.40.1.25 · Cited by 90
Predicting marital satisfaction from behavior: Do all roads really lead to Rome?
Gill, D. S., Christensen, A., & Fincham, F. D. (1999) · Personal Relationships · doi:10.1111/j.1475-6811.1999.tb00198.x · Cited by 83
Friends' and Acquaintances' Conversations I: Perceived Differences
Planalp, S., & Benson, A. (1992) · Journal of Social and Personal Relationships · doi:10.1177/0265407592094002 · Cited by 72
Helping Behaviors in the Natural Environment: Dimensions and Correlates of Helping
McGuire, A. M. (1994) · Personality and Social Psychology Bulletin · doi:10.1177/0146167294201004 · Cited by 71
Humor in task-oriented computer-mediated communication and human-computer interaction
Morkes, J., Kernal, H. K., & Nass, C. (1998) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/286498.286704 · Cited by 63
Relational Continuity Constructional Units and the Maintenance of Relationships
Gilbertson, J., Dindia, K., & Allen, M. R. (1998) · Journal of Social and Personal Relationships · doi:10.1177/0265407598156004 · Cited by 62
This Computer Responds to User Frustration Theory, Design, Results, and Implications
Klein, J., Moon, Y., & Picard, R. W. (2002) · Cited by 61
Self-citationSeven-Day Physical Activity Recall
&NA; (1997) · Medicine & Science in Sports & Exercise · doi:10.1097/00005768-199706001-00018 · Cited by 57
A Lay Theory of Relational Satisfaction with Best Friends
Cole, T., & Bradac, J. J. (1996) · Journal of Social and Personal Relationships · doi:10.1177/0265407596131004 · Cited by 55
Friends' and Acquaintances' Conversations II: Coded Differences
Planalp, S. (1993) · Journal of Social and Personal Relationships · doi:10.1177/0265407593103003 · Cited by 53
An empirical study on the trustworthiness of life-like interface agents
van Mulken, S., André, E., & Müller, J. M. (1999) · Cited by 50
Behavioral management techniques and exercise promotion
Knapp, D. (1988) · Medical Entomology and Zoology · Cited by 48
Longitudinal Changes in College Students' Exercise Participation
Pinto, B. M., Cherico, N. P., Szymanski, L., & Marcus, B. H. (1998) · Journal of American College Health · doi:10.1080/07448489809595615 · Cited by 46
Modeling Gaze Behavior as a Function of Discourse Structure
Torres, O. E., Cassell, J., & Prevost, S. (1998) · Cited by 44
Effects of echoic mimicry using hummed sounds on human–computer interaction
Suzuki, N., Takeuchi, Y., Ishii, K., & Okada, M. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00180-2 · Cited by 43
Celio, A. A., Winzelberg, A. J., Dev, P., & Taylor, C. B. (2002) · Journal of Psychiatric Practice · doi:10.1097/00131746-200201000-00003 · Cited by 27
Emotional usability of customer interfaces: focusing on cyber banking system interfaces
Kim, J., & Moon, J. Y. (1997) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1120212.1120395 · Cited by 26
An explanatory and “argumentative” interface for a model-based diagnostic system
Miller, C. A., & Larson, R. (1992) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/142621.142627 · Cited by 17
Towards the design of multimodal interfaces for handheld conversational characters
Bickmore, T. (2002) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/506443.506598 · Cited by 15
Self-citationWebDietAID: an interactive Web-based nutritional counselor.
Riva, A., Smigelski, C., & Friedman, R. (2000) · PubMed · Cited by 12
Behavior Change in the Human Services: An Introduction to Principles and Applications
Sundel, M., & Sundel, S. S. (1999) · Cited by 4
van Mulken, S. (1999) · Deutscher Universitätsverlag eBooks · doi:10.1007/978-3-322-97669-7_6 · Cited by 1
(1998)
Research works printed but not linked: 29
- Anselmi K., 1997, Relationship selling: How personal characteristics of salespeople affect buyer satisfaction, J. Soc. Behav. Person., 12, 539
- Bickmore T., 2002, Proceedings of the CHI'02
- Bickmore T., Proceedings of the CHI'01
- Brehm S. 1992. Intimate Relationships. McGraw-Hill New York. Brehm S. 1992. Intimate Relationships. McGraw-Hill New York.
- Cassell J., Proceedings of CHI'99
- Cassell J., Proceedings of the IUI 2000
- Cassell J., Proceedings of the SIGGRAPH '01
- Duck S. 1991. Understanding Relationships. Guilford Press New York. Duck S. 1991. Understanding Relationships. Guilford Press New York.
- Fogg B. J., Proceedings of CHI '99
- Friedman B., Proceedings of CHI'03 (Ft
- Horvath A., The Working Alliance: Theory, Research and Practice
- Jakobson R., Style in language
- Kelley H., 1983, Epilogue: An essential science
- Kim J., 2021, Proceedings of CHI 97
- Lee J., 2040, Proceedings of CHI'00
- Lester J. C., Proceedings of CHI '97
- Lim T., Facework and interpersonal relationships. The challenge of facework: Cross-cultural and interpersonal issues
- Luborsky L., The Working Alliance: Theory, Research and Practice
- Malinowski B., The Meaning of Meaning
- Miller C., Proceedings of the ACM Symposium on User Interface Software and Technology
- Moon Y., Intimate self-disclosure exchanges: Using computers to build reciprocal relationships with consumers. Harvard Business School
- Morkes J., Proceedings of CHI 98
- Reeves B. and Nass C. 1996. The Media Equation. Cambridge University Press Cambridge MA. Reeves B. and Nass C. 1996. The Media Equation. Cambridge University Press Cambridge MA.
- Richmond V., Nonverbal Behavior in Interpersonal Relations. Allyn & Bacon
- Shechtman N., Proceedings of CHI'03 (Ft
- Sunde M., 1999, Human Services: An Introduction to Principles and Applications
- Svennevig J., Getting Acquainted in Conversation. John Benjamins
- Torres O. E., Proceedings of the 1st International Workshop on Human-Computer Conversation.
- Walker M. A., Proceedings of Autonomous Agents 97
Cited by
AI-based chatbots in customer service and their effects on user compliance
Adam, M., Wessel, M., & Benlian, A. (2021) · Electronic Markets · doi:10.1007/s12525-020-00414-7 · Cited by 1,195
“By means of small talk, the actors can develop a common ground for the conversation (Bickmore and Picard 2005; Cassell and Bickmore 2003).”
BackgroundLuger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
Feil-Seifer, D., & Mataric, M. J. (2005) · 9th International Conference on Rehabilitation Robotics, 2005. ICORR 2005. · doi:10.1109/icorr.2005.1501143 · Cited by 972
Social Robots for Long-Term Interaction: A Survey
Leite, I., Martinho, C., & Paiva, A. (2013) · International Journal of Social Robotics · doi:10.1007/s12369-013-0178-y · Cited by 871
“While routine behaviours are those who “people engage in for other reasons which serve to maintain a relationship as a side effect” [7], strategic behaviours are those which “individuals enact with the conscious intent of preserving or improving the rela-tionship” [65].”
BackgroundDefining Internet-Supported Therapeutic Interventions
Barak, A., Klein, B., & Proudfoot, J. G. (2009) · Annals of Behavioral Medicine · doi:10.1007/s12160-009-9130-7 · Cited by 731
Metadata
- Authors
- Bickmore, T. W., & Picard, R. W.
- Affiliations
- Northeastern University (US)
- MIT Media Lab (US)
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2005-06
- Volume
- 12
- Issue
- 2
- Pages
- 293-327
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W1985945240
- Cited by
- 1,333
- References
- 103
- Citations by year
- 2026: 174 · 2025: 120 · 2024: 80 · 2023: 64 · 2022: 64 · 2021: 83 · 2020: 70 · 2019: 58 · 2018: 47 · 2017: 49 · 2016: 53 · 2015: 42 · 2014: 63 · 2013: 68 · 2012: 44
- FWCI
- 42.38
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Computer-Human Interaction (publishedVersion)
- Topics
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Keywords
- human-computer relationship, relational agents, long-term interaction, trust, relationship quality, exercise adoption, user-agent interaction, agent design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.