The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Reliability of Nominal Data Based on Qualitative Judgments
Abstract
Most research related to the reliability and validity of marketing measures has focused on multi-item quantitative scales. In contrast, little attention has been given to the quality of nominal scale data developed from qualitative judgments. Judgment-based (“coded”) nominal scale data are important and frequently used in marketing research-for example, in analysis of consumer responses to open-ended survey questions, in cognitive response research, in meta-analysis, and in content analysis. The authors address opportunities and challenges involved in evaluating and improving the quality of judgment-based nominal scale data, with specific emphasis on the use of multiple judges. They review approaches commonly used in other disciplines, then develop a new index of reliability that is more appropriate for the type of interjudge data typically found in marketing studies. Data from a cognitive response experiment are used to illustrate the new index and compare it with other common measures. The authors conclude with suggestions on how to improve the design of studies that rely on judgment-coded data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“When the described coding procedures were applied, the intercoder reliability according to Perreault and Leigh (1989) was .93, which exceeded the required threshold of .8.”
From A Meta-Analysis of Organizational Complaint Handling and Customer Responses (2011) · cites this paper as Perreault and Leigh (1989)
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
A Paradigm for Developing Better Measures of Marketing Constructs
Churchill, G. A. (1979) · Journal of Marketing Research · doi:10.1177/002224377901600110 · Cited by 16,549
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733
Reliability of Content Analysis: The Case of Nominal Scale Coding
Scott, W. A. (1955) · Public Opinion Quarterly · doi:10.1086/266577 · Cited by 1,853
Coefficient Kappa: Some Uses, Misuses, and Alternatives
Brennan, R. L., & Prediger, D. J. (1981) · Educational and Psychological Measurement · doi:10.1177/001316448104100307 · Cited by 1,676
Reliability: A Review of Psychometric Basics and Recent Marketing Practices
Peter, J. P. (1979) · Journal of Marketing Research · doi:10.2307/3150868 · Cited by 1,577
Large sample standard errors of kappa and weighted kappa.
Fleiss, J. L., Cohen, J., & Everitt, B. S. (1969) · Psychological Bulletin · doi:10.1037/h0028106 · Cited by 1,536
Bettman, J. R., & Park, C. W. (1980) · Journal of Consumer Research · doi:10.1086/208812 · Cited by 1,344
Cited by
Sorted by citations
Items 101–200 of 1,002
Park, N., & Mezias, J. M. (2005) · Strategic Management Journal · doi:10.1002/smj.489 · Cited by 262
Irritating aspects of the shopping environment.
d'Astous, A. (2000) · Journal of Business Research · doi:10.1016/s0148-2963(99)00002-8 · Cited by 267
THE POWER OF REFLECTION: An Empirical Examination of Nostalgia Advertising Effects
Muehling, D. D., Sprott, D. E., & Sprott, D. E. (2004) · Journal of Advertising · doi:10.1080/00913367.2004.10639165 · Cited by 281
Supply chain collaboration and sustainability: a profile deviation analysis
Blome, C., Paulraj, A., & Schuetz, K. (2014) · International Journal of Operations & Production Management · doi:10.1108/ijopm-11-2012-0515 · Cited by 279
Krishnamurthy, S., & Kucuk, S. U. (2008) · Journal of Business Research · doi:10.1016/j.jbusres.2008.09.003 · Cited by 263
Consumer evaluations of sponsorship programmes
d’Astous, A., & Bitz, P. (1995) · European Journal of Marketing · doi:10.1108/03090569510102504 · Cited by 261
‘It Was Such a Handy Term’: Management Fashions and Pragmatic Ambiguity*
Giroux, H. (2006) · Journal of Management Studies · doi:10.1111/j.1467-6486.2006.00623.x · Cited by 256
Environmental Advertising Claims: A Preliminary Investigation
Kangun, N., Carlson, L., & Grove, S. J. (1991) · Journal of Public Policy & Marketing · doi:10.1177/074391569101000203 · Cited by 255
McColl‐Kennedy, J. R., Hogan, S. J., Witell, L., & Snyder, H. (2016) · Journal of Business Research · doi:10.1016/j.jbusres.2016.07.006 · Cited by 270
Hardesty, D. M., & Bearden, W. O. (2003) · Journal of Retailing · doi:10.1016/s0022-4359(03)00004-6 · Cited by 254
PERSPECTIVE—The Myth of Firm Performance
Miller, C. C., Washburn, N. T., & Glick, W. H. (2012) · Organization Science · doi:10.1287/orsc.1120.0762 · Cited by 268
Stafford, M. R., Stafford, T. F., & Day, E. (2002) · Journal of Advertising · doi:10.1080/00913367.2002.10673664 · Cited by 247
A Resource-based View of Strategic Alliances and Firm Value in the Electronic Marketplace
Park, N., Mezias, J. M., & Song, J. (2004) · Journal of Management · doi:10.1016/j.jm.2002.11.001 · Cited by 265
Economic Incentives for Voluntary Reporting on Internal Risk Management and Control Systems
Deumes, R., & Knechel, W. R. (2008) · Auditing A Journal of Practice & Theory · doi:10.2308/aud.2008.27.1.35 · Cited by 244
The Self Online: The Utility of Personal Home Pages
Papacharissi, Z. (2002) · Journal of Broadcasting & Electronic Media · doi:10.1207/s15506878jobem4603_3 · Cited by 242
Olson, E. L., & Bakke, G. (2001) · Journal of Product Innovation Management · doi:10.1016/s0737-6782(01)00111-4 · Cited by 240
Countries and their products: A cognitive structure perspective
Shimp, T. A., Samiee, S., & Madden, T. J. (1993) · Journal of the Academy of Marketing Science · doi:10.1007/bf02894524 · Cited by 238
Hamilton, R. W., & Thompson, D. V. (2007) · Journal of Consumer Research · doi:10.1086/520073 · Cited by 237
Karna, A., Richter, A., & Riesenkampff, E. (2015) · Strategic Management Journal · doi:10.1002/smj.2379 · Cited by 235
The effects of consumer knowledge on responses to event sponsorships
Roy, D. P., & Cornwell, T. B. (2004) · Psychology and Marketing · doi:10.1002/mar.20001 · Cited by 234
Managerial Identification of Competitors
Clark, B. H., & Montgomery, D. B. (1999) · Journal of Marketing · doi:10.1177/002224299906300305 · Cited by 233
The Long-Term Predictive Validity of the Consumers' Need for Uniqueness Scale
Tian, K. T., & McKenzie, K. S. (2001) · Journal of Consumer Psychology · doi:10.1207/s15327663jcp1003_5 · Cited by 231
Chu, S., Deng, T., & Cheng, H. G. (2020) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-05-2020-0480 · Cited by 230
Conceptualizing Destination Brand Equity Dimensions from a Consumer-Based Brand Equity Perspective
Im, H. H., Kim, S., Elliot, S., & Han, H. (2012) · Journal of Travel & Tourism Marketing · doi:10.1080/10548408.2012.674884 · Cited by 245
Men, L. R., & Tsai, W. S. (2011) · Public Relations Review · doi:10.1016/j.pubrev.2011.10.006 · Cited by 227
Synergy Sources, Target Autonomy, and Integration in Acquisitions
Zaheer, A., Castañer, X., & Souder, D. (2011) · Journal of Management · doi:10.1177/0149206311403152 · Cited by 245
Narayanan, S., Narasimhan, R., & Schoenherr, T. (2014) · Journal of Operations Management · doi:10.1016/j.jom.2014.11.004 · Cited by 225
The Worth of Product Placement in Successful Films: An Event Study Analysis
Wiles, M. A., & Danielova, A. N. (2009) · Journal of Marketing · doi:10.1509/jmkg.73.4.044 · Cited by 220
The Internal Service Encounter
Gremler, D. D., Bitner, M. J., & Evans, K. R. (1994) · International Journal of Service Industry Management · doi:10.1108/09564239410057672 · Cited by 220
Lam, S. Y., Chiang, J., & Parasuraman, A. (2008) · Journal of Interactive Marketing · doi:10.1002/dir.20119 · Cited by 216
Framing Artificial Intelligence in American Newspapers
Chuan, C., Tsai, W., & Su, Y.-O.-C. (2019) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314285 · Cited by 184
Belhadi, A., Kamble, S., Subramanian, N., Singh, R. K., & Mani, V. (2024) · International Journal of Operations & Production Management · doi:10.1108/ijopm-11-2022-0737 · Cited by 214
Examining the success factors for mobile work in healthcare: A deductive study
Chatterjee, S., Chakraborty, S., Sarker, S., Sarker, S., & Lau, F. (2008) · Decision Support Systems · doi:10.1016/j.dss.2008.11.003 · Cited by 219
Behavioral supply management: a taxonomy of judgment and decision‐making biases
Carter, C. R., Kaufmann, L., & Michel, A. (2007) · International Journal of Physical Distribution & Logistics Management · doi:10.1108/09600030710825694 · Cited by 221
Internet Exchanges for Used Goods: An Empirical Analysis of Trade Patterns and Adverse Selection1
Ghose, A. (2009) · MIS Quarterly · doi:10.2307/20650292 · Cited by 210
Willems, K., Smolders, A., Brengman, M., Luyten, K., & Schöning, J. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.10.066 · Cited by 207
Moving from products to solutions: Strategic approaches for developing capabilities
PAIOLA, M. U., Saccani, N., Perona, M., & Gebauer, H. (2012) · European Management Journal · doi:10.1016/j.emj.2012.10.002 · Cited by 221
Measurement for the Social Sciences
Rossiter, J. R. (2010) · Springer eBooks · doi:10.1007/978-1-4419-7158-6 · Cited by 212
Doyle, J. P., Su, Y., & Kunkel, T. (2020) · European Sport Management Quarterly · doi:10.1080/16184742.2020.1806897 · Cited by 205
Without You, I'm Nothing: Performances of the Self on Twitter
Papacharissi, Z. (2012) · International journal of communication · Cited by 201
Stuck in the Past: Why Managers Persist with New Product Failures
Biyalogorsky, E., Boulding, W., & Staelin, R. (2006) · Journal of Marketing · doi:10.1509/jmkg.70.2.108 · Cited by 203
The Development, Change, and Transformation of Rhetorical Style in Magazine Advertisements 1954–1999
Phillips, B. J., & McQuarrie, E. F. (2002) · Journal of Advertising · doi:10.1080/00913367.2002.10673681 · Cited by 200
A Resource‐Process Framework of New Service Development
Froehle, C. M., & Roth, A. V. (2007) · Production and Operations Management · doi:10.1111/j.1937-5956.2007.tb00174.x · Cited by 199
Satisfiers and Dissatisfiers in the Online Environment
Holloway, B. B., & Beatty, S. E. (2008) · Journal of Service Research · doi:10.1177/1094670508314266 · Cited by 216
Handley, S. M., & Benton, W. C. (2013) · Journal of Operations Management · doi:10.1016/j.jom.2012.12.003 · Cited by 198
Opportunities for Research in Third-Party Logistics
Maloni, M. J., & Carter, C. R. (2006) · Transportation Journal · doi:10.2307/20713632 · Cited by 196
Product returns on the Internet: A case of mixed signals?
Bonifield, C. M., Cole, C. A., & Schultz, R. L. (2009) · Journal of Business Research · doi:10.1016/j.jbusres.2008.12.009 · Cited by 191
Social presence and digital dialogic communication: engagement lessons from top social CEOs
Men, L. R., Tsai, W. S., Chen, Z. F., & Ji, Y. G. (2018) · Journal of Public Relations Research · doi:10.1080/1062726x.2018.1498341 · Cited by 191
Is the World Flat or Spiky? Information Intensity, Skills, and Global Service Disaggregation
Mithas, S., & Whitaker, J. (2007) · Information Systems Research · doi:10.1287/isre.1070.0131 · Cited by 191
An examination on multiple celebrity endorsers in advertising
Hsu, C.-K., & McDonald, D. (2002) · Journal of Product & Brand Management · doi:10.1108/10610420210419522 · Cited by 186
The development of business model research: A bibliometric review
Budler, M., Župič, I., & Trkman, P. (2021) · Journal of Business Research · doi:10.1016/j.jbusres.2021.06.045 · Cited by 216
Measuring Soft-Sell Versus Hard-Sell Advertising Appeals
Okazaki, S., Mueller, B., & Taylor, C. R. (2010) · Journal of Advertising · doi:10.2753/joa0091-3367390201 · Cited by 185
Tangibility in Services Advertising: An Investigation of Verbal versus Visual Cues
Stafford, M. R. (1996) · Journal of Advertising · doi:10.1080/00913367.1996.10673504 · Cited by 184
Repeatability and Reproducibility of Decisions by Latent Fingerprint Examiners
Ulery, B. T., Hicklin, R. A., Buscaglia, J., & Roberts, M. A. (2012) · PLoS ONE · doi:10.1371/journal.pone.0032800 · Cited by 183
Stakeholder engagement and dialogic accounting
Bellucci, M., Simoni, L., Acuti, D., & Manetti, G. (2019) · Accounting Auditing & Accountability Journal · doi:10.1108/aaaj-09-2017-3158 · Cited by 211
Thirumalai, S., & Sinha, K. K. (2010) · Journal of Operations Management · doi:10.1016/j.jom.2010.11.009 · Cited by 188
Supply chain agility: scale development
Li, X., Goldsby, T. J., & Holsapple, C. W. (2009) · The International Journal of Logistics Management · doi:10.1108/09574090911002841 · Cited by 179
Customer Complaints and Recovery Effectiveness: A Customer Base Approach
Knox, G., & van Oest, R. D. (2014) · Journal of Marketing · doi:10.1509/jm.12.0317 · Cited by 188
Surucu-Balci, E., Iris, Ç., & Balcı, G. (2023) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.122978 · Cited by 188
Sustainable supply chain management: continuing evolution and future directions
Carter, C. R., Hatton, M. R., Wu, C., & Chen, X. (2019) · International Journal of Physical Distribution & Logistics Management · doi:10.1108/ijpdlm-02-2019-0056 · Cited by 177
Assessing the Use and Impact of Humor on Advertising Effectiveness: A Contingency Approach
Spotts, H. E., Weinberger, M. G., & Parsons, A. (1997) · Journal of Advertising · doi:10.1080/00913367.1997.10673526 · Cited by 175
Digital Transformation and Marketing Activities in Small and Medium-Sized Enterprises
Ziółkowska, M. (2021) · Sustainability · doi:10.3390/su13052512 · Cited by 200
The Impact of Information Level on the Effectiveness of U.S. and Korean Television Commercials
Taylor, C. R., Miracle, G. E., & Wilson, R. D. (1997) · Journal of Advertising · doi:10.1080/00913367.1997.10673514 · Cited by 174
Han, H., & Ryu, K. (2011) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2011.09.015 · Cited by 189
The effects of perceived scarcity on consumers’ processing of price information
Suri, R., Kohli, C. S., & Monroe, K. B. (2007) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-006-0008-y · Cited by 172
Aydın, G. (2019) · Journal of Hospitality Marketing & Management · doi:10.1080/19368623.2019.1588824 · Cited by 172
Marketing Research: A State-of-the-Art Review and Directions for the Twenty-First Century
Malhotra, N. K., Peterson, M., & Kleiser, S. B. (1999) · Journal of the Academy of Marketing Science · doi:10.1177/0092070399272004 · Cited by 180
Hughes, M. A., & Garrett, D. E. (1990) · Journal of Marketing Research · doi:10.1177/002224379002700206 · Cited by 177
Typologies of e‐commerce retail failures and recovery strategies
Forbes, L. P., Kelley, S. W., & Hoffman, K. D. (2005) · Journal of Services Marketing · doi:10.1108/08876040510609907 · Cited by 178
Fashion and the metaverse: Clarifying the domain and establishing a research agenda
Park, H., & Lim, R. E. (2023) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2023.103413 · Cited by 171
Verisimilitude or Advertising? Brand Appearances on Prime‐Time Television
Avery, R. J., & Ferraro, R. (2000) · Journal of Consumer Affairs · doi:10.1111/j.1745-6606.2000.tb00092.x · Cited by 169
Evolving Patterns of Organizational Beliefs in the Formation of Strategy
Frankwick, G. L., Ward, J. C., Hutt, M. D., & Reingen, P. H. (1994) · Journal of Marketing · doi:10.1177/002224299405800208 · Cited by 168
Corporate Risk Reporting: A Content Analysis of Narrative Risk Disclosures in Prospectuses
Deumes, R. (2008) · Journal of Business Communication · doi:10.1177/0021943607313992 · Cited by 167
Sex-Role Portrayals and the Gender of Nations
Milner, L. M., & Collins, J. M. (2000) · Journal of Advertising · doi:10.1080/00913367.2000.10673604 · Cited by 165
Paek, H., Kim, S., Hove, T. B., & Huh, J. Y. (2013) · Journal of Health Communication · doi:10.1080/10810730.2013.821560 · Cited by 165
How tourism activity shapes travel experience sharing: Tourist well-being and social context
Su, L.-J., Tang, B., & Nawijn, J. (2021) · Annals of Tourism Research · doi:10.1016/j.annals.2021.103316 · Cited by 148
The internet, information and empowerment
Harrison, T., Waite, K., & Hunter, G. L. (2006) · European Journal of Marketing · doi:10.1108/03090560610680961 · Cited by 165
From Invention to Innovation: Toward Developing an Integrated Innovation Model for Biotech Firms*
Khilji, S. E., Mroczkowski, T. F., & Bernstein, B. (2006) · Journal of Product Innovation Management · doi:10.1111/j.1540-5885.2006.00222.x · Cited by 164
Paek, H., Kim, K., & Hove, T. B. (2010) · Health Education Research · doi:10.1093/her/cyq063 · Cited by 164
Osei-Frimpong, K., Wilson, A. M., & Lemke, F. (2016) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.04.009 · Cited by 175
Avatars as information: Perception of consumers based on their avatars in virtual worlds
Bélisle, J., & Bodur, H. O. (2010) · Psychology and Marketing · doi:10.1002/mar.20354 · Cited by 171
Carter, C. R., & Ellram, L. M. (2003) · Journal of Supply Chain Management · doi:10.1111/j.1745-493x.2003.tb00152.x · Cited by 166
Cho, C.-H., & Cheon, H. J. (2005) · Journal of Advertising · doi:10.1080/00913367.2005.10639195 · Cited by 162
Pretesting Questionnaires: Content Analyses of Respondents' Concurrent Verbal Protocols
Bolton, R. N. (1993) · Marketing Science · doi:10.1287/mksc.12.3.280 · Cited by 161
Saeed, K. A., Grover, V., & Hwang, Y. (2005) · Journal of Management Information Systems · doi:10.1080/07421222.2003.11045835 · Cited by 159
Examination of Gender-role Portrayals in Television Advertising across Seven Countries
Paek, H., Nelson, M. R., & Vilela, A. M. (2010) · Sex Roles · doi:10.1007/s11199-010-9850-y · Cited by 175
Value co-creation among retailers and consumers: New insights into the furniture market
Andreu, L., Sánchez, I., & Mele, C. (2010) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2010.02.001 · Cited by 171
The effects of customer personality traits on the display of positive emotions
Tan, H., der Foo, M., & Kwek, M. (2004) · Academy of Management Journal · doi:10.2307/20159579 · Cited by 168
Mothersbaugh, D. L., Huhmann, B. A., & Franke, G. R. (2002) · Journal of Consumer Research · doi:10.1086/338211 · Cited by 161
Man to Man: A Content Analysis of Sole-Male Images in Male-Audience Magazines
Kolbe, R. H., & Albanese, P. J. (1996) · Journal of Advertising · doi:10.1080/00913367.1996.10673509 · Cited by 158
Racial Diversity and Firm Performance
Andrevski, G., Richard, O. C., Shaw, J. D., & Ferrier, W. J. (2011) · Journal of Management · doi:10.1177/0149206311424318 · Cited by 167
Antecedents and outcomes of perceived organizational support: a literature survey approach
Ahmed, I., & Nawaz, M. M. (2015) · Journal of Management Development · doi:10.1108/jmd-09-2013-0115 · Cited by 152
Disjunctions of Conjunctions, Cognitive Simplicity, and Consideration Sets
Hauser, J. R., Toubia, O., Evgeniou, T., Befurt, R., & Dzyabura, D. (2010) · Journal of Marketing Research · doi:10.1509/jmkr.47.3.485 · Cited by 154
A CIT investigation of servicescape failures and associated recovery strategies
Hoffman, K. D., Kelley, S. W., & Chung, B. C. (2003) · Journal of Services Marketing · doi:10.1108/08876040310482757 · Cited by 151
The State of Internet-Related Research in Communications, Marketing, and Advertising: 1994-2003
Cho, C.-H., & Khang, H. (2006) · Journal of Advertising · doi:10.2753/joa0091-3367350309 · Cited by 150
Using Message Strategy to Drive Consumer Behavioral Engagement on Social Media
Tafesse, W., & Wien, A. H. (2017) · Journal of Consumer Marketing · doi:10.1108/jcm-08-2016-1905 · Cited by 150
Information Technology, Network Structure, and Competitive Action
Chi, L., Ravichandran, T., & Andrevski, G. (2010) · Information Systems Research · doi:10.1287/isre.1100.0296 · Cited by 158
Examining Information Processing on the World Wide Web Using Think Aloud Protocols
Eveland, W. P., & Dunwoody, S. (2000) · Media Psychology · doi:10.1207/s1532785xmep0203_2 · Cited by 149
Schick, V., Rima, B., & Calabrese, S. K. (2009) · The Journal of Sex Research · doi:10.1080/00224490903308404 · Cited by 147
Gebauer, H., Arzt, A., Kohtamäki, M., Lamprecht, C., Parida, V., Witell, L., & Wortmann, F. (2020) · Industrial Marketing Management · doi:10.1016/j.indmarman.2020.10.006 · Cited by 147
Metadata
- Authors
- Perreault, W. D., & Leigh, L. E.
- Affiliations
- University of North Carolina at Chapel Hill (US)
- Birmingham–Southern College (US)
- Venue
- Journal of Marketing Research
- Publisher
- JSTOR
- Published
- 1989-05
- Volume
- 26
- Issue
- 2
- Pages
- 135
- ISSN
- 0022-2437 (ISSN-L)
- OpenAlex
- W2069585723
- Cited by
- 1,575
- References
- 42
- Citations by year
- 2026: 19 · 2025: 35 · 2024: 43 · 2023: 44 · 2022: 56 · 2021: 59 · 2020: 60 · 2019: 58 · 2018: 52 · 2017: 39 · 2016: 48 · 2015: 89 · 2014: 77 · 2013: 88 · 2012: 86
- FWCI
- 12.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Marketing Research (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- reliability statistics, nominal data, qualitative judgment, marketing activities, measurement validity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review