Human Values in Software Engineering: Contrasting Case Studies of Practice
The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Abstract
The growing diffusion of software in society and its influence on people demands from its creators that their work carefully considers human values such as transparency, social responsibility, and equality. But how do software practitioners address human values in software engineering practice? We interviewed 31 software practitioners from two organizations, each having a strong values framework, with the aim to understand: (a) practitioners’ perceptions of human values and their role in software engineering; (b) practices that practitioners use to address human values in software; and (c) challenges they face during this process. We report our findings from two contrasting case organizations on how practitioners “engineer” values in their unique organizational settings. We found evidence that organizational culture significantly contributes to how values are addressed in software. We summarize recommendations from the practitioners to support proactive engineering of values-conscious software.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“As suggested by [186], these issues could be overcome by simple organisational mechanisms, rather than technical solutions.”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [186]
In the citing paper
“At present, a key barrier and opportunity is that AI developers have only limited awareness of existing solutions that support ethical AI development, with organisational barriers persisting [186].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [186]
In the citing paper
“Empirical research in software engineering has noted that human values can be embedded into software development by evolution, rather than revolution, of existing practices [186].”
From Operationalising ethics in artificial intelligence for healthcare: a framework for AI developers (2023) · cites this paper as [186]
Citations
Cites
Qualitative research and evaluation methods
Patton, M. Q. (1980) · Cited by 35,734
Braun, V., & Clarke, V. (2012) · APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. · doi:10.1037/13620-004 · Cited by 10,299
(2017) · Cited by 9,643
Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers
Baxter, P., & Jack, S. (2015) · The Qualitative Report · doi:10.46743/2160-3715/2008.1573 · Cited by 7,594
Are There Universal Aspects in the Structure and Contents of Human Values?
Schwartz, S. H. (1994) · Journal of Social Issues · doi:10.1111/j.1540-4560.1994.tb01196.x · Cited by 5,793
An overview of the Schwartz theory of basic values
Schwartz, S. H. (2012) · Online Readings in Psychology and Culture · doi:10.9707/2307-0919.1116 · Cited by 4,147
A Theory of Cultural Values and Some Implications for Work
Schwartz, S. H. (1999) · Applied Psychology · doi:10.1111/j.1464-0597.1999.tb00047.x · Cited by 3,102
Refining the theory of basic individual values.
Schwartz, S. H., Cieciuch, J., Vecchione, M., Davidov, E., Fischer, R., Beierlein, C., Ramos, A., Verkasalo, M., Lönnqvist, J.-E., Demirutku, K., et al. (2012) · Journal of Personality and Social Psychology · doi:10.1037/a0029393 · Cited by 2,962
Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing
Ferretti, L., Wymant, C., Kendall, M., Zhao, L., Nurtay, A., Abeler-Dörner, L., Parker, M., Bonsall, D., & Fraser, C. (2020) · Science · doi:10.1126/science.abb6936 · Cited by 2,512
Selecting Empirical Methods for Software Engineering Research
Easterbrook, S., Singer, J., Storey, M.-A., & Damian, D. (2008) · Guide to Advanced Empirical Software Engineering · doi:10.1007/978-1-84800-044-5_11 · Cited by 1,167
Research works printed but not linked: 6
- 10.1016/s0065–2601(08)60281-6 · DOI · Paper page
- Rokeach, 1973, The Nature of Human Values.
- Smith, Beliefs, practices, and personalities of software engineers: A survey in a large software company, 15
- Fowler, 2001, The agile manifesto, Softw. Develop., 9, 28
- Schwaber, 2011, The scrum guide, Scrum Alliance, 21
- Patton, 2002, Qualitative interviewing, 344
Cited by
Socio-Technical Grounded Theory for Software Engineering
Hoda, R. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3106280 · Cited by 204
“ues [44], [60], and the human-in-the-loop in order to”
BackgroundSelf-citationOperationalising ethics in artificial intelligence for healthcare: a framework for AI developers
Solanki, P., Grundy, J., & Hussain, W. (2023) · AI and Ethics · doi:10.1007/s43681-022-00195-z · Cited by 157
“Empirical research in software engineering has noted that human values can be embedded into software development by evolution, rather than revolution, of existing practices [186].”
BackgroundInfluentialSelf-citationAI Ethics Principles in Practice: Perspectives of Designers and Developers
Sanderson, C., Douglas, D., Lu, Q., Schleiger, E., Whittle, J., Lacey, J., Newnham, G., Hajkowicz, S., Robinson, C., & Hansen, D. (2023) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2023.3257303 · Cited by 99
“The challenge in translating abstract human values into software and technologies is well recognised [22], [23], [39].”
BackgroundSelf-citationThe Making of Responsible Innovation and Technology: An Overview and Framework
Li, W., Yiğitcanlar, T., Browne, W. N., & Nili, A. (2023) · Smart Cities · doi:10.3390/smartcities6040093 · Cited by 106
“Some studies suggested that embedding human values, including universal and culture-specific values, throughout the innovation process would contribute to achieving a better alignment between technological advancements and societal desirability and acceptability [31,32,71].”
BackgroundMethodDefining blockchain governance principles: A comprehensive framework
Liu, Y., Lu, Q., Yu, G., Paik, H.-Y., & Zhu, L. (2022) · Information Systems · doi:10.1016/j.is.2022.102090 · Cited by 112
Towards a roadmap on software engineering for responsible AI
Lu, Q., Zhu, L., Xu, X., Whittle, J., & Xing, Z. (2022) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3522664.3528607 · Cited by 105
“There has been recently emerging research on casting human values into requirements in software engineering and their operationalization [37, 38], including the extension of value-based design methods [39], extension of human factor research on productivity and usability into human values…”
BackgroundSelf-citationAmugongo, L. M., Kriebitz, A., Boch, A., & Lütge, C. (2025) · AI and Ethics · doi:10.1007/s43681-023-00331-3 · Cited by 68
“A study in software engineering has found that incorporating human values into software development can be achieved through an evolutionary approach, rather than a revolutionary one, by building upon existing practices [91].”
Human-Centered AI Design in Reality: A Study of Developer Companies’ Practices
Hartikainen, M., Väänänen, K., Lehtiö, A., Ala-Luopa, S., & Olsson, T. (2022) · Nordic Human-Computer Interaction Conference · doi:10.1145/3546155.3546677 · Cited by 48
AI and Ethics—Operationalizing Responsible AI
Zhu, L., Xu, X., Lu, Q., Governatori, G., & Whittle, J. (2021) · Springer eBooks · doi:10.1007/978-3-030-72188-6_2 · Cited by 88
Self-citationBerntzen, M. N., Stray, V. G., Moe, N. B., & Hoda, R. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10349-0 · Cited by 62
“These quality criteria are applicable for assessing the trustworthiness of research and are often used within software engineering (e.g., Russo 2021; Hussain et al. 2022).”
MethodSelf-citationOperationalizing Human Values in Software Engineering: A Survey
Shahin, M., Hussain, W., Nurwidyantoro, A., Perera, H., Shams, R. A., Grundy, J., & Whittle, J. (2022) · IEEE Access · doi:10.1109/access.2022.3190975 · Cited by 59
“Similar to other studies in software engineering [102], [112], our findings suggest that building values awareness in the team members, creating values-specific roles for team members, and aligning devel-VOLUME 0, 2022 oper motivations with values expectations from technology is likely to result in…”
BackgroundResultInfluentialSelf-citationHow Can Human Values Be Addressed in AgileMethods A Case Study on SAFe
Hussain, W., Shahin, M., Hoda, R., Whittle, J., Perera, H., Nurwidyantoro, A., Shams, R. A., & Oliver, G. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3140230 · Cited by 60
“Even when values are considered explicitly, organisations like the one in this study tend to consider themmainly in the early phases of a project, with values discussions taking placemainly during project selection and early requirements of design stages [93].”
BackgroundInfluentialSelf-citationNurwidyantoro, A., Shahin, M., Chaudron, M. R. V., Hussain, W., Shams, R. A., Perera, H., Oliver, G., & Whittle, J. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106731 · Cited by 39
Self-citationSupporting Developers in Addressing Human-Centric Issues in Mobile Apps
Khalajzadeh, H., Shahin, M., Obie, H. O., Agrawal, P., & Grundy, J. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3212329 · Cited by 39
“, the guiding principles of what people consider important in life [59]), and its relationship with technology [60], [61].”
BackgroundApplying Human Values Theory to Software Engineering Practice: Lessons and Implications
Ferrario, M. A., & Winter, E. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3170087 · Cited by 37
“Hussain et al. [9] report on the difficulty of measuring values in SE projects.”
BackgroundMethodInfluentialExploring human resource management intelligence practices using machine learning models
Indarapu, S. R. K., Vodithala, S., Kumar, N., Kiran, S., Reddy, S. N., & Dorthi, K. (2023) · The Journal of High Technology Management Research · doi:10.1016/j.hitech.2023.100466 · Cited by 27
Using ChatGPT to Generate Human-Value User Stories as Inspirational Triggers
Marczak-Czajka, A., & Cleland‐Huang, J. (2023) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew57809.2023.00016 · Cited by 29
“These two approaches align well with Parera’s findings [2] which showed that either approach was effective for developing requirements that mapped to human values.”
BackgroundResultAn HCAI Methodological Framework: Putting it Into Action to Enable Human-Centered AI
Xu, W., Gao, Z., & Dainoff, M. J. (2026) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3631590 · Cited by 97
“This disconnect creates collaboration difficulties, with nonAI experts reporting misunderstandings due to the lack of shared workflows and common language [157].”
Enabling Human-Centered AI: A Methodological Perspective
Xu, W., & Gao, Z. (2024) · IEEE International Conference on Human-Machine Systems (ICHMS) · doi:10.1109/ichms59971.2024.10555771 · Cited by 26
The Implications of ‘Soft’ Requirements
Sutcliffe, A., Sawyer, P., & Bencomo, N. (2022) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re54965.2022.00022 · Cited by 17
The Impact of Considering Human Values during Requirements Engineering ActivitiesPreprint
Perera, H., Hoda, R., Shams, R. A., Nurwidyantoro, A., Shahin, M., Hussain, W., & Whittle, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.15293 · Cited by 21
“This aligns with one of our previous findings [40], the inability to translate human values into features being one of the common challenges in operationalizing human values in SE.”
BackgroundResultSelf-citationReimann, M., & Diewald, M. (2024) · New Technology Work and Employment · doi:10.1111/ntwe.12325 · Cited by 12
Obstacle Analysis in Requirements Engineering: Retrospective and Emerging Challenges
Letier, E., & van Lamsweerde, A. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3534318 · Cited by 12
“This effort would complement other research on modelling human values, which are also relevant to addressing the AI alignment problem [52].”
With Great Power Comes Great Responsibility: The Role of Software Engineers
Betz, S., & Penzenstadler, B. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715112 · Cited by 15
Obie, H. O., Shahin, M., Grundy, J., Turhan, B., Li, L., Hussain, W., & Whittle, J. (2021) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec53868.2021.00063 · Cited by 9
Self-citationHuman Values Violations in Stack Overflow: An Exploratory Study
Krishtul, S., Shahin, M., Obie, H. O., Khalajzadeh, H., Gai, F., Nasab, A. R., & Grundy, J. (2022) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3530019.3530027 · Cited by 8
“[21] argue that the maturity levels of companies in addressing human values may very well depend on their awareness and overall organisational culture.”
BackgroundAI and Ethics -- Operationalising Responsible AIPreprint
Zhu, L., Xu, X., Lu, Q., Guido Governatori, A., & Whittle, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08867 · Cited by 7
“Recently, there has been emerging research in human values in software engineering and their operationalisation [20, 21], including:”
BackgroundMethodInfluentialSelf-citationPrivacy-by-design AIoT vision for intelligent urban environments
Ivanovska, M., Kreft, J., Štruc, V., & Perš, J. (2025) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2025.103586 · Cited by 12
Shams, R. A., Shahin, M., Oliver, G., Perera, H., Whittle, J., Nurwidyantoro, A., & Hussain, W. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111648 · Cited by 9
Self-citationHelmer, L., Martens, C., Wegener, D., Akila, M., Becker, D. S., & Abbas, S. (2024) · doi:10.1145/3643691.3648584 · Cited by 18
“In [Hussain et al. 2022] it is stated that human values such as privacy, transparency, integrity, social justice and diversity are becoming increasingly important in the development of software.”
BackgroundArchitecting and Engineering Value-Based Ecosystems
Pelliccione, P., Bühnová, B., Gottschalk, S., Weber, I. M., & Engels, G. (2023) · Springer eBooks · doi:10.1007/978-3-031-36847-9_3 · Cited by 8
Obie, H. O., Ukwella, J., Madampe, K., Grundy, J., & Shahin, M. (2023) · IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · doi:10.1109/asew60602.2023.00010 · Cited by 9
“companies do consider human values in their practices, but the maturity of this consideration varies widely, depending on practitioners’ awareness and organisational culture, and suggests that embedding values in technology can be achieved through an evolution of existing practices [12].”
BackgroundNaveed, H. (2023) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C) · doi:10.1109/models-c59198.2023.00040 · Cited by 7
“The ones that exist have the following key limitations: 1) Several human-centric requirements such as human values [7], [44], [45], human, societal and environmental well-being [46], and privacy [47] were not considered.”
BackgroundImpact of Organisational Culture on the Requirement Engineering Activities
Spichkova, M., Alsanoosy, T., & Harland, J. (2021) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re51729.2021.00053 · Cited by 5
Inclusive Model-Driven Engineering for Accessible Software
Bork, D., Klikovits, S., Michael, J., Netz, L., & Rumpe⋆, B. (2025) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1109/models67397.2025.00030 · Cited by 7
“We can find ongoing discussions and research in various areas, e.g., AI and diversity [11], bias in data science [12], and diversity in development teams [13], [14].”
Requirements on and Procurement of Explainable Algorithms—A Systematic Review of the Literature
Franke, U., Hallström, C. H., Artman, H., & Dexe, J. (2024) · Advances in intelligent systems and computing · doi:10.1007/978-3-031-66635-3_4 · Cited by 5
Ma, P., Wang, Z., Li, Z., Ji, Z., Sun, A., Rahmel, J., & Wang, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3722554 · Cited by 8
Value-Based Software Engineering: A Systematic Mapping Study
Salleh, N., Mendes, E., Mendes, F. F., Lekamlage, C. D., & Petersen, K. (2022) · e-Informatica Software Engineering Journal · doi:10.37190/e-inf230106 · Cited by 12
Human Value Requirements in AI Systems: Empirical Analysis of Amazon Alexa
Shams, R. A., Bano, M., Zowghi, D., Lu, Q., & Whittle, J. (2023) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew57809.2023.00030 · Cited by 3
“In recent years, some studies have focused on human values in SE research [8] [7] [10] [11] [12].”
BackgroundSelf-citationTeaching Ethics in HCI Courses: Guidelines for Teachers and Future Designers
Silveira, I. F. (2023) · Lecture notes in educational technology · doi:10.1007/978-981-99-7353-8_35 · Cited by 4
Investigating the Developer Experience of LGBTQIAPN+ People in Agile Teams
Wassouf, E. R., Fukuda, P., & Fontão, A. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis66351.2025.00010 · Cited by 9
Extracting software requirements-related information from online news using DomText-WMDS
Dewi, M. R., Raharjana, I. K., Siahaan, D., & Jannah, N. (2023) · JURNAL INFOTEL · doi:10.20895/infotel.v15i3.950 · Cited by 1
Venturing ChatGPT's lens to explore human values in software artifacts: a case study of mobile APIs
Mougouei, D., Rafi, S., Fahmideh, M., Mougouei, E., Khan, J. A., Dam, H. K., Nurwidyantoro, A., & Chaudron, M. R. V. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2478278 · Cited by 2
“…Engineering (VBRE) (Thew and Sutcli ff e 2018), Value Sensitive Design (VSD) (Friedman et al. 2013), Contin-ual Value(s) Assessment (CVA) (Perera et al. 2020), and the Human Values Dashboard (Nurwidyantoro et al. 2023), have been proposed to integrate human values into the software…”
InfluentialSelf-citationRazzaq, A., Botterweck, G., Lai, Q., & Buckley, J. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112648 · Cited by 3
Wang, H., Wu, C., Chen, C., Turhan, B., Chen, S., & Whittle, J. (2022) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis55304.2022.9793554 · Cited by 1
Self-citationAlwadani, R., & Baslyman, M. (2023) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-023-08104-0 · Cited by 1
“In software engineering, human values are defined as a set of life goals of civilized societies (e.g., accessibility, privacy, security, and social justice) or personal desires of users (e.g., happiness, achievement, social recognition) that are reflected in or through software applications [5].”
InfluentialThe Crucial Role of Personal Values on Well-Being and Resilience in the Software Industry
Yürüm, O. R., & Özcan‐Top, Ö. (2024) · IEEE Software · doi:10.1109/ms.2024.3395059 · Cited by 1
Improving Ethical Approaches to Video Game Microtransactions for Mobile Games
Rigoni, W. A., Olsen, J. L., Kirkham, R., & Mueller, F. (2023) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/3638380.3638422 · Cited by 4
“A willingness to engage is critical to the successful implementation of the ethical framework within the industry, yet this willingness is often difficult to achieve through cultural, educational, and self-moral compass factors [38].”
BackgroundHuman Values in Requirements Engineering : RE’21 Tutorial
Whittle, J., & Hussain, W. (2021) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re51729.2021.00081
Self-citationInvestigating Social Aspects in Software Development Teams — A Research Preview
Heß, A., Kohl, M., Scherr, S. A., Storck, S., & Hammer, F. (2023) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew57809.2023.00011
“Moreover, besides fulfilling relevant quality aspects like a positive user experience and related emotions, it can be observed that human values respectively social values and their integration in software are becoming more and more important as digitization progresses [8][9][10].”
BackgroundVîlcu, A., Nacu, I., Vîrlan, B., Herghiligiu, I. V., Lupăcescu, S., & Dragomir, A. O. (2022) · Buletinul Institutului Politehnic din Iaşi. Secţia Electrotehnică. Energetică. Electronică · doi:10.2478/bipie-2022-0020
Hoda, R. (2024) · Springer eBooks · doi:10.1007/978-3-031-60533-8_4 · Cited by 1
Self-citationStrategies to Embed Human Values in Mobile Apps: What do End-Users and Practitioners Think?
Shams, R. A., Shahin, M., Oliver, G., Whittle, J., Hussain, W., Perera, H., & Nurwidyantoro, A. (2025) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis66351.2025.00025
“…Bangladeshi female farmers’ values in agriculture apps, we argue that the following 5 strategies are close to some of the strategies reported in [34], [35]: “Strategy 1: Adding new features to extract and/or meet farmers’ values” , “Strategy 6: applying human-centered approaches to elicit…”
Self-citationA Vision for Value-Aligned AI-Driven Systems
Obie, H. O. (2025) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1109/aiware69974.2025.00023
Oruthotaarachchi, C. R., & Wijayanayake, J. I. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108046
Qayum, A., Qureshi, A. U. H., & Razzaq, A. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108091
Based on past experience: highlighting potential human value issues in domain modelling
Kaur, J. (2023) · eScholarship@McGill (McGill) · doi:10.82308/14932
Advancing Evidence-Based Social Sustainability in Software Engineering: A Research Roadmap
Ayoola, B., Lisboa, A., Santos, R. D. S., & Ralph, P. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3805556
Based on Past Experience: Highlighting Potential Human Value Issues in Domain Modelling
Kaur, J., & Mussbacher, G. (2023) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew57809.2023.00054
“conducted a case study [17] to understand changing software development practices by corporations to accommodate human values properly while designing software.”
Background
Metadata
- Authors
- Hussain, W., Perera, H., Whittle, J., Nurwidyantoro, A., Hoda, R., Shams, R. A., & Oliver, G.
- Affiliations
- Monash University (AU)
- ORCID
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2022-05-01
- Volume
- 48
- Issue
- 5
- Pages
- 1818-1833
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W3099082554
- Cited by
- 59
- References
- 59
- Citations by year
- 2026: 4 · 2025: 10 · 2024: 6 · 2023: 19 · 2022: 12
- FWCI
- 9.17
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Ethics in Business and Education
- Keywords
- human values, software professionals, organizational culture, transparency, social responsibility, equality, software ethics, practitioner perceptions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.