One size does not fit all: a grounded theory and online survey study of developer preferences for security warning types
Open accessThe DOI matches its Crossref record; 65 of the 67 references it cites were found in a registry or an index.
Abstract
A wide range of tools exist to assist developers in creating secure software. Many of these tools, such as static analysis engines or security checkers included in compilers, use warnings to communicate security issues to developers. The effectiveness of these tools relies on developers heeding these warnings, and there are many ways in which these warnings could be displayed. Johnson et al. [46] conducted qualitative research and found that warning presentation and integration are main issues. We built on Johnson et al.'s work and examined what developers want from security warnings, including what form they should take and how they should integrate into their workflow and work context. To this end, we conducted a Grounded Theory study with 14 professional software developers and 12 computer science students as well as a focus group with 7 academic researchers to gather qualitative insights. To back up the theory developed from the qualitative research, we ran a quantitative survey with 50 professional software developers. Our results show that there is significant heterogeneity amongst developers and that no one warning type is preferred over all others. The context in which the warnings are shown is also highly relevant, indicating that it is likely to be beneficial if IDEs and other development tools become more flexible in their warning interactions with developers. Based on our findings, we provide concrete recommendations for both future research as well as how IDEs and other security tools can improve their interaction with developers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In order to verify that participants had programming knowledge, we asked a brief prescreening question before proceeding with the study that focused on participants' ability to read and interpret a for-loop-which has been used in other user studies [7].”
From Do Users Write More Insecure Code with AI Assistants? (2023) · cites this paper as [7]
Citations
Cites
The Discovery of Grounded Theory: Strategies for Qualitative Research.
Wagner, H. R., Glaser, B. G., & Strauss, A. L. (1968) · Social Forces · doi:10.2307/2575405 · Cited by 46,361
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · cited 1×
“The coding schemes were compared, and inter-coder agreement was calculated using Cohen’s kappa coefficient (κ) [24].”
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449
(2006) · Cited by 18,730
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
Basics of Qualitative Research
Ozanne, J. L., Strauss, A., & Corbin, J. (1992) · Journal of Marketing Research · doi:10.2307/3172751 · Cited by 17,850 · cited 2×
“As suggested by qualitative research experts [25], we present an explanatory theory that is grounded in the data collected from our participants:”
(2014) · Cited by 15,747
The Research Act: A Theoretical Introduction to Sociological Methods
Fox, W. S., & Denzin, N. K. (1979) · Contemporary Sociology · doi:10.2307/2065439 · Cited by 6,284
Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013) · Sociological Methods & Research · doi:10.1177/0049124113500475 · Cited by 2,974 · cited 1×
“In order to obtain the inter-coder agreement, we followed the approach of [18].”
The SAGE Handbook of Grounded Theory
Bryant, A., & Charmaz, K. (2007) · SAGE · doi:10.4135/9781848607941 · Cited by 2,798 · cited 3×
“As suggested by Charmaz [19], we used the idea of theoretical sorting to develop a diagram out of our memos, themes and categories offering an initial analytic frame (see supplementary material).”
Sampling in qualitative research. Purposeful and theoretical sampling: Merging or clear boundaries?
Coyne, I. T. (1997) · Journal of Advanced Nursing · doi:10.1046/j.1365-2648.1997.t01-25-00999.x · Cited by 2,402 · cited 3×
“The theoretical sampling approach [26] was used to recruit the participants.”
The Quality of Qualitative Research
Seale, C. (1999) · SAGE Publications Ltd eBooks · doi:10.4135/9780857020093 · Cited by 2,012
Quality in Qualitative Research
Seale, C. (1999) · Qualitative Inquiry · doi:10.1177/107780049900500402 · Cited by 1,743 · cited 2×
“This was done in order to ensure data reliability and validity by using multiple datacollection methods (methodological triangulation) [28, 29, 39, 62].”
Android permissions: user attention, comprehension, and behavior
Felt, A. P., Ha, E., Egelman, S., Haney, A., Chin, E., & Wagner, D. (2012) · Proceedings of the Eighth Symposium on Usable Privacy and Security · doi:10.1145/2335356.2335360 · Cited by 1,022 · cited 1×
“This research shows that end users often struggle with security warnings, but that taking human factors into account can improve adherence to these warnings [6, 7, 13–15, 17, 31, 33–35, 58, 64–66].”
Response interpolation and scale sensitivity
Finstad, K. (2010) · Journal of Usability Studies archive · doi:10.5555/2835434.2835437 · Cited by 989
POWELL, R. A., & SINGLE, H. M. (1996) · International Journal for Quality in Health Care · doi:10.1093/intqhc/8.5.499 · Cited by 701
Maintaining mental models: a study of developer work habits
LaToza, T. D., Venolia, G., & DeLine, R. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134355 · Cited by 628
You've been warned: an empirical study of the effectiveness of web browser phishing warnings
Egelman, S., Cranor, L. F., & Hong, J. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357219 · Cited by 607 · cited 1×
“This research shows that end users often struggle with security warnings, but that taking human factors into account can improve adherence to these warnings [6, 7, 13–15, 17, 31, 33–35, 58, 64–66].”
Denzin’s Paradigm Shift: Revisiting Triangulation in Qualitative Research
Fusch, P., Fusch, G. E., & Ness, L. R. (2018) · Journal of Social Change · doi:10.5590/josc.2018.10.1.02 · Cited by 551 · cited 2×
“…triangulation via the application of multiple external analysis methods, such as having more than one researcher exploring the phenomenon ( investigator triangulation ) and using multiple data-collection methods as interviews and focus groups ( methodological triangulation ) [28, 29, 39, 62].”
Why eve and mallory love android: an analysis of android SSL (in)security
Fahl, S., Harbach, M., Muders, T., Baumgärtner, L., Freisleben, B., & Smith, M. (2012) · Proceedings of the 2012 ACM conference on Computer and communications security · doi:10.1145/2382196.2382205 · Cited by 490 · cited 1×
“Examples of severe security issues in software development are the use of deprecated security parameters or functions for end user password storage in a database endangering a large amount of sensitive user data [4, 52–54] or the misuse of TLS enabling man-in-the middle attacks [32].”
Self-citationThe Research Act: A Theoretical Introduction to Sociological Methods.
Bylund, H. B., & Denzin, N. K. (1972) · Contemporary Sociology · doi:10.2307/2063497 · Cited by 490
Alice in warningland: a large-scale field study of browser security warning effectiveness
Akhawe, D., & Felt, A. P. (2013) · USENIX Security Symposium · Cited by 483
Mora, M., Gelman, O., Steenkamp, A. L., & Raisinghani, M. S. (2011) · IGI Global eBooks · doi:10.4018/978-1-4666-0179-6 · Cited by 397 · cited 1×
“Since the focus group was used as a complimentary data acquisition mechanism, we evaluated it accordingly to the interviews by applying the coding techniques of GT methodology [19, 50, 51, 56].”
Research-based guidelines for warning design and evaluation
Wogalter, M. S., Conzola, V. C., & Smith-Jackson, T. L. (2002) · Applied Ergonomics · doi:10.1016/s0003-6870(02)00009-1 · Cited by 397 · cited 2×
“[67], who found previously that the location of warnings is crucial for warning design.”
Are Attention Check Questions a Threat to Scale Validity?
Kung, F. Y. H., Kwok, N., & Brown, D. J. (2018) · Applied Psychology · doi:10.1111/apps.12108 · Cited by 380 · cited 1×
“standard practice, we also added an attention check question [48] to our survey to filter out careless respondents.”
Crying Wolf: An Empirical Study of SSL Warning Effectiveness
Sunshine, J., Egelman, S., Almuhimedi, H., Atri, N., & Cranor, L. F. (2018) · KiltHub Repository · doi:10.1184/r1/6621806 · Cited by 331
Warnings and Risk Communication
(1999) · doi:10.1201/9780203983836 · Cited by 307
Predicting human interruptibility with sensors: a Wizard of Oz feasibility study
Hudson, S., Fogarty, J., Atkeson, C., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J., & Yang, J. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642657 · Cited by 295 · cited 1×
“Hudson et al. [44] presented promising results by using sensors (audio and video recording) to form statistical models to predict the interruptibil-ity of users in order to assess when a task can be interrupted.”
Why don’t software developers use static analysis tools to find bugs
Johnson, B., Song, Y., Murphy-Hill, E. R., & Bowdidge, R. W. (2013) · Cited by 268
I've got 99 problems, but vibration ain't one: a survey of smartphone users' concerns
Felt, A. P., Egelman, S., & Wagner, D. (2012) · Proceedings of the second ACM workshop on Security and privacy in smartphones and mobile devices · doi:10.1145/2381934.2381943 · Cited by 260
Proceedings of Graphics Interface 2006
Gutwin, C. A., & Mann, S. (2006) · Cited by 226
Bridging the Gap in Computer Security Warnings: A Mental Model Approach
Bravo-Lillo, C., Cranor, L. F., Downs, J., & Komanduri, S. (2011) · IEEE Security & Privacy Magazine · doi:10.1109/msp.2010.198 · Cited by 224
Developers are Not the Enemy!: The Need for Usable Security APIs
Green, M., & Smith, M. (2016) · IEEE Security & Privacy · doi:10.1109/msp.2016.111 · Cited by 221 · cited 1×
“Consequently, improving security warnings for developers is a desirable research goal [43].”
Self-citationWhat developers want and need from program analysis: an empirical study
Christakis, M., & Bird, C. (2016) · Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/2970276.2970347 · Cited by 219 · cited 1×
“For instance, Christakis and Bird [23] surveyed 375 developers from Microsoft to investigate their perceptions of automatic program analysis.”
The Sage Handbook of Grounded Theory.
Gilgun, J. F. (2009) · Journal of Family Theory & Review · doi:10.1111/j.1756-2589.2009.00013.x · Cited by 207
(2012) · IGI Global eBooks · doi:10.4018/978-1-46660-179-6 · Cited by 201
Comparing the Usability of Cryptographic APIs
Acar, Y., Backes, M., Fahl, S., Garfinkel, S., Kim, D., Mazurek, M. L., & Stransky, C. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.52 · Cited by 198 · cited 1×
“Examples of severe security issues in software development are the use of deprecated security parameters or functions for end user password storage in a database endangering a large amount of sensitive user data [4, 52–54] or the misuse of TLS enabling man-in-the middle attacks [32].”
Acar, Y., Fahl, S., & Mazurek, M. L. (2016) · IEEE Cybersecurity Development (SecDev) · doi:10.1109/secdev.2016.013 · Cited by 173 · cited 1×
“The recruitment of a reasonable number of professional software developers for quantitative research studies is challenging [5, 9, 49, 52].”
Your attention please: designing security-decision UIs to make genuine risks harder to ignore
Bravo-Lillo, C., Komanduri, S., Cranor, L. F., Reeder, R. W., Sleeper, M., Downs, J., & Schechter, S. (2013) · Proceedings of the Ninth Symposium on Usable Privacy and Security · doi:10.1145/2501604.2501610 · Cited by 145 · cited 1×
“This research shows that end users often struggle with security warnings, but that taking human factors into account can improve adherence to these warnings [6, 7, 13–15, 17, 31, 33–35, 58, 64–66].”
Improving SSL Warnings: Comprehension and Adherence
Felt, A. P., Ainslie, A., Reeder, R. W., Consolvo, S., Thyagaraja, S., Bettes, A., Harris, H., & Grimes, J. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702442 · Cited by 139 · cited 1×
“This research shows that end users often struggle with security warnings, but that taking human factors into account can improve adherence to these warnings [6, 7, 13–15, 17, 31, 33–35, 58, 64–66].”
Statistical Methods for Rates and Proportions.
Marshall, T. F. D. C., & Fleiss, J. L. (1976) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2988144 · Cited by 132
Measuring the effects of interruptions on task performance in the user interface
Bailey, B. P., Konstan, J. A., & Carlis, J. V. (2002) · SMC 2000 Conference Proceedings. 2000 IEEE International Conference on Systems, Man and Cybernetics. 'Cybernetics Evolving to Systems, Humans, Organizations, and their Complex Interactions' (Cat. No.00CH37166) · doi:10.1109/icsmc.2000.885940 · Cited by 127 · cited 1×
“Furthermore, Bailey et al. [10] found that participants performed worse on interrupted than on non-interrupted tasks.”
Gathering evidence: use of visual security cues in web browsers
Whalen, T., & Inkpen, K. M. (2005) · Cited by 123
An Experience Sampling Study of User Reactions to Browser Warnings in the Field
Reeder, R. W., Felt, A. P., Consolvo, S., Malkin, N., Thompson, C., & Egelman, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174086 · Cited by 114
CogniCrypt: Supporting developers in using cryptography
Kruger, S., Nadi, S., Reif, M., Ali, K., Mezini, M., Bodden, E., Gopfert, F., Gunther, F., Weinert, C., Demmler, D., et al. (2017) · 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115707 · Cited by 109 · cited 1×
“, [12, 47, 55]) and there is already work underway looking at these different types of systems.”
'Think secure from the beginning': A Survey with Software Developers
Assal, H., & Chiasson, S. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300519 · Cited by 104 · cited 2×
“The recruitment of a reasonable number of professional software developers for quantitative research studies is challenging [5, 9, 49, 52].”
Why Do Developers Get Password Storage Wrong?: A Qualitative Usability Study
Naiakshina, A., Danilova, A., Tiefenau, C., Herzog, M., Dechand, S., & Smith, M. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134082 · Cited by 96 · cited 1×
“This observation was also found in a password storage study conducted with computer science students by Naiakshina et al. [53].”
Self-citationDo Developers Read Compiler Error Messages?
Barik, T., Smith, J., Lubick, K., Holmes, E., Feng, J., Murphy-Hill, E., & Parnin, C. (2017) · 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2017.59 · Cited by 86 · cited 3×
“[12] conducted an eye-tracking study with student developers to investigate Java compiler error messages.”
A Stitch in Time: Supporting Android Developers in WritingSecure Code
Nguyen, D. C., Wermke, D., Acar, Y., Backes, M., Weir, C., & Fahl, S. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3133977 · Cited by 86 · cited 2×
“[55] proposed the plugin FixDroid, which offered warnings and quick fix dialogues for security issues in the Android Studio IDE, and found that developers approved FixDroid, which helped produce more secure code.”
Wurster, G., & van Oorschot, P. C. (2008) · New Security Paradigms Workshop · doi:10.1145/1595676.1595691 · Cited by 80 · cited 1×
“Wurster and van Oorschot [68] have suggested handling security warnings as blocking errors to enhance security.”
Impact of interruption style on end-user debugging
Robertson, T. J., Prabhakararao, S., Burnett, M., Cook, C., Ruthruff, J. R., Beckwith, L., & Phalgune, A. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985729 · Cited by 71 · cited 2×
“They found that negotiated-style interruptions were more accepted than immediate reactions [60] but can differ in their intensity.”
Harder to Ignore? Revisiting Pop-Up Fatigue and Approaches to Prevent It
Bravo-Lillo, C., Cranor, L. F., Komanduri, S., Schechter, S., & Sleeper, M. (2014) · Symposium On Usable Privacy and Security · Cited by 63
Naiakshina, A., Danilova, A., Gerlitz, E., von Zezschwitz, E., & Smith, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300370 · Cited by 62 · cited 1×
“The recruitment of a reasonable number of professional software developers for quantitative research studies is challenging [5, 9, 49, 52].”
Self-citationHow Developers Diagnose Potential Security Vulnerabilities with a Static Analysis Tool
Smith, J., Johnson, B., Murphy-Hill, E. R., Chu, B., & Lipford, H. R. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2810116 · Cited by 58 · cited 1×
“[63] conducted an exploratory study with 10 novice and experienced software developers to explore how they resolve security defects while using static analysis tools.”
Your reputation precedes you: history, reputation, and the chrome malware warning
Almuhimedi, H., Felt, A. P., Reeder, R. W., & Consolvo, S. (2014) · Symposium On Usable Privacy and Security · Cited by 50
Buck, R. W., Khan, M. M. H., Fagan, M. E., & Coman, E. (2017) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2017.1314612 · Cited by 47
How should compilers explain problems to developers?
Barik, T., Ford, D., Murphy-Hill, E., & Parnin, C. (2018) · Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3236024.3236040 · Cited by 41 · cited 2×
“[11] investigated compiler errors and how developers understood error messages.”
Gorski, P. L., Iacono, L. L., Wermke, D., Stransky, C., Möller, S., Acar, Y., & Fahl, S. (2018) · Publication Server of Bonn-Rhein-Sieg University of Applied Sciences (Bonn-Rhein-Sieg University of Applied Sciences) · Cited by 39
A cross-tool communication study on program analysis tool notifications
Johnson, B., Pandita, R., Smith, J., Ford, D., Elder, S., Murphy-Hill, E. R., Heckman, S. S., & Sadowski, C. (2016) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2950290.2950304 · Cited by 37 · cited 1×
“[45] conducted a qualitative study, in which 26 participants interacted with warnings from FindBugs, the Eclipse Compiler, and EclEmma.”
Deception Task Design in Developer Password Studies: Exploring a Student Sample
Naiakshina, A., Danilova, A., Tiefenau, C., & Smith, M. (2018) · Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) · Cited by 30
Self-citationHabituation effects in computer security warning
Amran, A., Zaaba, Z. F., & Singh, M. M. (2018) · Information Security Journal A Global Perspective · doi:10.1080/19393555.2018.1505008 · Cited by 21 · cited 1×
“, used to notify users about potential risks of their actions and security issues [8].”
Habituation effects in computer security warning
Amran, A., Zaaba, Z. F., & Singh, M. M. (2018) · Information Security Journal A Global Perspective · doi:10.1080/19393555.2018.1448492 · Cited by 17
A Week to Remember: The Impact of Browser Warning Storage Policies
Weinberger, J., & Felt, A. P. (2016) · Symposium On Usable Privacy and Security · Cited by 15
Impact of high-intensity negotiated-style interruptions on end-user debugging
Robertson, T. J., Lawrance, J., & Burnett, M. (2005) · Journal of Visual Languages & Computing · doi:10.1016/j.jvlc.2005.09.002 · Cited by 5 · cited 2×
“found that high-intensity negotiated-style interruptions had a similar effect as immediate-style interruptions [59].”
Research works printed but not linked: 8
- [n.d.]. Amazon Mechanical Turk (MTurk). https://www.mturk.com/ Accessed: August 2019.
- [n.d.]. Qualtrics. https://www.qualtrics.com Accessed: August 2019.
- [n.d.]. XING. https://www.xing.com/ Accessed: August 2019.
- NK Denzin. 1970. The Research Act in Sociology (London, Croom Helm). Denzin The Research Act in Sociology 1970 (1970).
- Don A Dillman. 2011. Mail and Internet surveys: The tailored design method-2007 Update with new Internet, visual, and mixed-mode guide. John Wiley & Sons.
- 10.5555/2486788.2486877 · doi:10.5555/2486788.2486877
- 10.5555/3155562.3155681 · doi:10.5555/3155562.3155681
- David L Morgan. 1996. Focus groups. Annual review of sociology 22, 1 (1996), 129--152.
Cited by
Do Users Write More Insecure Code with AI Assistants?
Perry, N., Srivastava, M., Kumar, D., & Boneh, D. (2023) · Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3576915.3623157 · Cited by 259
“…the entry to the database • Incorrect if it does not add the entry to the database • Secure if there are no security mistakes • Insecure if it is vulnerable to SQL injections • Unknown if the security of the answer cannot be determined • N/A If there is no answer, the answer is incomplete, or it is”
Do you Really Code? Designing and Evaluating Screening Questions for Online Surveys with Programmers
Danilova, A., Naiakshina, A., Horstmann, S., & Smith, M. (2021) · 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) · doi:10.1109/icse43902.2021.00057 · Cited by 62
Self-citationBeyond Bot Detection: Combating Fraudulent Online Survey Takers✱
Zhang, Z., Zhu, S., Mink, J., Xiong, A., Song, L., & Wang, G. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512230 · Cited by 84
“Finally, to identify eligible participants, screening questions are often used before the survey [1, 5, 7, 14, 15].”
Background(2022) · doi:10.1145/3491102.3501957 · Cited by 47
Ortloff, A.-M., Fassl, M., Ponticello, A., Martius, F., Mertens, A., Krombholz, K., & Smith, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580766 · Cited by 65
Self-citation
Metadata
- Authors
- Danilova, A., Naiakshina, A., & Smith, M.
- Affiliations
- University of Bonn (DE)
- Venue
- Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2020-06-27
- Pages
- 136-148
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W3090957174
- Cited by
- 25
- References
- 67
- Citations by year
- 2026: 3 · 2025: 1 · 2024: 2 · 2023: 6 · 2022: 6 · 2021: 7
- FWCI
- 3.79
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) (publishedVersion, cc-by)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Funders
- ERC (678341)
- Topics
- Physical Sciences › Computer Science › Information Systems › Web Application Security Vulnerabilities
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- security warnings, developer preferences, integrated development environment, static analysis, security tools, developer workflows, software security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.