Quantifying Phishing Susceptibility for Detection and Behavior Decisions
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Objective: We use signal detection theory to measure vulnerability to phishing attacks, including variation in performance across task conditions. Background: Phishing attacks are difficult to prevent with technology alone, as long as technology is operated by people. Those responsible for managing security risks must understand user decision making in order to create and evaluate potential solutions. Method: Using a scenario-based online task, we performed two experiments comparing performance on two tasks: detection, deciding whether an e-mail is phishing, and behavior, deciding what to do with an e-mail. In Experiment 1, we manipulated the order of the tasks and notification of the phishing base rate. In Experiment 2, we varied which task participants performed. Results: In both experiments, despite exhibiting cautious behavior, participants’ limited detection ability left them vulnerable to phishing attacks. Greater sensitivity was positively correlated with confidence. Greater willingness to treat e-mails as legitimate was negatively correlated with perceived consequences from their actions and positively correlated with confidence. These patterns were robust across experimental conditions. Conclusion: Phishing-related decisions are sensitive to individuals’ detection ability, response bias, confidence, and perception of consequences. Performance differs when people evaluate messages or respond to them but not when their task varies in other ways. Application: Based on these results, potential interventions include providing users with feedback on their abilities and information about the consequences of phishing, perhaps targeting those with the worst performance. Signal detection methods offer system operators quantitative assessments of the impacts of interventions and their residual vulnerability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · cited 1×
“…of the general U.S. population, they are more varied than convenience samples like university students (Crump, McDonnell & Gureckis, 2013; Mason & Suri, 2012). mTurk studies often recruit some participants who click through tasks without performing them or perform multiple tasks…”
BackgroundEvaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617
Dhamija, R., Tygar, J. D., & Hearst, M. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124861 · Cited by 1,376
Speed-accuracy tradeoff and information processing dynamics
Wickelgren, W. A. (1977) · Acta Psychologica · doi:10.1016/0001-6918(77)90012-9 · Cited by 1,304 · cited 1×
“Wickelgren (1977) shows how, even when payoffs are clear, people may lack the feedback needed to estimate how well they are achieving their desired trade-offs.”
BackgroundMacmillan, N. A., & Creelman, C. D. (2004) · doi:10.4324/9781410611147 · Cited by 1,269
Sheng, S., Holbrook, M., Kumaraguru, P., Cranor, L. F., & Downs, J. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753383 · Cited by 620
So long, and no thanks for the externalities: the rational rejection of security advice by users
Herley, C. (2009) · Proceedings of the 2009 workshop on New security paradigms workshop · doi:10.1145/1719030.1719050 · Cited by 524
Blase, K. A., Van Dyke, M., Fixsen, D. L., & Bailey, F. W. (2012) · Handbook of Implementation Science for Psychology in Education · doi:10.1017/cbo9781139013949.004 · Cited by 395
Lichtenstein, S., & Fischhoff, B. (1980) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(80)90052-5 · Cited by 375 · cited 2×
“…strongly related to d’ for the detection task, more weakly for the behavior task – consistent with the common result that confidence is positively, but imperfectly, correlated with knowledge (Fischhoff & MacGregor, 1986; Lichtenstein & Fischhoff, 1980; Moore & Healy, 2008; Parker & Stone, 2014).”
BackgroundResultSelf-citation
Cited by
Human Cognition Through the Lens of Social Engineering Cyberattacks
Montañez, R., Golob, E., & Xu, S. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01755 · Cited by 106
“Fourth, Canfield et al. [2016] find that individuals’ inability to detect phishing messages increases their susceptibility, regardless of their cautionary behavior.”
BackgroundKleitman, S., Law, M. K. H., & Kay, J. (2018) · PLOS ONE · doi:10.1371/journal.pone.0205089 · Cited by 51
Chen, J., Mishler, S., & Hu, B. (2021) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3051137 · Cited by 40
Phishing with Malicious QR Codes
Sharevski, F., Devine, A., Pieroni, E., & Jachim, P. (2022) · Proceedings of the 2022 European Symposium on Usable Security · doi:10.1145/3549015.3554172 · Cited by 39
“People’s vigilance, or the ability to detect anomalies for a sustained period, has been poor when it comes to phishing, so it is hardly probable for a positive change to happen when dealing with malicious QR codes [11, 72].”
Background
Metadata
- Authors
- Canfield, C. I., Fischhoff, B., & Davis, A.
- Venue
- Human Factors: The Journal of the Human Factors and Ergonomics Society
- Publisher
- SAGE Publications
- Published
- 2016-09-27
- Volume
- 58
- Issue
- 8
- Pages
- 1158-1172
- ISSN
- 0018-7208 (ISSN-L)
- 1547-8181
- OpenAlex
- W2510870740
- Cited by
- 129
- References
- 12
- FWCI
- 19.32
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Quantifying Phishing Susceptibility for Detection and Behavior Decisions. https://app.cochat.ai/papers/10.1177/0018720816665025
- The paper
- Canfield, C. I., Fischhoff, B., & Davis, A. (2016). Quantifying Phishing Susceptibility for Detection and Behavior Decisions. Human Factors: The Journal of the Human Factors and Ergonomics Society. https://doi.org/10.1177/0018720816665025
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.