Field studies of psychologically targeted ads face threats to internal validity
Open accessAbstract
Proceedings of the National Academy of Sciences (PNAS), a peer reviewed journal of the National Academy of Sciences (NAS) - an authoritative source of high-impact, original research that broadly spans the biological, physical, and social sciences.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Because users were not randomly assigned to ads, the study may have introduced unintended variance in the user sets exposed to each ad.
“Because the assignment of users between ads was not random, of course, it may have generated unintended variance in the set of users exposed to each ad 30 .”
From Understanding, explaining, and utilizing medical artificial intelligence (2021) · cites this paper as [30]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Psychological targeting as an effective approach to digital mass persuasion
Matz, S. C., Kosinski, M., Nave, G., & Stillwell, D. J. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710966114 · Cited by 892 · see citing works
Gordon, B. R., Zettelmeyer, F., Bhargava, N., & Chapsky, D. (2019) · Marketing Science · doi:10.1287/mksc.2018.1135 · Cited by 287 · see citing works
Self-citationGhost Ads: Improving the Economics of Measuring Online Ad Effectiveness
Johnson, G. A., Lewis, R. A., & Nubbemeyer, E. (2017) · Journal of Marketing Research · doi:10.1509/jmr.15.0297 · Cited by 173 · see citing works
Self-citationGordon, B. R., Zettelmeyer, F., Bhargava, N., & Chapsky, D. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3033144 · Cited by 68 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Understanding, explaining, and utilizing medical artificial intelligence
Cadario, R., Longoni, C., & Morewedge, C. K. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01146-0 · Cited by 342 · see citing works
Algorithm-mediated social learning in online social networks
Brady, W. J., Jackson, J. C., Lindström, B., & Crockett, M. J. (2023) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2023.06.008 · Cited by 132 · see citing works
A Framework for the Study of Persuasion
Druckman, J. N. (2022) · Annual Review of Political Science · doi:10.1146/annurev-polisci-051120-110428 · Cited by 126 · see citing works
Quantifying the potential persuasive returns to political microtargeting
Tappin, B. M., Wittenberg, C., Hewitt, L. B., Berinsky, A. J., & Rand, D. G. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2216261120 · Cited by 83 · see citing works
- Authors
- Eckles, D., Gordon, B. R., & Johnson, G. A.
- Affiliations
- Massachusetts Institute of Technology (US)
- Northwestern University (US)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2018-05-18
- Volume
- 115
- Issue
- 23
- Pages
- E5254-E5255
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC6003331
- OpenAlex
- W2803619622
- Cited by
- 66
- References
- 4
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 7 · 2023: 11 · 2022: 9 · 2021: 13 · 2020: 11 · 2019: 5 · 2018: 2
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
- PsyArXiv (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Sociology and Political Science › Digital Marketing and Social Media
- Social Sciences › Business, Management and Accounting › Marketing › Consumer Market Behavior and Pricing
- Keywords
- targeted advertising, internal validity, field experiments, advertising effects, consumer behavior
- MeSH terms
- Advertising, Face, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
