The DOI matches its Crossref record; 55 of the 57 references it cites were found in a registry or an index.
A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook
Abstract
Observational methods often fail to accurately recover the treatment effects generated from randomized advertising experiments on Facebook.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Large-scale studies suggest that most of the time, due to methodological issues, the efficiency of targeting is largely overstated (Gordon et al., 2019;Lewis and Reiley, 2014), and that, overall,”
From How online advertising targets consumers: The uses of categories and algorithmic tools by audience planners (2024) · cites this paper as Gordon et al., 2019
In the citing paper
“computer generated categories and optimization are not just overstated (Aral, 2020;Gordon et al., 2019), they also neither match the power of demographics as standards, nor the idiosyncratic meaningfulness of custom audiences.”
From How online advertising targets consumers: The uses of categories and algorithmic tools by audience planners (2024) · cites this paper as Gordon et al., 2019
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597
Rosenbaum, P. R., & Rubin, D. B. (1985) · The American Statistician · doi:10.2307/2683903 · Cited by 6,472
SOME PRACTICAL GUIDANCE FOR THE IMPLEMENTATION OF PROPENSITY SCORE MATCHING
Caliendo, M., & Kopeinig, S. (2008) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2007.00527.x · Cited by 6,238
Matching Methods for Causal Inference: A Review and a Look Forward
Stuart, E. A. (2010) · Statistical Science · doi:10.1214/09-sts313 · Cited by 5,478
Recent Developments in the Econometrics of Program Evaluation
Imbens, G. W., & Wooldridge, J. M. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.1.5 · Cited by 4,926
Propensity Score-Matching Methods for Nonexperimental Causal Studies
Dehejia, R., & Wahba, S. (2002) · The Review of Economics and Statistics · doi:10.1162/003465302317331982 · Cited by 4,864
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Identification and estimation of local average treatment effects
Imbens, G. W., & Angrist, J. D. (1994) · Econometrica · doi:10.2307/2951620 · Cited by 4,101
Flagged first, then by citations
2 of the 57 references couldn’t be matched to a known paper.
Research works printed but not linked: 10
- Abraham M, 2008, Harvard Bus. Rev., 86, 28
- Berkovich P, 2016, Nielsen J. Measurement
- Bickel P, 1997, Statistica Sinicia, 7, 1
- Eckles D, Bakshy E (2017) Bias and high-dimensional adjustment in observational studies of Peer effects. Working paper, MIT Sloan School of Management, Cambridge, MA.
- Ferraro PJ, 2017, J. Assoc. Environ. Resource Econom., 4, 281
- LaLonde RJ, 1986, Amer. Econom. Rev., 76, 604
- Oster E, 2017, J. Bus. Econom. Statist.
- Pham TT, Shen Y (2017) A deep causal inference approach to measuring the effects of forming group loans in online non-profit microfinance platform. Working paper, Stanford Graduate School of Business, Stanford, CA. Available at https://arxiv.org/pdf/1706.02795.pdf. · arXiv
- Rosenbaum P, 1983, J. Royal Statist. Soc. Ser. B, 45, 212
- Tibshirani R, 1996, J. Royal Statist. Soc., 58, 267
Cited by
Sorted by citations
Items 201–287 of 287
The Statistics behind Online Controlled Experiments
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.023
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.015
Twyman’s Law and Experimentation Trustworthiness
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.005
Advanced Topics for Analyzing Experiments
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.022
Sample Ratio Mismatch and Other Trust-Related Guardrail Metrics
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.027
Complementary and Alternative Techniques to Controlled Experiments
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.013
Metrics for Experimentation and the Overall Evaluation Criterion
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.010
Experimentation Platform and Culture
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.006
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.025
Institutional Memory and Meta-Analysis
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.011
Measuring Long-Term Treatment Effects
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.029
Variance Estimation and Improved Sensitivity: Pitfalls and Solutions
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.024
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.019
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.017
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.007
Advanced Topics for Building an Experimentation Platform
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.016
Triggering for Improved Sensitivity
Kohavi, R., Tang, D., & XU, Y. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108653985.026
USING THE ADVERTISING MARKET RESEARCH TO CREATE EFFECTIVE ADVERTISING
Князєва, О., Muntian, I., & Значек, Р. Р. (2021) · Intellect XXІ · doi:10.32782/2415-8801/2021-3.4
Fair Effect Attribution in Parallel Online Experiments
Buchholz, A., Bellini, V., Benedetto, G. D., Stein, Y., Ruffini, M., & Moerchen, F. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3487553.3524211
A Field Experiment Using Social Media Ads to Encourage COVID-19 VaccinationPreprint
Çat, O., Chang, J., Hlatky, R., Li, H., & Nielson, D. (2022) · Research Square · doi:10.21203/rs.3.rs-2319578/v1
Nonparametric Bandits Leveraging Informational Externalities to Learn the Demand CurvePreprint
Weaver, I. C., & Kumar, V. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4263133
Which Firms Gain from Digital Advertising? Evidence from a Field ExperimentPreprint
Dai, D., Kim, H., & Luca, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4349557
Liang, Y., Chen, X., Han, S., Zhang, J., & Chen, Y. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4482611
Daga, G., Kossuth, L., Boruchowicz, C., & Bóo, F. L. (2023) · Research Square · doi:10.21203/rs.3.rs-3203862/v1
Sokolovskiy, K., Zueva, V., & Kolosova, O. (2023) · Information Development · doi:10.1177/02666669231200455
The Role of Repugnance in Markets: How the Jared Fogle Scandal Affected Patronage of SubwayPreprint
Cawley, J., Eddelbuettel, J. C. P., Cunningham, S., Eisenberg, M. D., Mathios, A., & Avery, R. J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4603116
Ward, J., Wenger, J. B., Cabreros, I., Schwam, D., Bond, C. A., & Absher, S. (2023) · RAND Corporation eBooks · doi:10.7249/rra2108-1
Measuring Heterogeneity in TV Advertising Elasticities: Evidence from 135 BrandsPreprint
Knight, S., Hsieh, T.-Y., & Bart, Y. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4638083
Liang, Y., Chen, X., Han, S., Zhang, J., & Chen, Y. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4641827
A Review –of Sales Encourage Online Consumer selected FMCG Product
Srivastava, S., & Nalla, B. (2023) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem27238
Digitization and Community ParticipationPreprint
Pocchiari, M., & Roos, J. M. T. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4671557
Wanshu, N., Huang, L., & Li, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4845707
The Role of Digital Ad Clutter in Ad Viewership, Memory, and Conversion OutcomesPreprint
Melessa, S., & Larsen, P. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4872995
The role of repugnance in markets: How the Jared Fogle scandal affected patronage of subway
Cawley, J., Eddelbuettel, J. C. P., Cunningham, S., Eisenberg, M. D., Mathios, A., & Avery, R. J. (2024) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2024.106848
Causal inference under limited outcome observability: A case study with Pinterest Conversion Lift
Kang, M. K. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3715073.3715078
Decomposing Treatment Effects Using a Latent Themes ApproachPreprint
Boughanmi, K., Iyengar, R., & Park, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5181192
Model of an information system for analysis and evaluation of advertising in social networks
Роїк, О. О. (2025) · Vìsnik Nacìonalʹnogo unìversitetu Lʹvìvsʹka polìtehnìka Serìâ Ìnformacìjnì sistemi ta merežì · doi:10.23939/sisn2025.17.239
The value of dignity appeals: evidence from a social media experiment
Bollen, P. H., Kymlicka, W., Lieberman, E. S., & Read, B. M. (2025) · Political Science Research and Methods · doi:10.1017/psrm.2025.10022
Garba, K. T. G., & Umoh, E. R. U. (2025) · African Journal of Stability and Development (AJSD) · doi:10.53982/ajsd.2025.1701.14-j
Making Free Players Pay: Selling Endowments in Virtual WorldsPreprint
Runge, J., & Ryan, C. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5372427
Are Exposure Bucks Worth It? Evidence from InstagramPreprint
Ching, A. T., & Goetz, D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5371610
Optimización de Campañas para Redes Sociales
García-Martínez, M.-E., Rodríguez-Valero, D., & Delgado, F. O. (2025) · VISUAL REVIEW International Visual Culture Review / Revista Internacional de Cultura Visual · doi:10.62161/revvisual.v17.5855
Maximizing Reach and Efficiency of Digital Out-of-Home AdvertisingPreprint
Lee, S., Bart, Y., & Pauwels, K. H. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5728803
Chen, X., Li, Z. G., Liang, Y., & Zhu, R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5733704
Advertising as Information or Reminder: Efficiency Gains from Personalizing Ad Length and FrequencyPreprint
Goli, A., Reiley, D., Tungodden, J., & Qiu, W. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5745603
Schwartz, D., Navarrete, M., & Goić, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5674310
Muñoz-Cordero, B., Romero-Feregrino, R., Feregrino, R. R., Cabello, R. R., Rocha, V., Martinez-Medina, R., & Urrutia, L. A. F. (2025) · JMIR Formative Research · doi:10.2196/82889
Chen, J., Guo, Y., Liu–Thompkins, Y., & Guo, H. Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5472986
Are Display Ad Content and Landing Page Ad Content Complements or Substitutes? A Field ExperimentPreprint
Kim, Y., & Kalyanam, K. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5620810
An instrument-free approach for estimating multinomial logit models with endogenous variables
de Grange, L., Imbert, M. M., Troncoso, R., Pezoa, R., & González, F. (2026) · Journal of Choice Modelling · doi:10.1016/j.jocm.2026.100595
Prospecting Versus Retargeting: Insights from a Geography-Based Field Experiment
Larson, J. S., & Dotson, J. P. (2026) · Journal of Interactive Marketing · doi:10.1177/10949968261422740
Emerging Frontiers in Marketing: Exploring the Potential of Novel Data and Methodologies
Dong, S., Xiao, L., & Kwon, J. (2026) · Australasian Marketing Journal (AMJ) · doi:10.1177/14413582261420453 · Cited by 1
Li, A., Chen, Y., & Shin, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6146606
Knight, S., Hsieh, T.-Y., & Bart, Y. (2026) · Journal of Marketing Research · doi:10.1177/00222437261435578
Wang, Y., Zhong, J., Cui, Y., & Guo, Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6438559
Four Essays in Digital Economics : Algorithmic Bias and Personal Data
Jean, C. (2021) · theses.fr (ABES) · doi:10.70675/1fd30da9zcfabz4d0dzbc1fze9c92e66d617
Causal Inference from Heterogeneous Data with Missing Data : Application to Critical Care Management
Mayer, I. (2021) · theses.fr (ABES) · doi:10.70675/2f432d6cz11f8z4d0bza530zbc10fbe668bc
Nguyen, T. P. L. (2026) · Future Business Journal · doi:10.1186/s43093-026-00823-8
Correcting Selection Bias in Behavioral Intent Measurement using Double Machine LearningPreprint
Aggarwal, D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6540318
A Unified Framework for Observational Incrementality Measurement in Multi-Treatment SystemPreprint
Aggarwal, D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6566098
Advertising Effectiveness and Local Economic Conditions: Evidence from Housing Cost ShiftsPreprint
Knight, S., Hsieh, T.-Y., & Bart, Y. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6708458
Candan, S. K., & Alakaş, E. Ö. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6911338
Hrnjadovic, D., & Dost, F. (2026) · Business Horizons · doi:10.1016/j.bushor.2026.06.009
Candan, S. K., & Alakaş, E. Ö. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6992862
Ferguson, S. (2026) · American Impact Review · doi:10.66308/air.e2026011
The financial impact of marketing: Looking back and moving forward
Dekimpe, M. G., & Hanssens, D. M. (2026) · Marketing Strategy Journal · doi:10.1016/j.msj.2026.100018
Silveira, C. M. (2026) · American Impact Review · doi:10.66308/air.e2026014
Confronting Deepfake Technology with Respect to Legal, Ethical, and Technological Perspectives
Gangwar, V. P., Sharma, R., Kumar, R., & Tiwari, R. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-27563-9_17
Lai, L., Cheng, Z., & Liu, Y. (2026) · Journal of Marketing Analytics · doi:10.1057/s41270-026-00515-w
Performance, Metrics, and Value: A Systematic Literature Review on Measurement in Digital Marketing
Nastasa, A. C., CICEU, L.-D., BURLAN, I. C. M., & Mogoș, O. C. (2026) · Proceedings of the ... International Conference on Business Excellence · doi:10.2478/picbe-2026-0274
Bai, J., & Wu, Q. (2026) · International Journal of Electronics and Communications Systems · doi:10.24042/ijecs.v6i1.30533
Do brands need search advertising?
Beknazarova, I., Cohen, J., Driesener, C. B., Hartnett, N., & Bellman, S. (2026) · Journal of Business Research · doi:10.1016/j.jbusres.2026.116485
Ye, Y., & Xue, P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7447235
Espinal, E. A., Orejuela, A. R., & Herrera, P. A. L. (2026) · Journal of theoretical and applied electronic commerce research · doi:10.3390/jtaer21100337
Generative Search: Evidence from a Large-Scale Field Experiment
Zheng, S., Zhu, Y., Ye, X., & Shen, L. (2026) · Management Science · doi:10.1287/mnsc.2025.02458 · Cited by 1
Predicted Incrementality by Experimentation for Ad Measurement
Gordon, B. R., Moakler, R., & Zettelmeyer, F. (2026) · Management Science · doi:10.1287/mnsc.2023.01108
Self-citationClaro, D. P., Vieira, V. A., de Gregorio, F., & Plouffe, C. R. (2026) · Journal of Advertising Research · doi:10.1080/00218499.2026.2719856
Foss, N. J., & Hünermund, P. (2026) · Journal of Management Scientific Reports · doi:10.1177/27550311261495585
Spitsin, V., & Spitsina, D. V. (2026) · Вестник Омского университета. Серия «Экономика» · doi:10.24147/1812-3988.2026.24(2).121-133
Huang, J., Liu, R., & Zhou, Y. (2026) · Journal of Forecasting · doi:10.1002/for.70231
Chen, Q., & Chen, R.-T. (2026) · Journal of Theoretical and Applied Electronic Commerce Research · doi:10.3390/jtaer21110388
Çetintürk, N. (2026) · Bucak İşletme Fakültesi Dergisi · doi:10.38057/bifd.1980516
Rafieian, O. (2026) · Marketing Science · doi:10.1287/mksc.2023.0382
Metadata
- Authors
- Gordon, B. R., Zettelmeyer, F., Bhargava, N., & Chapsky, D.
- Affiliations
- Northwestern University (US)
- Meta (United States) (US)
- Corresponding author
- Gordon, Brett R.
- Zettelmeyer, Florian
- Chapsky, Dan
- Venue
- Marketing Science
- Published
- 2019-03
- Volume
- 38
- Issue
- 2
- Pages
- 193-225
- ISSN
- 0732-2399 (ISSN-L)
- 1526-548X
- OpenAlex
- W2564917231
- Cited by
- 287
- References
- 57
- Citations by year
- 2026: 41 · 2025: 38 · 2024: 24 · 2023: 41 · 2022: 35 · 2021: 31 · 2020: 45 · 2019: 14
- FWCI
- 62.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Marketing Science (publishedVersion)
- Topics
- Social Sciences › Business, Management and Accounting › Marketing › Digital Marketing and Social Media
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Business, Management and Accounting › Marketing › Consumer Market Behavior and Pricing
- Keywords
- advertising measurement, randomized field experiment, observation method, treatment effect estimation, Facebook, social media advertising, causal inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review