The DOI matches its Crossref record; 55 of the 57 references it cites were found in a registry or an index.
A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook
Abstract
Observational methods often fail to accurately recover the treatment effects generated from randomized advertising experiments on Facebook.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Large-scale studies suggest that most of the time, due to methodological issues, the efficiency of targeting is largely overstated (Gordon et al., 2019;Lewis and Reiley, 2014), and that, overall,”
From How online advertising targets consumers: The uses of categories and algorithmic tools by audience planners (2024) · cites this paper as Gordon et al., 2019
In the citing paper
“computer generated categories and optimization are not just overstated (Aral, 2020;Gordon et al., 2019), they also neither match the power of demographics as standards, nor the idiosyncratic meaningfulness of custom audiences.”
From How online advertising targets consumers: The uses of categories and algorithmic tools by audience planners (2024) · cites this paper as Gordon et al., 2019
Citations
Cites
The central role of the propensity score in observational studies for causal effects
ROSENBAUM, P. R., & RUBIN, D. B. (1983) · Biometrika · doi:10.1093/biomet/70.1.41 · Cited by 31,241
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
Data Analysis Using Regression and Multilevel/Hierarchical Models
Gelman, A., & Hill, J. (2006) · doi:10.1017/cbo9780511790942 · Cited by 13,597
Rosenbaum, P. R., & Rubin, D. B. (1985) · The American Statistician · doi:10.2307/2683903 · Cited by 6,472
SOME PRACTICAL GUIDANCE FOR THE IMPLEMENTATION OF PROPENSITY SCORE MATCHING
Caliendo, M., & Kopeinig, S. (2008) · Journal of Economic Surveys · doi:10.1111/j.1467-6419.2007.00527.x · Cited by 6,238
Matching Methods for Causal Inference: A Review and a Look Forward
Stuart, E. A. (2010) · Statistical Science · doi:10.1214/09-sts313 · Cited by 5,478
Recent Developments in the Econometrics of Program Evaluation
Imbens, G. W., & Wooldridge, J. M. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.1.5 · Cited by 4,926
Propensity Score-Matching Methods for Nonexperimental Causal Studies
Dehejia, R., & Wahba, S. (2002) · The Review of Economics and Statistics · doi:10.1162/003465302317331982 · Cited by 4,864
Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction
Imbens, G. W., & Rubin, D. B. (2015) · doi:10.1017/cbo9781139025751 · Cited by 4,210
Identification and estimation of local average treatment effects
Imbens, G. W., & Angrist, J. D. (1994) · Econometrica · doi:10.2307/2951620 · Cited by 4,101
Selection on observed and unobserved variables: Assessing the effectiveness of Catholic schools
Altonji, J. G., Elder, T. E., & Taber, C. R. (2005) · Journal of Political Economy · doi:10.1086/426036 · Cited by 3,706
Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review
Imbens, G. W. (2004) · Review of Economics and Statistics · doi:10.1162/003465304323023651 · Cited by 2,994
Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score
Hirano, K., Imbens, G. W., & Ridder, G. (2003) · Econometrica · doi:10.1111/1468-0262.00442 · Cited by 2,602
Bayesian inference for causal effects: The role of randomization
Rubin, D. B. (1978) · The Annals of Statistics · doi:10.1214/aos/1176344064 · Cited by 2,503
Using Propensity Scores to Help Design Observational Studies: Application to the Tobacco Litigation
Rubin, D. B. (2001) · Health Services and Outcomes Research Methodology · doi:10.1023/a:1020363010465 · Cited by 2,228 · cited 1×
“This pair of conditions is commonly known as the Stable Unit Treatment Value Assumption (SUTVA), a term coined in Rubin (1978).”
Covariate Balancing Propensity Score
Imai, K., & Ratkovic, M. (2013) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12027 · Cited by 1,379
McCaffrey, D. F., Ridgeway, G., & Morral, A. R. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.4.403 · Cited by 1,317
From Social to Sale: The Effects of Firm-Generated Content in Social Media on Customer Behavior
Kumar, A., Bezawada, R., Rishika, R., Janakiraman, R., & Kannan, P. K. (2015) · Journal of Marketing · doi:10.1509/jm.14.0249 · Cited by 1,156
Best Practices in Quantitative Methods
Osborne, J. W. (2008) · Sage eBooks · doi:10.4135/9781412995627 · Cited by 1,065
Inverse probability weighted estimation for general missing data problems
Wooldridge, J. M. (2007) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.02.002 · Cited by 1,029
The Effectiveness of Adjustment by Subclassification in Removing Bias in Observational Studies
Cochran, W. G. (1968) · Biometrics · doi:10.2307/2528036 · Cited by 961
Matching on the Estimated Propensity Score
Abadie, A., & Imbens, G. W. (2016) · Econometrica · doi:10.3982/ecta11293 · Cited by 904
Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain
Belloni, A., Chen, D. L., Chernozhukov, V., & Hansen, C. (2012) · Econometrica · doi:10.3982/ecta9626 · Cited by 892
On the Failure of the Bootstrap for Matching Estimators
Abadie, A., & Imbens, G. W. (2008) · Econometrica · doi:10.3982/ecta6474 · Cited by 814
Online Display Advertising: Targeting and Obtrusiveness
Avi Goldfarb, B., & Tucker, C. E. (2011) · Marketing Science · doi:10.1287/mksc.1100.0583 · Cited by 801
Large Sample Confidence Regions Based on Subsamples under Minimal Assumptions
Politis, D. N., & Romano, J. P. (1994) · The Annals of Statistics · doi:10.1214/aos/1176325770 · Cited by 748
Comparison of Multivariate Matching Methods: Structures, Distances, and Algorithms
Gu, X. S., & Rosenbaum, P. R. (1993) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.1993.10474623 · Cited by 702 · cited 1×
“But this metric may not work well with a large number of covariates (Gu and Rosenbaum 1993) and can be computationally demanding.”
Ichino, A., Mealli, F., & Nannicini, T. (2008) · Journal of Applied Econometrics · doi:10.1002/jae.998 · Cited by 601
Sensitivity to Exogeneity Assumptions in Program Evaluation
Imbens, G. W. (2003) · American Economic Review · doi:10.1257/000282803321946921 · Cited by 528
Westreich, D. J., Lessler, J., & Funk, M. J. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.11.020 · Cited by 520
Matching Methods in Practice: Three Examples
Imbens, G. W. (2015) · The Journal of Human Resources · doi:10.3368/jhr.50.2.373 · Cited by 492
Changing Names with Style: Mutual Fund Name Changes and Their Effects on Fund Flows
COOPER, M. J., GULEN, H., & RAU, P. R. (2005) · The Journal of Finance · doi:10.1111/j.1540-6261.2005.00818.x · Cited by 472
Approximate Residual Balancing: Debiased Inference of Average Treatment Effects in High Dimensions
Athey, S. C., Imbens, G. W., & Wager, S. (2018) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/rssb.12268 · Cited by 373
Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment
Blake, T., Nosko, C., & Tadelis, S. (2015) · Econometrica · doi:10.3982/ecta12423 · Cited by 371
Stable weights that balance covariates for estimation with incomplete outcome data
Zubizarreta, J. R. (2015) · Journal of the American Statistical Association · doi:10.1080/01621459.2015.1023805 · Cited by 369 · cited 1×
“12Researchers have recently developed more sophisticated methods for estimating causal effects (Imai and Ratkovic 2014), including those that blend insights from operations research (Zubizarreta 2012, Zubizarreta 2015) and machine learning (Athey, Imbens, and Wager forthcoming).”
A Three-step Method for Choosing the Number of Bootstrap Repetitions
Andrews, D. W. K., & Buchinsky, M. (2000) · Econometrica · doi:10.1111/1468-0262.00092 · Cited by 309
The Unfavorable Economics of Measuring the Returns to Advertising *
Lewis, R. A., & Rao, J. M. (2015) · The Quarterly Journal of Economics · doi:10.1093/qje/qjv023 · Cited by 305
Does alcohol advertising promote adolescent drinking? Results from a longitudinal assessment
Ellickson, P. L., Collins, R. L., Hambarsoomians, K., & McCaffrey, D. F. (2005) · Addiction · doi:10.1111/j.1360-0443.2005.00974.x · Cited by 295
Optimal multivariate matching before randomization.
Greevy, R., Lu, B., Silber, J., & Rosenbaum, P. (2004) · Biostatistics · doi:10.1093/biostatistics/5.2.263 · Cited by 233
Zubizarreta, J. R. (2012) · Journal of the American Statistical Association · doi:10.1080/01621459.2012.703874 · Cited by 213
Lewis, R. A., & Reiley, D. H. (2014) · Quantitative Marketing and Economics · doi:10.1007/s11129-014-9146-6 · Cited by 211
Efficiency Wages Reconsidered: Theory and Evidence
Murphy, K., & Topel, R. (1990) · Palgrave Macmillan UK eBooks · doi:10.1007/978-1-349-10688-2_8 · Cited by 197
Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness
Johnson, G. A., Lewis, R. A., & Nubbemeyer, E. (2017) · Journal of Marketing Research · doi:10.1509/jmr.15.0297 · Cited by 179
Lewis, R. A., Rao, J. M., & Reiley, D. (2011) · International World Wide Web Conference (WWW) · doi:10.1145/1963405.1963431 · Cited by 108
Effect of temporal spacing between advertising exposures: Evidence from online field experiments
Sahni, N. S. (2015) · Quantitative Marketing and Economics · doi:10.1007/s11129-015-9159-9 · Cited by 103
Estimating the Causal Effects of Marketing Interventions Using Propensity Score Methodology
Rubin, D., & Waterman, R. P. (2006) · Statistical Science · doi:10.1214/088342306000000259 · Cited by 94
Ferraro, P. J., & Miranda, J. J. (2014) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2014.03.008 · Cited by 85
Do Digital Video Recorders Influence Sales?
Bronnenberg, B. J., Dubé, J., & Mela, C. F. (2010) · Journal of Marketing Research · doi:10.1509/jmkr.47.6.998 · Cited by 79
When Less Is More: Data and Power in Advertising Experiments
Johnson, G. A., Lewis, R. A., & Reiley, D. (2016) · Marketing Science · doi:10.1287/mksc.2016.0998 · Cited by 79
EFFECTS OF COVARIATE OMISSION AND CATEGORIZATION WHEN ANALYSING RANDOMIZED TRIALS WITH THE COX MODEL
Schmoor, C., & Schumacher, M. C. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19970215)16:3 · Cited by 73
Estimating the causal effects of marketing interventions using propensity score methodologyPreprint
Rubin, D. B., & Waterman, R. P. (2006) · Cited by 69
Internet versus Television Advertising: A Brand-Building Comparison
Draganska, M., Hartmann, W. R., & Stanglein, G. (2013) · Journal of Marketing Research · doi:10.1509/jmr.13.0124 · Cited by 62
Kalyanam, K., McAteer, J., Marek, J., Hodges, J. S., & Lin, L. (2017) · Quantitative Marketing and Economics · doi:10.1007/s11129-017-9188-7 · Cited by 50
Arceneaux, K., Gerber, A. S., & GREEN, D. P. (2010) · Sociological Methods & Research · doi:10.1177/0049124110378098 · Cited by 43
Zaltman, G., Zaltman, L., Sturgess, D. J., Lee, A., Fujikawa, Y., & Carbone, L. P. (2008) · Harvard business review · Cited by 17
Research works printed but not linked: 10
- Abraham M, 2008, Harvard Bus. Rev., 86, 28
- Berkovich P, 2016, Nielsen J. Measurement
- Bickel P, 1997, Statistica Sinicia, 7, 1
- Eckles D, Bakshy E (2017) Bias and high-dimensional adjustment in observational studies of Peer effects. Working paper, MIT Sloan School of Management, Cambridge, MA.
- Ferraro PJ, 2017, J. Assoc. Environ. Resource Econom., 4, 281
- LaLonde RJ, 1986, Amer. Econom. Rev., 76, 604
- Oster E, 2017, J. Bus. Econom. Statist.
- Pham TT, Shen Y (2017) A deep causal inference approach to measuring the effects of forming group loans in online non-profit microfinance platform. Working paper, Stanford Graduate School of Business, Stanford, CA. Available at https://arxiv.org/pdf/1706.02795.pdf. · arXiv
- Rosenbaum P, 1983, J. Royal Statist. Soc. Ser. B, 45, 212
- Tibshirani R, 1996, J. Royal Statist. Soc., 58, 267
Cited by
Sorted by citations
The future of social media in marketing
Appel, G., Grewal, L., Hadi, R., & Stephen, A. T. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00695-1 · Cited by 1,893
“More recently, the value of advertising on social media continues to be explored (e.g., Gordon et al. 2019), as well as how it interacts with other forms of media such as television (e.g., Fossen and Schweidel 2016, 2019) and affects new product adoption through diffusion of information mechanisms…”
BackgroundTrustworthy Online Controlled Experiments: A Practical Guide to A/B Testing
Kohavi, R., Tang, D., & Xu, Y. (2020) · doi:10.1017/9781108653985 · Cited by 297
Online and offline retailing: What we know and directions for future research
Ratchford, B. T., Soysal, G., Zentner, A., & Gauri, D. K. (2022) · Journal of Retailing · doi:10.1016/j.jretai.2022.02.007 · Cited by 232
Online Display Advertising Markets: A Literature Review and Future Directions
Choi, H., Mela, C. F., Balseiro, S. R., & Leary, A. (2020) · Information Systems Research · doi:10.1287/isre.2019.0902 · Cited by 181
Inefficiencies in Digital Advertising Markets
Gordon, B. R., Jerath, K., Katona, Z., Narayanan, S., Shin, J., & Wilbur, K. C. (2020) · Journal of Marketing · doi:10.1177/0022242920913236 · Cited by 195
Self-citation
Metadata
- Authors
- Gordon, B. R., Zettelmeyer, F., Bhargava, N., & Chapsky, D.
- Affiliations
- Northwestern University (US)
- Meta (United States) (US)
- Corresponding author
- Gordon, Brett R.
- Zettelmeyer, Florian
- Chapsky, Dan
- Venue
- Marketing Science
- Published
- 2019-03
- Volume
- 38
- Issue
- 2
- Pages
- 193-225
- ISSN
- 0732-2399 (ISSN-L)
- 1526-548X
- OpenAlex
- W2564917231
- Cited by
- 287
- References
- 57
- Citations by year
- 2026: 41 · 2025: 38 · 2024: 24 · 2023: 41 · 2022: 35 · 2021: 31 · 2020: 45 · 2019: 14
- FWCI
- 62.78
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Marketing Science (publishedVersion)
- Topics
- Social Sciences › Business, Management and Accounting › Marketing › Digital Marketing and Social Media
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Business, Management and Accounting › Marketing › Consumer Market Behavior and Pricing
- Keywords
- advertising measurement, randomized field experiment, observation method, treatment effect estimation, Facebook, social media advertising, causal inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
