The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
The Unfavorable Economics of Measuring the Returns to Advertising *
Abstract
Twenty-five large field experiments with major U.S. retailers and brokerages, most reaching millions of customers and collectively representing $2.8 million in digital advertising expenditure, reveal that measuring the returns to advertising is difficult. The median confidence interval on return on investment is over 100 percentage points wide. Detailed sales data show that relative to the per capita cost of the advertising, individual-level sales are very volatile; a coefficient of variation of 10 is common. Hence, informative advertising experiments can easily require more than 10 million person-weeks, making experiments costly and potentially infeasible for many firms. Despite these unfavorable economics, randomized control trials represent progress by injecting new, unbiased information into the market. The inference challenges revealed in the field experiments also show that selection bias, due to the targeted nature of advertising, is a crippling concern for widely employed observational methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Harrison, A., Robinson, C., Ashenfelter, O., & Layard, R. (1989) · The Canadian Journal of Economics · doi:10.2307/135687 · Cited by 3,153
Statistical Power Analysis for the Behavioral Sciences
(1977) · doi:10.1016/c2013-0-10517-x · Cited by 2,457
Does Management Matter? Evidence from India *
Bloom, N., Eifert, B., Mahajan, A., McKenzie, D., & Roberts, J. (2013) · The Quarterly Journal of Economics · doi:10.1093/qje/qjs044 · Cited by 1,418
Lee, I.-M., Cook, N. R., Gaziano, J. M., Gordon, D., Ridker, P. M., Manson, J. E., Hennekens, C. H., & Buring, J. E. (2005) · JAMA · doi:10.1001/jama.294.1.56 · Cited by 1,137
Does management matter? Evidence from India
Bloom, N., Eifert, B., Mahajan, A., McKenzie, D., & Roberts, J. (2011) · doi:10.1596/1813-9450-5573 · Cited by 603
Lodish, L. M., Abraham, M., Kalmenson, S., Livelsberger, J., Lubetkin, B., Richardson, B., & Stevens, M. E. (1995) · Journal of Marketing Research · doi:10.1177/002224379503200201 · Cited by 452
Statistical Power and Effect Size in Marketing Research
Sawyer, A. G., & Ball, A. D. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800302 · Cited by 269
Improving the sensitivity of online controlled experiments by utilizing pre-experiment data
Deng, A., Xu, Y., Kohavi, R., & Walker, T. (2013) · Proceedings of the sixth ACM international conference on Web search and data mining · doi:10.1145/2433396.2433413 · Cited by 223
Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment
Blake, T., Nosko, C., & Tadelis, S. (2014) · National Bureau of Economic Research · doi:10.3386/w20171 · Cited by 217
Cited by
Goldfarb, A., & Tucker, C. (2019) · Journal of Economic Literature · doi:10.1257/jel.20171452 · Cited by 2,744
Economics in the age of big data
Einav, L., & Levin, J. (2014) · Science · doi:10.1126/science.1243089 · Cited by 498
Hewett, K., Rand, W., Rust, R. T., & van Heerde, H. J. (2016) · Journal of Marketing · doi:10.1509/jm.15.0033 · Cited by 366
Gordon, B. R., Zettelmeyer, F., Bhargava, N., & Chapsky, D. (2019) · Marketing Science · doi:10.1287/mksc.2018.1135 · Cited by 287
Personalization in Email Marketing: The Role of Noninformative Advertising Content
Sahni, N. S., Wheeler, S. C., & Chintagunta, P. (2018) · Marketing Science · doi:10.1287/mksc.2017.1066 · Cited by 303
Metadata
- Authors
- Lewis, R. A., & Rao, J. M.
- Venue
- The Quarterly Journal of Economics
- Publisher
- Oxford University Press (OUP)
- Published
- 2015-07-06
- Volume
- 130
- Issue
- 4
- Pages
- 1941-1973
- ISSN
- 0033-5533 (ISSN-L)
- 1531-4650
- OpenAlex
- W1745376753
- Cited by
- 283
- References
- 24
- FWCI
- 81.95
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Unfavorable Economics of Measuring the Returns to Advertising *. https://app.cochat.ai/papers/10.1093/qje/qjv023
- The paper
- Lewis, R. A., & Rao, J. M. (2015). The Unfavorable Economics of Measuring the Returns to Advertising *. The Quarterly Journal of Economics. https://doi.org/10.1093/qje/qjv023
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.