The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Big Data: New Tricks for Econometrics
Open accessAbstract
Computers are now involved in many economic transactions and can capture data associated with these transactions, which can then be manipulated and analyzed. Conventional statistical and econometric techniques such as regression often work well, but there are issues unique to big datasets that may require different tools. First, the sheer size of the data involved may require more powerful data manipulation tools. Second, we may have more potential predictors than appropriate for estimation, so we need to do some kind of variable selection. Third, large datasets may allow for more flexible relationships than simple linear models. Machine learning techniques such as decision trees, support vector machines, neural nets, deep learning, and so on may allow for more effective ways to model complex relationships. In this essay, I will describe a few of these tools for manipulating and analyzing big data. I believe that these methods have a lot to offer and should be more widely known and used by economists.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Modern Applied Statistics with S
Venables, W. N., & Ripley, B. D. (2002) · Statistics and Computing · doi:10.1007/978-0-387-21706-2 · Cited by 24,305
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
An Introduction to Statistical Learning
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7 · Cited by 12,106
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
(2019) · AEA Papers and Proceedings · doi:10.1257/pandp.109.612 · Cited by 10,902
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436
(2012) · Cited by 9,062
Friedman, J. H. (2002) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(01)00065-2 · Cited by 7,067
Cited by
Biau, G., & Scornet, E. (2016) · Test · doi:10.1007/s11749-016-0481-7 · Cited by 4,149
Why Are There Still So Many Jobs? The History and Future of Workplace Automation
Autor, D. H. (2015) · Journal of Economic Perspectives · doi:10.1257/jep.29.3.3 · Cited by 3,917
Angrist, J. D., & Krueger, A. B. (2001) · Journal of Economic Perspectives · doi:10.1257/jep.15.4.69 · Cited by 2,355
Machine Learning: An Applied Econometric Approach
Mullainathan, S., & Spiess, J. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.87 · Cited by 1,989
Athey, S., Tibshirani, J., & Wager, S. (2019) · The Annals of Statistics · doi:10.1214/18-aos1709 · Cited by 1,870
Marketing Analytics for Data-Rich Environments
Wedel, M., & Kannan, P. K. (2016) · Journal of Marketing · doi:10.1509/jm.15.0413 · Cited by 1,261
Machine Learning Methods That Economists Should Know About
Athey, S., & Imbens, G. W. (2019) · Annual Review of Economics · doi:10.1146/annurev-economics-080217-053433 · Cited by 1,163
Forecasting: theory and practice
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M. Z., Barrow, D. K., Ben Taieb, S., Bergmeir, C., Bessa, R. J., Bijak, J., Boylan, J. E., et al. (2022) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.001 · Cited by 921
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Predictably Unequal? The Effects of Machine Learning on Credit Markets
Fuster, A., Goldsmith-Pinkham, P. S., Ramadorai, T., & Walther, A. (2017) · doi:10.2139/ssrn.3072038 · Cited by 769
Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500
Krauss, C., Do, X. A., & Huck, N. (2017) · European Journal of Operational Research · doi:10.1016/j.ejor.2016.10.031 · Cited by 750
Artificial Intelligence: The Ambiguous Labor Market Impact of Automating Prediction
Agrawal, A., Gans, J. S., & Goldfarb, A. (2019) · Journal of Economic Perspectives · doi:10.1257/jep.33.2.31 · Cited by 685
The View from Above: Applications of Satellite Data in Economics
Donaldson, D., & Storeygard, A. (2016) · Journal of Economic Perspectives · doi:10.1257/jep.30.4.171 · Cited by 654
Will digital technologies transform agriculture in developing countries?
Deichmann, U., Goyal, A., & Mishra, D. (2016) · Agricultural Economics · doi:10.1111/agec.12300 · Cited by 587
Predictably Unequal? The Effects of Machine Learning on Credit Markets
FUSTER, A., GOLDSMITH‐PINKHAM, P., RAMADORAI, T., & WALTHER, A. (2022) · The Journal of Finance · doi:10.1111/jofi.13090 · Cited by 872
Online labour index: Measuring the online gig economy for policy and research
Kässi, O., & Lehdonvirta, V. (2018) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2018.07.056 · Cited by 547
Smith, G. (2018) · Journal of Big Data · doi:10.1186/s40537-018-0143-6 · Cited by 532
Economics in the age of big data
Einav, L., & Levin, J. (2014) · Science · doi:10.1126/science.1243089 · Cited by 498
Big Data and Data Science Methods for Management Research
George, G., Osinga, E. C., Lavie, D., & Scott, B. A. (2016) · Academy of Management Journal · doi:10.5465/amj.2016.4005 · Cited by 395
Born to be green: new insights into the economics and management of green entrepreneurship
Demirel, P., Li, Q. C., Rentocchini, F., & Tamvada, J. P. (2019) · Small Business Economics · doi:10.1007/s11187-017-9933-z · Cited by 388
Forecasting tourism demand with composite search index
Li, X., Pan, B., Law, R., & Huang, X. (2017) · Tourism Management · doi:10.1016/j.tourman.2016.07.005 · Cited by 367
Big Data sources and methods for social and economic analyses
Blazquez, D., & Domenech, J. (2018) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2017.07.027 · Cited by 343
Model Averaging and Its Use in Economics
Steel, M. F. J. (2020) · Journal of Economic Literature · doi:10.1257/jel.20191385 · Cited by 321
What Is Different About Digital Strategy? From Quantitative to Qualitative Change
Adner, R., Puranam, P., & Zhu, F. (2019) · Strategy Science · doi:10.1287/stsc.2019.0099 · Cited by 328
Big Data in Agriculture: A Challenge for the Future
Coble, K. H., Mishra, A. K., Ferrell, S., & Griffin, T. (2018) · Applied Economic Perspectives and Policy · doi:10.1093/aepp/ppx056 · Cited by 325
Brown, R., & Rocha, A. (2020) · Journal of Business Venturing Insights · doi:10.1016/j.jbvi.2020.e00174 · Cited by 325
Polanyi's Paradox and the Shape of Employment Growth
Autor, D. (2014) · doi:10.3386/w20485 · Cited by 323
Theory building with big data-driven research–Moving away from the “What” towards the “Why”
Kar, A. K., & Dwivedi, Y. K. (2020) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2020.102205 · Cited by 301
The Billion Prices Project: Using Online Prices for Measurement and Research
Cavallo, A., & Rigobon, R. (2016) · Journal of Economic Perspectives · doi:10.1257/jep.30.2.151 · Cited by 300
Machine Learning for Sociology
Molina, M., & Garip, F. (2019) · Annual Review of Sociology · doi:10.1146/annurev-soc-073117-041106 · Cited by 295
Predicting property prices with machine learning algorithms
Ho, W. K. O., Tang, B.-S., & Wong, S. W. (2021) · Journal of Property Research · doi:10.1080/09599916.2020.1832558 · Cited by 293
Corporate digital responsibility in service firms and their ecosystems
Wirtz, J., Kunz, W. H., Hartley, N., & Tarbit, J. (2023) · Journal of Service Research · doi:10.1177/10946705221130467 · Cited by 291
The GDP-Temperature relationship: Implications for climate change damages
Newell, R. G., Prest, B. C., & Sexton, S. E. (2021) · Journal of Environmental Economics and Management · doi:10.1016/j.jeem.2021.102445 · Cited by 280
Kushwaha, A. K., Kumar, P., & Kar, A. K. (2021) · Industrial Marketing Management · doi:10.1016/j.indmarman.2021.08.011 · Cited by 258
Yoon, J. (2021) · Computational Economics · doi:10.1007/s10614-020-10054-w · Cited by 256
Causal inference in economics and marketing
Varian, H. R. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510479113 · Cited by 254
Self-citationTapia, C., Abajo, B., Feliu, E., Mendizabal, M., Martinez, J. A., Fernández, J. G., Laburu, T., & Lejarazu, A. (2017) · Ecological Indicators · doi:10.1016/j.ecolind.2017.02.040 · Cited by 238
Lassopack: model selection and prediction with regularized regression in Stata
Ahrens, A., Hansen, C. B., & Schaffer, M. E. (2020) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x20909697 · Cited by 212
Chen, N.-C., Drouhard, M., Kocielnik, R., Suh, J., & Aragon, C. R. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185515 · Cited by 152
Liu, X. (2020) · Industrial Marketing Management · doi:10.1016/j.indmarman.2019.02.021 · Cited by 140
Satisficing: Integrating Two Traditions
Artinger, F. M., Gigerenzer, G., & Jacobs, P. (2022) · Journal of Economic Literature · doi:10.1257/jel.20201396 · Cited by 90
Data-Driven Policy Making: The Policy Lab Approach
van Veenstra, A. F., & Kotterink, B. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-64322-9_9 · Cited by 60
Marketing analytics using anonymized and fragmented tracking data
Kakatkar, C., & Spann, M. (2019) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2018.10.001 · Cited by 59
Machine Learning in Policy Evaluation: New Tools for Causal Inference
Kreif, N., & DiazOrdaz, K. (2019) · Oxford Research Encyclopedia of Economics and Finance · doi:10.1093/acrefore/9780190625979.013.256 · Cited by 57
Using Machine Learning in Burnout Prediction: A Survey
Grządzielewska, M. (2021) · Child and Adolescent Social Work Journal · doi:10.1007/s10560-020-00733-w · Cited by 46
Artificial intelligence as structural estimation: Deep Blue, Bonanza, and AlphaGo
Igami, M. (2020) · The Econometrics Journal · doi:10.1093/ectj/utaa005 · Cited by 28
Bhandary, R., & Ghosh, B. K. (2025) · Journal of Risk and Financial Management · doi:10.3390/jrfm18010023 · Cited by 27
Artificial Intelligence: The Ambiguous Labor Market Impact of Automating Prediction
Agrawal, A. K., Gans, J. S., & Goldfarb, A. (2019) · doi:10.2139/ssrn.3341456 · Cited by 21
Retooling Poverty Targeting Using Out-of-Sample Validation and Machine Learning
McBride, L., & Nichols, A. (2016) · doi:10.1596/1813-9450-7849 · Cited by 12
Bailey, A. L., Blackstock‐Bernstein, A., Ryan, E., & Pitsoulakis, D. (2016) · Data Mining and Learning Analytics · doi:10.1002/9781118998205.ch15 · Cited by 7
Metadata
- Authors
- Varian, H. R.
- Venue
- Journal of Economic Perspectives
- Publisher
- American Economic Association
- Published
- 2014-05-01
- Volume
- 28
- Issue
- 2
- Pages
- 3-28
- ISSN
- 0895-3309 (ISSN-L)
- OpenAlex
- W2155419203
- Cited by
- 1,557
- References
- 47
- FWCI
- 184.85
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.