The Parable of Google Flu: Traps in Big Data Analysis
Open accessThe DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index; 5 of the 8 sources its claims cite were checked for support.
Abstract
Large errors in flu prediction were largely avoidable, which offers lessons for the use of big data.
Overview
CoChat Summary
TL;DR
The paper uses the failure of Google Flu Trends (GFT) to illustrate two traps in big data analysis: “big data hubris” and algorithm dynamics. It argues that GFT’s large flu-prediction errors were largely avoidable and that big data should supplement, not replace, traditional data and methods.
Key findings
- In February 2013, GFT predicted more than double the proportion of doctor visits for influenza-like illness (ILI) than the CDC; from 21 August 2011 to 1 September 2013, GFT reported overly high flu prevalence in 100 out of 108 weeks, and it overshot the 2011–2012 season by more than 50%.
- GFT errors were not random: they showed temporal autocorrelation and seasonality, meaning last week’s errors predicted this week’s errors and error direction/magnitude varied with time of year.
- Simple alternatives outperformed GFT: out-of-sample mean absolute error was 0.486 for GFT, 0.311 for lagged CDC, and 0.232 for combined GFT and CDC; all differences were statistically significant at P < 0.05. Even 3-week-old CDC data did better than GFT at projecting current flu prevalence.
- The authors attribute GFT’s problems to overfitting (initial GFT matched 50 million search terms to 1152 data points and missed the nonseasonal 2009 H1N1 pandemic) and to “algorithm dynamics,” including Google search algorithm changes and user behavior changes.
Methods
- Design: policy forum/case study comparing GFT estimates with CDC ILI surveillance data and alternative forecasting models.
- Data/sample: GFT and CDC flu prevalence estimates, including the period from 21 August 2011 to 1 September 2013; the initial GFT methodology reportedly fit 50 million search terms to 1152 data points.
- Main analysis: comparison of GFT alone, lagged CDC data with 52-week seasonality variables, and a combined GFT + CDC model using lagged CDC estimates, lagged GFT error, and seasonality variables; reported mean absolute error and statistical significance; Google Correlate used to examine correlated search terms. Supplementary materials contain data and methods.
Limitations
- The paper states that GFT never documented the 45 search terms used, released examples appear misleading, and Google Correlate is limited to national-level data whereas GFT was developed using regional correlations; therefore the original GFT analyses cannot be replicated from the information provided.
- The paper states that Google cannot make its full data available to outsiders because of privacy issues, though it argues there is no such constraint on derivative, aggregated data.
- Evident from the text: this is a commentary/policy forum rather than a full empirical article, with key data and methods placed in supplementary materials; the quantitative comparison depends on GFT and CDC data and model specifications not fully detailed in the main text.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 2 Not valid · 2 Not checked · 8 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“This ad hoc method of throwing out peculiar search terms failed when GFT completely missed the nonseasonal 2009 infl uenza A–H1N1 pandemic ( 2, 14).”
What the sources say
- Partly supportedcites [2] matched to “Reassessing Google Flu Trends Data for Detection of Seasonal and Pandemic Influenza: A Comparative Epidemiological Study at Three Geographic Scales” (2013) sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says the original GFT model missed the spring 2009 pandemic wave, supporting the claim about its failure to detect the nonseasonal outbreak. It does not say GFT completely missed the entire 2009 pandemic. The Introduction describes automated selection of the original model’s search terms, not an ad hoc method of throwing out peculiar terms. (LLM, conf=0.86)
- Disputedcites [14] S. Cook et al., PLOS ONE 6, e23610 (2011). sourceExperimental — accuracy not yet measured
The Abstract (Methodology/Principal Findings) says the original GFT model underestimated the magnitude of illness during the 2009 H1N1 pandemic, not that it completely missed it. The Abstract (Conclusions) says both models performed well during the pandemic, although the updated model performed better. The Discussion notes that pandemic-specific queries were excluded, but does not describe this as an ad hoc method of throwing out peculiar terms. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Oddly, the few search terms offered in the papers ( 14) do not seem to be strongly related with either GFT or the CDC data (SM)—we surmise that the authors felt an unarticulated need to cloak the actual search terms identifi ed.”
What the sources say
- Disputedcites [14] S. Cook et al., PLOS ONE 6, e23610 (2011). sourceExperimental — accuracy not yet measured
The Results section (‘Search Behavior During pH1N1’; page unavailable) says that all three search queries shown in Figure 3 closely tracked CDC ILINet data before the pandemic, and ‘symptoms of flu’ continued to do so during it. The paper does report that the bronchitis and pneumonia queries underestimated ILINet data during the pandemic, but that does not support the broad claim that the offered terms were not strongly related to the data. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The Internet has opened the way for improving standard surveys, experiments, and health reporting ( 35).”
What the sources say
- Partly supportedcites [35] A. J. Berinsky et al., Polit. Anal. 20, 351–368 (2012). sourceExperimental — accuracy not yet measured
Section 2 describes using an Internet platform to run surveys with embedded experiments, and the Conclusion says MTurk can make experimental research easier and less expensive. The paper does not address improvements to health reporting; the disease scenario in section 4.3.2 is a framing experiment, not health reporting. Page numbers are unavailable in our copy. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although not widely reported until 2013, the new GFT has been persistently overestimating flu prevalence for a much longer time.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“From 21 August 2011 to 1 September 2013, GFT reported overly high fl u prevalence 100 out of 108 weeks.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
3 more claims
Create a free account to see all 14 claims and every source verdict.
- It may state two separate claims as one. 5
- Probably not a substantial claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Google Trends predicted more than twice the number of influenza-like illnesses than the CDC in February 2013 46 .”
From Unrepresentative big surveys significantly overestimated US vaccine uptake (2021) · cites this paper as [46]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was one of the main reasons behind the failure of Google Flu Trends, since the data used for training the model in the first few years was not representative of the trends in subsequent years [15].”
From Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data (2017) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Social data research often relies on counting entities. These entities can be users, links, or messages, and the description is a summary of these counts (Lazer et al., 2014; Salganik, 2017). Yet, simple counts can mislead if it is unclear what is counted and how.”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This unintelligibility is also subject to functional biases (section 4.1), as it depends on the mechanisms available on each social platform (e.g., having a like button, but not a dislike one), as well as to variations in platform algorithms and mechanisms in response to users actions (Lazer et al., 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It also includes temporal validity, which captures to what extent constructs change over time (Howison et al., 2011) and may invalidate previous conclusions about societal and/or platform phenomena; e.g., see the case of Google Flu Trends (Lazer et al., 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 14 MARCH 2014 VOL 343 SCIENCE www.sciencemag.org 1204
“The 2009 version of GFT has weathered other media panics related to the fl u, including the 2005–2006 influenza A/H5N1 ("bird flu") outbreak”
“GFT engineers updated the algorithm in 2009, and this model has run ever since”
Citations
Cites
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Twitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206
Detecting influenza epidemics using search engine query data
Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009) · Nature · doi:10.1038/nature07634 · Cited by 4,295
Cohen, D. (1998) · The Women's Review of Books · doi:10.2307/4022859 · Cited by 4,261
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Big Data : The Management Revolution
マカフィー, ア., ブリニョルフソン, エ., & 有賀, 裕. (2013) · Harvard business review · Cited by 3,549
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
Self-citationPredicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment
Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14009 · Cited by 2,649
Multiscale mobility networks and the spatial spreading of infectious diseases
Balcan, D., Colizza, V., Gonçalves, B., Hu, H., Ramasco, J. J., & Vespignani, A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0906910106 · Cited by 1,380
Self-citationPredicting consumer behavior with web search
Goel, S., Hofman, J. M., Lahaie, S., Pennock, D. M., & Watts, D. J. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1005962107 · Cited by 636
Research works printed but not linked: 7
- A. McAfee, E. Brynjolfsson, Harv. Bus. Rev. 90, 60 (2012).
- AAAI Conference on Weblogs and Social Media, Atlanta, Georgia, 11 to 15 July 2010 (Association for Advancement of Artifi cial Intelligence, 2010), pp. 178.
- P. Copeland et al., Int. Soc. Negl. Trop. Dis. 2013, 3
- I. M. Hall et al., Epidemiol. Infect. 135, 372 (2007).
- E. Mustafaraj, P. Metaxas, in Proceedings of the Web- Sci10, Raleigh, NC, 26 and 27 April 2010 (Web Science Trust, 2010); http://journal.webscience.org/317/.
- J. Ratkiewicz et al., in Proceedings of 5th International AAAI Conference on Weblogs and Social Media, San Francisco, CA, 7 to 11 August 2011 (AAAI, 2011), p. 297–304.
- 2013; http://chronicle.com/article/Researchers-Struggle- to-Secure/141591/.
Cited by
Items 1,501–1,600 of 2,513
Personal Data Collection and Usage for Mobile Marketing. Customer Awareness and Perception
Yartey, D., Omojola, O., Amodu, L. O., Ndubueze, N., Adeyeye, B., & Adesina, E. (2021) · WSEAS TRANSACTIONS ON BUSINESS AND ECONOMICS · doi:10.37394/23207.2021.18.5 · Cited by 5
“Big data has been found viable in locating trends in areas such as business as seen in [33], politics [34], education, [35], health [36], etc. Big data, in the long run, becomes a worthwhile social driver when data from the above-mentioned sectors are well gathered and analyzed to predict events…”
BackgroundChallenges and Opportunities for Statistics in the Next 25 Years
Kettenring, J. R., Koehler, K. J., & McKenzie, J. D., Jr. (2015) · The American Statistician · doi:10.1080/00031305.2015.1033987 · Cited by 5
“She focused on the health sciences which are collecting, combining, and analyzing both wide and deep forms of “Big Data,” which perhaps more appropriately should be called “All Data” (Lazer et al. 2014).”
BackgroundBig Data for Urban Health and Well-Being
Wang, S. J., & Moriarty, P. (2018) · Springer eBooks · doi:10.1007/978-3-319-73610-5_7 · Cited by 5
Global and local “teachable moments”: The role of Nobel Prize and national pride
Baram‐Tsabari, A., & Segev, E. (2018) · Public Understanding of Science · doi:10.1177/0963662518768410 · Cited by 7
“, 2009) was found to complement but not replace traditional epidemiological surveillance networks (Butler, 2013; Lazer et al., 2014), suggesting this is also the case for detecting interest in other topics.”
BackgroundBig Data em Pesquisas Populacionais sobre Câncer: Desafios e Oportunidades
Thuler, L. C. S., & Hartz, Z. (2019) · Revista Brasileira de Cancerologia · doi:10.32635/2176-9745.rbc.2019v65n4.718 · Cited by 5
Crowd-Squared: Amplifying the Predictive Power of Search Trend DataPreprint
Brynjolfsson, E., Geva, T., & Reichman, S. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2513559 · Cited by 5
Liautaud, P. R., Huybers, P., & Santillana, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.09911 · Cited by 10
“By way of example, attempts to predict seasonal influenza from Google search patterns appeared skillful over multiple seasons, but failed to forecast the 2009 H1N1 (“Swine Flu”) pandemic and overestimated the spread of the 2013 seasonal influenza in the United States by a factor of nearly two [2, 6, 22, 40].”
BackgroundGoogle Trends Analysis of COVID-19Preprint
Nguyen, H. L., Pan, Z., Abu-gellban, H., Jin, F., & Zhang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.03847 · Cited by 5
Automated Image Analysis for Studying Online BehaviourPreprint
Schwemmer, C., Unger, S., & Heiberger, R. H. (2022) · doi:10.31235/osf.io/t62sd · Cited by 5
Hao, P. (2019) · Wiley eBooks · doi:10.1002/9781118568446.eurs0306 · Cited by 6
Big Data for Ecological Models
Kress, M. M. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-409548-9.10557-3 · Cited by 5
Yang, C., Fang, H., Guo, S., Xia, Z., Zhang, X., Pan, X., Mu, H., Yuan, B., Li, C., & Du, P. (2025) · Cities · doi:10.1016/j.cities.2025.106515 · Cited by 6
Ngu, B. C., Fozao, K. F., Onabid, M. A., Nkwanyang, L. T., & Akongneh, Z. (2026) · Ore and Energy Resource Geology · doi:10.1016/j.oreoa.2026.100122 · Cited by 4
The Integration of Explanation and Prediction in Behavioral Science
Almaatouq, A., & Na, R. W. (2026) · Current Directions in Psychological Science · doi:10.1177/09637214261419427 · Cited by 6
“This criterion is integrative because out-of-sample accuracy alone does not indicate that a model has captured causal relationships; it may reflect idiosyncratic correlations that break when conditions shift (Lazer et al., 2014).”
Investigating the Impact of Control in AI-Assisted Decision-Making - An Experimental Study
Meske, C., & Ünal, E. (2024) · Mensch und Computer (MuC) · doi:10.1145/3670653.3677476 · Cited by 4
THANOS: A Predictive Model of Electoral Campaigns Using Twitter Data and Opinion Polls
Ghosh, D., Boettcher, W. A., Johnston, R., & Lahiri, S. N. (2025) · Data Science in Science · doi:10.1080/26941899.2025.2484180 · Cited by 10
Leveraging Data Analytics for Predictive Consumer Behavior Modelling
Tariq, M. U. (2025) · Advances in marketing, customer relationship management, and e-services book series · doi:10.4018/979-8-3693-3799-8.ch011 · Cited by 7
Improving meningitis surveillance and diagnosis with machine learning: Insights from São Paulo
Victor, A., Santos, D. A. M., Nery, E. K., Mori, D. P., Lucas, P. C. D. C., Cammarota, D., Montero, G. L. F., Filho, F. N. B., Yu, A. L. F., & Carvalhanas, T. R. M. P. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000925 · Cited by 5
“Additionally, the quality and consistency of data collected through surveillance systems can vary, potentially affecting ML model performance [40].”
Towards a Personalized Treatment of Head and Neck Cancer
Gross, A. M., & Cohen, E. E. W. (2015) · American Society of Clinical Oncology Educational Book · doi:10.14694/edbook_am.2015.35.28 · Cited by 4
Some Practical and Ethical Challenges Posed by Big Data
Fuller, M. (2015) · Issues in science and religion: publications of the European Society for the Study of Science and Theology · doi:10.1007/978-3-319-23944-6_10 · Cited by 4
From evolution to revolution: understanding mutability in large and disruptive human groups
Whitaker, R. M., Felmlee, D., Verma, D. C., Preece, A. D., & Williams, G.-R. (2017) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2267996 · Cited by 4
Population Health as a Complex Adaptive System of Systems
Page, S. E., & Zelner, J. (2020) · Oxford University Press eBooks · doi:10.1093/oso/9780190880743.003.0003 · Cited by 6
Chapter 17: Global Public Health Disaster Management and Technology
Nelson, E., Bustamante, N. D., & Kivlehan, S. M. (2020) · Springer eBooks · doi:10.1007/978-3-030-46355-7_22 · Cited by 4
Collecting data from Internet (and other platform) users for mental health research
Lahoud, A. A., Gladstone, T. R., Clark, S. B., & Flessner, C. A. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-822201-0.00015-0 · Cited by 4
Decoding Human Behavior with Big Data? Critical, Constructive Input from the Decision Sciences
Katsikopoulos, K., & Canellas, M. C. (2022) · The AI Magazine · doi:10.1609/aimag.v43i1.7381 · Cited by 13
“Accepting such assertions uncritically is tantamount to committing a big data hubris (Lazer et al. 2014), where “small” data and simple rules are considered inherently inferior to big data processed by complex models.”
BackgroundMethodFighting the “System”: A Pilot Project on the Opacity of Algorithms in Political Communication
Bonneau, J., Grondin-Robillard, L., Ménard, M., & Mondoux, A. (2022) · Transforming communications · doi:10.1007/978-3-030-96180-0_5 · Cited by 4
Enhancing Online Epidemic Supervising System by Compartmental and GRU Fusion Model
Ma, J., Wang, X., Wang, Y., et al. (2022) · Mobile Information Systems · doi:10.1155/2022/3303854 · Cited by 4
Heye, T. J., Segeroth, M., Franzeck, F. C., & Vosshenrich, J. (2023) · European Radiology · doi:10.1007/s00330-023-10424-6 · Cited by 5
Rahman, T., Hossain, F., Tonmoy, T. H., Nuvvula, S., Chapman, B. P., Gupta, R., Lover, A., Dinglasan, R., & Stephanie, C. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1279392 · Cited by 4
Yang, S., Chen, J., Liang, R., Wang, Y., & Li, K. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.143652 · Cited by 5
Du, F., & Mao, L. (2024) · Spatial and Spatio-temporal Epidemiology · doi:10.1016/j.sste.2024.100691 · Cited by 5
Ganitidis, T., Αθανασίου, Μ., Mitsis, K., Zarkogianni, K., & Nikita, K. S. (2025) · Journal of Medical Internet Research · doi:10.2196/66919 · Cited by 7
The role of artificial intelligence in predicting disease outbreaks: A multidisciplinary approach
Alfardan, A. A. I., Alharbi, R. F. R., Alrammaal, W., Alharbi, F. S., Almotairi, M. S., Almutairi, M. N., Almutairi, N. S., Hazazi, M. M. M., Alanazi, M. M. M., & Alharbi, F. M. M. (2024) · International Journal of Health Sciences · doi:10.53730/ijhs.v8ns1.15327 · Cited by 6
Digitalization, Cultural Production, Exchange, and Consumption
Shaban, A. (2024) · Springer eBooks · doi:10.1007/978-981-97-9278-8_5 · Cited by 8
Anglin, K. (2024) · AERA Open · doi:10.1177/23328584241303495 · Cited by 11
“The reasons for the failure of Google Flu Trends are not known, but one hypothesis is that changes in Google’s search platform—for example, the incorporation of suggested search terms for users—dramatically changed the nature of the underlying search data (Lazer et al., 2014).”
From A to Google: How Technology Is Impacting Information and Leadership
Davenport, B. (2014) · Journal of Leadership Studies · doi:10.1002/jls.21329 · Cited by 4
Beyond crystal balls: crosscutting solutions in global health to prepare for an unpredictable future
Alonso, W. J., McCormick, B., Miller, M. A., Schuck‐Paim, C., & Asrar, G. (2015) · BMC Public Health · doi:10.1186/s12889-015-2285-1 · Cited by 4
“[16, 24]) and rarely tested [25, 26].”
Thorner, A. R., Cao, B., Jiang, T., Warner, A. J., & Bonis, P. A. (2016) · Open Forum Infectious Diseases · doi:10.1093/ofid/ofw043 · Cited by 4
Martínez‐García, M. Á., & Dinh‐Xuan, A. T. (2016) · European Respiratory Journal · doi:10.1183/13993003.00060-2016 · Cited by 4
“…virus before any other communications media by analysing the millions of related searches undertaken on Google in the days beforehand (at least in its original model in 2008 – 2009; subsequent reports have asserted that Google Flu Trends ’ predictions had sometimes been very inaccurate) [11, 12].”
BackgroundSharples, S., & Houghton, R. J. (2016) · Ergonomics · doi:10.1080/00140139.2016.1151946 · Cited by 4
“In other contexts, this data analytics approach has been used by organisations such as google to predict the onset of flu outbreaks (with varying success – see lazer et al. 2014).”
MethodGollust, S. E., Qin, X., Wilcock, A. D., Baum, L. M., Barry, C. L., Niederdeppe, J., Fowler, E. F., & Karaca‐Mandic, P. (2016) · Medical Care Research and Review · doi:10.1177/1077558716660944 · Cited by 4
Saiani, P. P. (2016) · Revista Colombiana de Sociología · doi:10.15446/rcs.v39n2.58973 · Cited by 6
At the Crossroads: Lessons and Challenges in Computational Social Science
Borge-Holthoefer, J., Moreno, Y., & Yasseri, T. (2016) · Frontiers research topics · doi:10.3389/978-2-88945-021-3 · Cited by 4
Social Media as Mirror of Society
Manzoor, A. (2017) · Advances in business information systems and analytics book series · doi:10.4018/978-1-5225-2031-3.ch007 · Cited by 4
Sensing and Modeling Human Behavior Using Social Media and Mobile Data
Mehrotra, A., & Musolesi, M. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-409548-9.09673-1 · Cited by 6
“In fact, the risk is to derive models that provide a non-realistic description of the reality or they fit the characteristics of very specific scenarios (Lazer et al. 2014).”
BackgroundMaking Online Outbreak Surveillance Work for all
Wang, Y. A., & Barry, M. (2017) · Annals of Global Health · doi:10.1016/j.aogh.2017.09.002 · Cited by 4
“One problem was overfitting: 50 million candidate Google search queries were tested for fit with historic data on influenza activity, so the odds were high that some of the queries would match by chance.(11) Another problem was that Google’s search algorithm as well as user behavior changed over time, so the system ne…”
BackgroundReconfigurable logic embedded architecture of support vector machine linear kernel
Sirkunan, J., Shaikh-Husin, N., Andromeda, T., et al. (2017) · International Conference on Electrical Engineering, Computer Science and Informatics (EECSI) · doi:10.1109/eecsi.2017.8239081 · Cited by 4
“[17], [18] demand short training time for fast adaptation with new user habits and since the characteristics of data change over time.”
BackgroundTriage, Machine Learning, Algorithms, and Becoming the Borg
Schenkel, S. M., & Wears, R. L. (2018) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2018.02.010 · Cited by 5
What Is Statistical Arbitrage?
Lazzarino, M., Berrill, J., & Šević, A. (2018) · Theoretical Economics Letters · doi:10.4236/tel.2018.85063 · Cited by 6
Big Data and Discovery Sciences in Psychiatry
Na, K., Han, C., & Kim, Y. (2019) · Advances in experimental medicine and biology · doi:10.1007/978-981-32-9721-0_1 · Cited by 4
Thoughts on Data Science in business and industry
Steinberg, D. M., & Aronovich, E. (2020) · Applied Stochastic Models in Business and Industry · doi:10.1002/asmb.2510 · Cited by 4
Spirituality and Childbirth: Existential Meanings and Contemporary Practice Realities
Crowther, S. (2019) · Health and Social Care Chaplaincy · doi:10.1558/hscc.37288 · Cited by 5
Algorithm Governance Framework for Media Regulation in Nigerian Media System
Bashir, A. S. (2020) · The Journal of Society & Media · doi:10.26740/jsm.v4n1.p180-198 · Cited by 5
Cybernetic cities: designing and controlling adaptive and robust urban systems
Gershenson, C., Santi, P., & Ratti, C. (2021) · Edward Elgar Publishing eBooks · doi:10.4337/9781789900125.00018 · Cited by 15
“And simply data mining big data can be misleading (Lazer et al. 2014).”
BackgroundReturn on Investment in Machine Learning: Crossing the Chasm between Academia and Business
Mizgajski, J. T., Szymczak, A., Morzy, M., Augustyniak, Ł., Szymański, P., & Żelasko, P. (2020) · Foundations of Computing and Decision Sciences · doi:10.2478/fcds-2020-0015 · Cited by 7
“• real-world examples of over-fitting of ML models [77], 284 J.”
BackgroundChoi, H. K., Choi, W. S., & Han, E. (2021) · Scientific Reports · doi:10.1038/s41598-021-90511-0 · Cited by 4
Legewie, N. M., & Fasang, A. E. (2021) · Edward Elgar Publishing eBooks · doi:10.4337/9781788975544.00014 · Cited by 5
“…users covered in the processes that generate digital trace data, and the search and newsfeed algorithms that digital platforms employ (e.g., Google search, Facebook, or Twitter; Lazer et al. 2014), are constantly shifting and changing; a process Salganik (2018) refers to as ‘drifting’ (see below).”
BackgroundInfluentialTrust Building and Credit Reporting with Big Data in the Digital Age
Peng, W., & Zhu, F. (2021) · Springer eBooks · doi:10.1007/978-3-030-65117-6_29 · Cited by 5
Big Data in Academic Research: Challenges, Pitfalls, and Opportunities
Raubenheimer, J. (2021) · Policy implications of research in education · doi:10.1007/978-3-030-76841-6_1 · Cited by 6
Sultan, S., Acharya, Y., Zayed, O., Elzomour, H., Parodi, J. C., Soliman, O. I. I., & Hynes, N. (2022) · Journal of Endovascular Therapy · doi:10.1177/15266028221102660 · Cited by 4
Faster indicators of chikungunya incidence using Google searches
Miller, S., Preis, T., Mizzi, G., et al. (2022) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0010441 · Cited by 4
Seasonal Patterns and Trends in Dermatoses in Poland
Klimiuk, K. B., Krefta, D., Kołkowski, K., Flisikowski, K., Sokołowska‐Wojdyło, M., & Balwicki, Ł. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19158934 · Cited by 6
“The Google searches do not indicate the actual incidence of symptoms—recommendations are not only influenced by what others searched for [51], but elements of panic and “social contagion” can be present in search behavior [52].”
BackgroundThe lead time and geographical variations of Baidu Search Index in the early warning of COVID-19
Ruan, Y., Huang, T., Zhou, W., Zhu, J., Liang, Q., Zhong, L., Tang, X., Liu, L., Chen, S., & Xie, Y. (2023) · Scientific Reports · doi:10.1038/s41598-023-41939-z · Cited by 4
Ten propositions on machine learning in official statistics
van Delden, A., Burger, J., & Puts, M. (2023) · AStA Wirtschafts- und Sozialstatistisches Archiv · doi:10.1007/s11943-023-00330-0 · Cited by 5
“Google Flu Trends is a well-known illustration of the weakness of a predictive model based on correlations (Lazer et al. 2014).”
Ziakas, P. D., & Mylonakis, E. E. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000462 · Cited by 6
AutoML Insights: Gaining Confidence to Operationalize Predictive Models
Stoica, F., & Stoica, L. F. (2024) · Business, management and economics · doi:10.5772/intechopen.1004861 · Cited by 5
Models, simulation and convergence in the polity: An essay
Loveridge, D. (2015) · Futures · doi:10.1016/j.futures.2015.11.001 · Cited by 4
Promoting of agrotourism on the internet – A lesson from the Visegrad Group countries
Król, K. (2017) · Journal of Agribusiness and Rural Development · doi:10.17306/j.jard.2017.00383 · Cited by 4
Ethnography, Documents, and Big Data: Reflections on Teaching with David Hakken
Shankar, K. (2018) · Anthropology of Work Review · doi:10.1111/awr.12133 · Cited by 4
Design Choices for Automated Disease Surveillance in the Social Web
Magumba, M. A., Nabende, P., & Mwebaze, E. (2018) · Online Journal of Public Health Informatics · doi:10.5210/ojphi.v10i2.9312 · Cited by 4
“…development but poorly on live data is that of that of Google Flu Trends which was reported to have obtained a 0.97 mean correlation with actual data for the 2008-2009 flu American flu season in the initial paper [37] before grossly over-estimating the 2011-2012 and the 2012-2013 seasons [95,96].”
MethodInfluentialGigerenzer, G. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108684354.010 · Cited by 6
Physics-Guided Neural Network with Model Discrepancy Based on Upper Troposphere Wind Prediction
Fukui, K., Tanaka, J., Tomita, T., & Numao, M. (2019) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2019.00078 · Cited by 7
“Representative example is Google Flu Trends, which predict influenza spreading from search engine queries, it overestimated 50% the flu spreading in the future even they have achieved good performance in the validation data [1].”
BackgroundRodríguez‐González, A., Tuñas, J. M., Peces-Barba, D. F., Menasalvas, E., Jaramillo, A., Cotarelo, M., Fernández, A. J. C., Arce, A., & de Miguel, Á. G. (2020) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms49503.2020.00053 · Cited by 4
Arendt, F., & Mestas, M. (2021) · Health Communication · doi:10.1080/10410236.2020.1871168 · Cited by 5
“Albeit that search engine volume has its limitations (Lazer et al., 2014; Nuti et al., 2014), it has successfully been used in previous cross-national research as an indicator for public attention (see Arendt, 2018).”
BackgroundMethodLamata, P. (2020) · Heart and Metabolism · doi:10.31887/hm.2020.82/plamata · Cited by 5
Whatever it takes to understand a central banker — Embedding their words using neural networks
Baumgärtner, M., & Zahner, J. (2025) · Journal of International Economics · doi:10.1016/j.jinteco.2025.104101 · Cited by 5
‘Big data’ and policy learning
Dunleavy, P. (2016) · Policy Press eBooks · doi:10.1332/policypress/9781447329367.003.0009 · Cited by 3
Future Trends in Cloud Computing and Big Data
Mohapatra, S., Paikaray, J., & Samal, N. (2015) · Journal of Computer Sciences and Applications · doi:10.12691/jcsa-3-6-6 · Cited by 6
“In business models where high-level analytics services may be delivered by the Cloud, human expertise cannot be easily replaced by machine learning and Big Data analysis [27]; in certain scenarios, there may be a need for human analysts to remain in the loop [25].”
BackgroundAnalysing spatiotemporal patterns of antibiotics prescriptions
Hutka, L., & Bernard, L. (2026) · Qucosa (Saxon State and University Library Dresden) · doi:10.25368/2026.191 · Cited by 3
Final Remarks on Big Data Analysis and Its Impact on Society and Science
Stefanowski, J., & Japkowicz, N. (2015) · Studies in big data · doi:10.1007/978-3-319-26989-4_13 · Cited by 4
Does Only Size Matter or Is There Still a Place for Single-Center Studies in the Era of Big Data?
Einav, S., & O’Connor, M. (2016) · Anesthesia & Analgesia · doi:10.1213/ane.0000000000001614 · Cited by 4
Quantitative evaluation of public spaces using crowd replication
Hemminki, S., Kuribayashi, K., Konomi, S., Nurmi, P., & Tarkoma, S. (2016) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/2996913.2996946 · Cited by 4
Duncan, H. L., & Auerbach, J. M. (2017) · Journal of Public Health Management and Practice · doi:10.1097/phh.0000000000000506 · Cited by 4
“experts have written reports and articles encouraging action.(4-8) It is in the midst of these discussions that the”
BackgroundThe Limits of Policy Analytics: Early Examples and the Emerging Boundary of Possibilities
Longo, J., & Dobell, A. R. (2018) · Politics and Governance · doi:10.17645/pag.v6i4.1561 · Cited by 7
Weyer, J. (2018) · Springer eBooks · doi:10.1007/978-3-662-57765-3_6 · Cited by 4
Weißenberger, B. E., & Bauch, K. A. (2019) · Springer eBooks · doi:10.1007/978-3-658-24576-4_30 · Cited by 7
Big Spatial Data: lehetőségek, kihívások és tapasztalatok
Jakobi, Á. (2019) · Területi Statisztika · doi:10.15196/ts590101 · Cited by 6
Using Google Analytics to Analyze Users of a Massive Open Online Course
Yamba-Yugsi, M., Luján‐Mora, S., & Pacheco-Romero, H. (2019) · International Conference on Information Systems and Computer Science (INCISCOS) · doi:10.1109/inciscos49368.2019.00051 · Cited by 4
Qualitative Big Data’s Challenges and Solutions: An Organizing Review
Suvivuo, S. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.119 · Cited by 6
“Researchers also tend to work under the “Ideal User Assumption” assuming all the users are operating in good faith and not trying to game the system or engaging in any opportunistic behaviour [21, 23].”
BackgroundFusing Visual Features and Metadata to Detect Flooding in Flickr Images
Jony, R. I., Woodley, A., & Perrin, D. (2020) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta51227.2020.9363418 · Cited by 5
Forecasting the future number of pertussis cases using data from Google Trends
Nann, D., Walker, M. D., Frauenfeld, L., Ferenci, T., & Sulyok, M. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e08386 · Cited by 6
“Although offering great potential, GFT was plagued with problems, primarily due to over fi tting [18].”
BackgroundFlu Detector: Estimating influenza-like illness rates from online user-generated contentPreprint
Lampos, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03494 · Cited by 6
“In late 2015, amidst severe criticism (Olson et al., 2013; Lazer et al., 2014) and bad press due to significant mis-predictions in the past flu seasons, the GFT service was unfortunately discontinued.”
BackgroundMetadata and Relational Architecture: Advancing Arrangement, Agency, and Access with New Methodology
Morton‐Aiken, J. (2017) · Digital Commons - URI (University of Rhode Island) · doi:10.23860/diss-morton-aiken-jenna-2017 · Cited by 6
The Potential of Social Media Intelligence to Improve People’s Lives: Social Media Data for Good
Young, A. Y., & Verhulst, S. G. V. (2017) · doi:10.15868/socialsector.40384 · Cited by 4
Sequence to Sequence with Attention for Influenza Prevalence Prediction using Google TrendsPreprint
Kondo, K., Ishikawa, A., & Kimura, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.02786 · Cited by 4
“[3] also reported the prediction of ILI using Google Flu Trends along with the data disclosed by the Centers for Disease Control and Prevention (CDC)(1).”
MethodUnderstanding and Explaining Automated DecisionsPreprint
Powell, A. B., Joshi, A., Carfantan, P.-M., Bourke, G., Hutchinson, I., & Eichholzer, A. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3309779 · Cited by 4
Detecting and modelling real percolation and phase transitions of information on social mediaPreprint
Xie, J., Meng, F., Sun, J., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.02804 · Cited by 4
“Online social media create large-scale networks that enable information spreading to occur rapidly and widely [1-11] and play an important role in a variety of applications [1,10,12-31].”
BackgroundNIST Big Data Interoperability Framework: volume 4, security and privacy, version 2
NIST Big Data Public Working Group (2018) · doi:10.6028/nist.sp.1500-4r1 · Cited by 4
de Vries, I. (2020) · Digital Repository Service (Northeastern University) · doi:10.17760/d20384390 · Cited by 6
Don’t 'Research Fast and Break Things': On the Ethics of Computational Social SciencePreprint
Leslie, D. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4134621 · Cited by 4
Metadata
- Authors
- Lazer, D., Kennedy, R., King, G., & Vespignani, A.
- Affiliations
- Northeastern University (US)
- Harvard University (US)
- Corresponding author
- Lazer, David
- Venue
- Science
- Published
- 2014-03-14
- Volume
- 343
- Issue
- 6176
- Pages
- 1203-1205
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2068181924
- Semantic Scholar
- 00d681cfff7f0224ace607e515f4ed510e792df9
- Cited by
- 2,523
- References
- 30
- Citations by year
- 2026: 113 · 2025: 101 · 2024: 121 · 2023: 150 · 2022: 168 · 2021: 229 · 2020: 273 · 2019: 261 · 2018: 272 · 2017: 249 · 2016: 271 · 2015: 219 · 2014: 91
- FWCI
- 208.16
- Type
- journal article
- Language
- en
- Open access
- green
- dash.harvard.edu
- Licence
- cc-by
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Epidemiology › Data-Driven Disease Surveillance
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- Google Flu Trends, influenza surveillance, big data, prediction error, data quality
- MeSH terms
- Centers for Disease Control and Prevention, U.S., Data Collection, Data Interpretation, Statistical, Disease Outbreaks, Humans, Influenza, Human, Office Visits, Prognosis, United States, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.