The Parable of Google Flu: Traps in Big Data Analysis
Open accessThe DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index; 5 of the 8 sources its claims cite were checked for support.
Abstract
Large errors in flu prediction were largely avoidable, which offers lessons for the use of big data.
Overview
CoChat Summary
TL;DR
The paper uses the failure of Google Flu Trends (GFT) to illustrate two traps in big data analysis: “big data hubris” and algorithm dynamics. It argues that GFT’s large flu-prediction errors were largely avoidable and that big data should supplement, not replace, traditional data and methods.
Key findings
- In February 2013, GFT predicted more than double the proportion of doctor visits for influenza-like illness (ILI) than the CDC; from 21 August 2011 to 1 September 2013, GFT reported overly high flu prevalence in 100 out of 108 weeks, and it overshot the 2011–2012 season by more than 50%.
- GFT errors were not random: they showed temporal autocorrelation and seasonality, meaning last week’s errors predicted this week’s errors and error direction/magnitude varied with time of year.
- Simple alternatives outperformed GFT: out-of-sample mean absolute error was 0.486 for GFT, 0.311 for lagged CDC, and 0.232 for combined GFT and CDC; all differences were statistically significant at P < 0.05. Even 3-week-old CDC data did better than GFT at projecting current flu prevalence.
- The authors attribute GFT’s problems to overfitting (initial GFT matched 50 million search terms to 1152 data points and missed the nonseasonal 2009 H1N1 pandemic) and to “algorithm dynamics,” including Google search algorithm changes and user behavior changes.
Methods
- Design: policy forum/case study comparing GFT estimates with CDC ILI surveillance data and alternative forecasting models.
- Data/sample: GFT and CDC flu prevalence estimates, including the period from 21 August 2011 to 1 September 2013; the initial GFT methodology reportedly fit 50 million search terms to 1152 data points.
- Main analysis: comparison of GFT alone, lagged CDC data with 52-week seasonality variables, and a combined GFT + CDC model using lagged CDC estimates, lagged GFT error, and seasonality variables; reported mean absolute error and statistical significance; Google Correlate used to examine correlated search terms. Supplementary materials contain data and methods.
Limitations
- The paper states that GFT never documented the 45 search terms used, released examples appear misleading, and Google Correlate is limited to national-level data whereas GFT was developed using regional correlations; therefore the original GFT analyses cannot be replicated from the information provided.
- The paper states that Google cannot make its full data available to outsiders because of privacy issues, though it argues there is no such constraint on derivative, aggregated data.
- Evident from the text: this is a commentary/policy forum rather than a full empirical article, with key data and methods placed in supplementary materials; the quantitative comparison depends on GFT and CDC data and model specifications not fully detailed in the main text.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 2 Not valid · 2 Not checked · 8 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“This ad hoc method of throwing out peculiar search terms failed when GFT completely missed the nonseasonal 2009 infl uenza A–H1N1 pandemic ( 2, 14).”
What the sources say
- Partly supportedcites [2] matched to “Reassessing Google Flu Trends Data for Detection of Seasonal and Pandemic Influenza: A Comparative Epidemiological Study at Three Geographic Scales” (2013) sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says the original GFT model missed the spring 2009 pandemic wave, supporting the claim about its failure to detect the nonseasonal outbreak. It does not say GFT completely missed the entire 2009 pandemic. The Introduction describes automated selection of the original model’s search terms, not an ad hoc method of throwing out peculiar terms. (LLM, conf=0.86)
- Disputedcites [14] S. Cook et al., PLOS ONE 6, e23610 (2011). sourceExperimental — accuracy not yet measured
The Abstract (Methodology/Principal Findings) says the original GFT model underestimated the magnitude of illness during the 2009 H1N1 pandemic, not that it completely missed it. The Abstract (Conclusions) says both models performed well during the pandemic, although the updated model performed better. The Discussion notes that pandemic-specific queries were excluded, but does not describe this as an ad hoc method of throwing out peculiar terms. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Oddly, the few search terms offered in the papers ( 14) do not seem to be strongly related with either GFT or the CDC data (SM)—we surmise that the authors felt an unarticulated need to cloak the actual search terms identifi ed.”
What the sources say
- Disputedcites [14] S. Cook et al., PLOS ONE 6, e23610 (2011). sourceExperimental — accuracy not yet measured
The Results section (‘Search Behavior During pH1N1’; page unavailable) says that all three search queries shown in Figure 3 closely tracked CDC ILINet data before the pandemic, and ‘symptoms of flu’ continued to do so during it. The paper does report that the bronchitis and pneumonia queries underestimated ILINet data during the pandemic, but that does not support the broad claim that the offered terms were not strongly related to the data. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The Internet has opened the way for improving standard surveys, experiments, and health reporting ( 35).”
What the sources say
- Partly supportedcites [35] A. J. Berinsky et al., Polit. Anal. 20, 351–368 (2012). sourceExperimental — accuracy not yet measured
Section 2 describes using an Internet platform to run surveys with embedded experiments, and the Conclusion says MTurk can make experimental research easier and less expensive. The paper does not address improvements to health reporting; the disease scenario in section 4.3.2 is a framing experiment, not health reporting. Page numbers are unavailable in our copy. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although not widely reported until 2013, the new GFT has been persistently overestimating flu prevalence for a much longer time.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“From 21 August 2011 to 1 September 2013, GFT reported overly high fl u prevalence 100 out of 108 weeks.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
3 more claims
Create a free account to see all 14 claims and every source verdict.
- It may state two separate claims as one. 5
- Probably not a substantial claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Google Trends predicted more than twice the number of influenza-like illnesses than the CDC in February 2013 46 .”
From Unrepresentative big surveys significantly overestimated US vaccine uptake (2021) · cites this paper as [46]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was one of the main reasons behind the failure of Google Flu Trends, since the data used for training the model in the first few years was not representative of the trends in subsequent years [15].”
From Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data (2017) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Social data research often relies on counting entities. These entities can be users, links, or messages, and the description is a summary of these counts (Lazer et al., 2014; Salganik, 2017). Yet, simple counts can mislead if it is unclear what is counted and how.”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This unintelligibility is also subject to functional biases (section 4.1), as it depends on the mechanisms available on each social platform (e.g., having a like button, but not a dislike one), as well as to variations in platform algorithms and mechanisms in response to users actions (Lazer et al., 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It also includes temporal validity, which captures to what extent constructs change over time (Howison et al., 2011) and may invalidate previous conclusions about societal and/or platform phenomena; e.g., see the case of Google Flu Trends (Lazer et al., 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 14 MARCH 2014 VOL 343 SCIENCE www.sciencemag.org 1204
“The 2009 version of GFT has weathered other media panics related to the fl u, including the 2005–2006 influenza A/H5N1 ("bird flu") outbreak”
“GFT engineers updated the algorithm in 2009, and this model has run ever since”
Citations
Cites
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Twitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206
Detecting influenza epidemics using search engine query data
Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009) · Nature · doi:10.1038/nature07634 · Cited by 4,295
Cohen, D. (1998) · The Women's Review of Books · doi:10.2307/4022859 · Cited by 4,261
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Big Data : The Management Revolution
マカフィー, ア., ブリニョルフソン, エ., & 有賀, 裕. (2013) · Harvard business review · Cited by 3,549
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
Self-citationPredicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment
Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14009 · Cited by 2,649
Multiscale mobility networks and the spatial spreading of infectious diseases
Balcan, D., Colizza, V., Gonçalves, B., Hu, H., Ramasco, J. J., & Vespignani, A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0906910106 · Cited by 1,380
Self-citationPredicting consumer behavior with web search
Goel, S., Hofman, J. M., Lahaie, S., Pennock, D. M., & Watts, D. J. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1005962107 · Cited by 636
Research works printed but not linked: 7
- A. McAfee, E. Brynjolfsson, Harv. Bus. Rev. 90, 60 (2012).
- AAAI Conference on Weblogs and Social Media, Atlanta, Georgia, 11 to 15 July 2010 (Association for Advancement of Artifi cial Intelligence, 2010), pp. 178.
- P. Copeland et al., Int. Soc. Negl. Trop. Dis. 2013, 3
- I. M. Hall et al., Epidemiol. Infect. 135, 372 (2007).
- E. Mustafaraj, P. Metaxas, in Proceedings of the Web- Sci10, Raleigh, NC, 26 and 27 April 2010 (Web Science Trust, 2010); http://journal.webscience.org/317/.
- J. Ratkiewicz et al., in Proceedings of 5th International AAAI Conference on Weblogs and Social Media, San Francisco, CA, 7 to 11 August 2011 (AAAI, 2011), p. 297–304.
- 2013; http://chronicle.com/article/Researchers-Struggle- to-Secure/141591/.
Cited by
Items 1,901–2,000 of 2,513
Holmes, M. H. (2016) · Texts in computational science and engineering · doi:10.1007/978-3-319-30256-0_9 · Cited by 1
Big Data, Big Expectations, and Big Judgements
Brophy, J. M. (2022) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2022.07.001 · Cited by 1
Challenges and Limitations of Social Data Analysis Approaches
Mondal, S., & Rehena, Z. (2022) · Smart computing and intelligence · doi:10.1007/978-981-19-1408-9_14 · Cited by 2
Hygiene, J. O. P. M. A. (2022) · PubMed · doi:10.15167/2421-4248/jpmh2022.63.2s2 · Cited by 2
How Could Machine Learning Help Healthcare Informatics Predict Coronavirus?
Strang, K. D. (2022) · Advances in healthcare information systems and administration book series · doi:10.4018/978-1-6684-5499-2.ch002 · Cited by 1
Detecting Personal Health Mentions from Social Media Using Supervised Machine Learning
Yin, Z., Ni, C., Fabbri, D., Rosenbloom, S. T., & Malin, B. (2022) · Cognitive informatics in biomedicine and healthcare · doi:10.1007/978-3-031-07696-1_12 · Cited by 1
Butt, Z. A. (2023) · Elsevier eBooks · doi:10.1016/b978-0-443-15299-3.00010-5 · Cited by 2
Validity Analysis of Network Big Data
Wang, P., Lv, H., Zheng, X., et al. (2023) · Journal of Web Engineering · doi:10.13052/jwe1540-9589.2234 · Cited by 1
“The invalid data makes incorrect results, undermining the validity of the analysis, which is the reason why Google failed to predict influenza in 2013 [17].”
Result‘Big data’ and policy learning
Dunleavy, P. (2016) · Policy Press eBooks · doi:10.56687/9781447329381-013 · Cited by 1
Vergara, J. V., Rodriguez, M. Y., Phillips, J., Dohler, E., Villodas, M. L., Wilson, A. B., & Joseph, K. (2023) · Urban Studies · doi:10.1177/00420980231189403 · Cited by 1
“Most popular machine learning (and statistical) methods can identify spurious correlations between features and outcomes that can, down the road, lead to erroneous predictions (Lazer et al., 2014) and/or misinformed policy decisions (Radford and Joseph, 2020).”
BackgroundPan, X., Hounye, A. H., Zhao, Y., Cao, C., Wang, J., Abidi, M. V., Hou, M., Xiong, L., & Chai, X. (2023) · Journal of Medical Internet Research · doi:10.2196/44795 · Cited by 2
Beyond Quantity: Research with Subsymbolic AI
Sudmann, A., Echterhölter, A., Ramsauer, M., Retkowski, F., Schröter, J., & Waibel, A. (2023) · AI critique · doi:10.1515/9783839467664 · Cited by 14
Application of Deep Learning in Social Network Public Opinion Sentiment
Wu, L. (2023) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001423520146 · Cited by 1
Dijital Tedarik Zinciri Yönetimi ve Büyük Veri
Derici, S. (2024) · Akademisyen Kitabevi eBooks · doi:10.37609/akya.2946 · Cited by 3
Big data and artificial intelligence for pandemic preparedness
Butt, Z. A. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-21598-8.00005-1 · Cited by 3
Prognosis of Dengue Incidences in India Using LSTM with Multivariate Data
Sharma, S., & Gupta, Y. K. (2024) · Lecture notes in networks and systems · doi:10.1007/978-981-99-9040-5_34 · Cited by 1
Risiken und Gefahren der ‚Künstlichen‘ ‚Intelligenz‘
Kreissl, R., & von Laufenberg, R. (2024) · Springer eBooks · doi:10.1007/978-3-658-43521-9_10 · Cited by 1
Imputing Unreported Hate Crimes Using Google Search Data
Dharmapala, D., & Huq, A. Z. (2024) · Journal of law & empirical analysis. · doi:10.1177/2755323x241274653 · Cited by 1
“It also re fl ects endogenous changes by Google ’ s search algorithm (Lazer et al., 2014).”
Pathogen stress heightens sensorimotor dimensions in the human collective semantic space
Fu, Z., Chen, H., Liu, Z., Sun, M., Liu, Z., & Bi, Y. (2025) · Communications Psychology · doi:10.1038/s44271-024-00183-5 · Cited by 1
Nardello, C., Iannarilli, L., & Giaroli, E. (2024) · Progress in IS · doi:10.1007/978-3-031-73514-1_7 · Cited by 2
Abat-Roy, V. (2025) · Recherches qualitatives · doi:10.7202/1119816ar · Cited by 1
Estimating The Size Of The Secondary Real Estate Market Based On Internet Data Sources
Beręsewicz, M. (2014) · Folia Oeconomica Stetinensia · doi:10.1515/foli-2015-0012 · Cited by 1
An Approach to Derive User Preferences from Multiple-Choice Questions in Online Reviews
Tilly, R. (2015) · AIS Electronic Library (AISeL) · Cited by 2
Studying Social Network Sites via computational methods
Spiliotopoulos, T. (2015) · International Conference on Research Challenges in Information Science (RCIS) · doi:10.1109/rcis.2015.7128912 · Cited by 1
“For example, recent research identifies two big “traps” in big-data analysis [40].”
BackgroundInfluentialScientific Big Data Analytics by HPC
Lippert, T., Mallmann, D., & Riedel, M. (2016) · JuSER (Forschungszentrum Jülich) · Cited by 1
Le risorse e il controllo. I big data oltre il mito
Saiani, P. P. (2016) · SOCIOLOGIA E RICERCA SOCIALE · doi:10.3280/sr2016-109004 · Cited by 1
Google Flu Trends y los datos masivos: ¿extrapolable al ébola?
López, P. J. L. (2016) · TECHNO REVIEW International Technology Science and Society Review /Revista Internacional de Tecnología Ciencia y Sociedad · doi:10.37467/gka-revtechno.v5.465 · Cited by 1
Big Data Management in Financial Services
Vidal-García, J., & Vidal, M. (2016) · Advances in data mining and database management book series · doi:10.4018/978-1-5225-0182-4.ch014 · Cited by 1
A Study on the Efficient Estimation of the Payment Intention in the Mail Order Industry
Takahashi, M., Azuma, H., & Tsuda, K. (2016) · Procedia Computer Science · doi:10.1016/j.procs.2016.08.154 · Cited by 1
Chapter 2. Experimental research: Problems and opportunities in the big-data era
Cremers, H. R. (2017) · John Benjamins Publishing Company eBooks · doi:10.1075/z.210.02cre · Cited by 1
Forecasting Australian port throughput: Lessons and Pitfalls in the era of Big Data
Tyshetskiy, Y., Banerjee, S., Mathews, G., & Vitsounis, T. K. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.51235 · Cited by 1
Research on application model of big data technology in the A1A2 system
Zhang, G., Sun, Y., Li, P., et al. (2017) · IEEE International Conference on Big Data Analysis (ICBDA) · doi:10.1109/icbda.2017.8078870 · Cited by 1
“[23] Lazer D, Kennedy R, King G, Vespignani A.”
BackgroundBrophy, J. M. (2017) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2017.10.015 · Cited by 1
Modelling prediction of unemployment statistics using web technologies
Popescu, M. (2017) · HOLISTICA – Journal of Business and Public Administration · doi:10.1515/hjbpa-2017-0023 · Cited by 1
Distance Education in Romanian Higher Education
Ene, A. B. (2018) · HOLISTICA – Journal of Business and Public Administration · doi:10.1515/hjbpa-2018-0005 · Cited by 1
Big Data Approaches to the Study of Digital Media
Schroeder, R., & Cowls, J. (2018) · Springer eBooks · doi:10.1007/978-94-024-1202-4_13-1 · Cited by 1
Cao, L. (2018) · Data analytics · doi:10.1007/978-3-319-95092-1_2 · Cited by 2
Big Data as an Alternative to Surveys in Quantitative Social Research
Chylíková, J. (2018) · Naše společnost · doi:10.13060/1214438x.2018.1.16.416 · Cited by 1
Privacy in Geospatial Applications and Location-Based Social Networks
Bilogrevic, I. (2018) · Springer eBooks · doi:10.1007/978-3-319-98161-1_8 · Cited by 2
Symbolic Analysis of Machine Behaviour and the Emergence of the Machine Language
Ritt, R., & O'Leary, P. L. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04070-3_24 · Cited by 1
Politics and Epistemology of Big Data: A Critical Assessment
Numerico, T. (2019) · Philosophical studies series · doi:10.1007/978-3-030-01800-9_8 · Cited by 3
Big Data Challenges from a Public Health Informatics Perspective
Birnbaum, D. (2019) · Lecture notes in bioengineering · doi:10.1007/978-3-030-06109-8_4 · Cited by 1
Methodological Transparency and Big Data: A Critical Comparative Analysis of Institutionalization
Sanfilippo, M. R., & McCoy, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-15742-5_5 · Cited by 1
Social Media Sensing Framework for Population Health
Esperança, Á., Miled, Z. B., & Mahoui, M. (2019) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc.2019.8666534 · Cited by 1
“Previous work [10] shows that relying on social media for health data sensing can lead to prediction errors.”
BackgroundAn Analysis of the Value of Data Ecosystem Tools for Industry 4.0
Page, T. (2019) · International Journal of Innovation in the Digital Economy · doi:10.4018/ijide.2019100102 · Cited by 1
Kinder-Kurlanda, K. E. (2019) · Medienkulturen im digitalen Zeitalter · doi:10.1007/978-3-658-27511-2_6 · Cited by 1
The 4 R’s – Reason, REDCAP, Review and Research - In a Large Healthcare Organization
Bain, C., Gilbert, A., Jomon, B., Thompson, R., Kelly, D., & Manus, C. M. (2015) · Health Informatics - An International Journal · doi:10.5121/hiij.2015.4402 · Cited by 1
Questões Epistemológicas em Mineração de Dados Educacionais
Bispo, E. L. (2019) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/cbie.sbie.2019.1541 · Cited by 4
“Entretanto, questões epistemológicas envolvendo algumas asserções científicas em relação ao Big Data, por exemplo, existem na literatura [Leonelli 2014, Boyd e Crawford 2012].”
BackgroundМальцева, А. В., Matveev, M. S., & Moiseeva, M. B. (2019) · Siberian Socium · doi:10.21684/2587-8484-2019-3-3-74-84 · Cited by 1
Paolillo, J. C. (2019) · International Conference on Social Media and Society (SMSociety) · doi:10.1145/3328529.3328543 · Cited by 1
Reinforcement Learning Augmented Optimization for Smart Mobility
Overko, R., Ordóñez-Hurtado, R., Zhuk, S. M., & Shorten, R. N. (2019) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9029961 · Cited by 3
‘Watch the Flu’: A Tweet Monitoring Tool for Epidemic Intelligence of Influenza in Australia
Jin, B., Joshi, A., Sparks, R. S., et al. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i09.7095 · Cited by 1
“Past work using search query statistics to predict influenza infection counts has been criticised for over-estimating the counts (Lazer et al. 2014).”
BackgroundUsing Google Search Trends to Estimate Global Patterns in Learning
Arslan, S., Tiwari, M., & Piech, C. (2020) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3386527.3405913 · Cited by 1
“We add the important cautionary note: though this data is large in quantity, it is not the case that "big data" is a panacea for challenges in estimating literacy [26].”
BackgroundProduct Customer Demand Mining and Its Functional Attribute Configuration Driven by Big Data
Liu, D., Huang, X., & Huang, K. (2020) · Communications in computer and information science · doi:10.1007/978-981-15-7981-3_11 · Cited by 1
Mackenzie, J. S., & Jeggo, M. H. (2019) · MDPI eBooks · doi:10.3390/books978-3-03921-296-5 · Cited by 4
Algoritmos, viralização e contágio psíquico: o agravamento da industrialização do espírito
Contrera, M. S., & Torres, L. (2020) · Texto Digital · doi:10.5007/1807-9288.2020v16n2p93 · Cited by 1
Sosyal medyada otomatik halk sağlığı takibi: Güncel bir derleme
Küçük, D., Arıcı, N., & Küçük, E. E. (2021) · Ömer Halisdemir Üniversitesi Mühendislik Bilimleri Dergisi · doi:10.28948/ngumuh.778948 · Cited by 2
Promoting Resilience in the Face of Fundamental Uncertainty
Jani, A. R. (2021) · IntechOpen eBooks · doi:10.5772/intechopen.97816 · Cited by 1
Digital Media and Collective Action
Jungherr, A. M., Rivero, G., & Gayo-Avello, D. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108297820.005 · Cited by 1
Dogo, E. M., Makaba, T., Afolabi, O. J., & Ajibo, C. A. (2021) · EAI/Springer Innovations in Communication and Computing · doi:10.1007/978-3-030-66042-0_4 · Cited by 5
Der digital turn in Public Health. ePublic Health und Digital Public Health Studies
Urban, M. (2021) · Sozialwissenschaftliche Gesundheitsforschung · doi:10.1007/978-3-658-30377-8_10 · Cited by 1
Katsikopoulos, K. V. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108891325.020 · Cited by 1
Google Search in Exchange Rate Models: Hype or Hope?
Herzog, B., & dos Santos, L. (2021) · Journal of risk and financial management · doi:10.3390/jrfm14110512 · Cited by 3
Wu, J., He, J., & Christakos, G. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-816341-2.00010-1 · Cited by 1
Vallero, D. A. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-821953-9.00014-3 · Cited by 1
Research Ethics in Data: New Technologies, New Challenges
Combe, C. G. (2020) · Springer eBooks · doi:10.1007/978-3-030-16759-2_13 · Cited by 1
Use of artificial intelligence for predicting infectious disease
Kang, S., & Lee, D.-H. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-91907-4.00001-7 · Cited by 1
Act Now: A Call to Action for Scholarly Preparation in Mathematics Among Nurse Scientists
Park, C. S. (2018) · DergiPark (Istanbul University) · Cited by 1
Chasing the Wrong Cloud: Mapping the 2019 Vaping Epidemic Using Data from Social Media
Gera, P., & Ciampaglia, G. L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17114-7_1 · Cited by 1
Labour law in front of digital ecenomy challenges
Jašarević, S., & Božičić, D. (2021) · Zbornik radova Pravnog fakulteta Novi Sad · doi:10.5937/zrpfns55-29667 · Cited by 1
AI Techniques for Forecasting Epidemic Dynamics: Theory and Practice
Adiga, A., Lewis, B. L., Levin, S. A., MARATHE, M. V., Poor, H. V., Ravi, S. S., Rosenkrantz, D. J., Stearns, R. E., Venkatramanan, S., Vullikanti, A. K. S., et al. (2022) · Springer eBooks · doi:10.1007/978-3-031-08506-2_9 · Cited by 5
Advances in the Application of Biological Big Data in Medicine
Luo, Z. (2022) · Academic Journal of Medicine & Health Sciences · doi:10.25236/ajmhs.2022.030204
Risk Analysis in Big DataPreprint
Bakdash, J. Z., & Marusich, L. R. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2641726 · Cited by 2
“Data: Quality and Sources Many of the analysis challenges for big data are not unique, but are pertinent to analysis of all data (Lazer et al. 2014; Silver 2015).”
Shen, B. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1403.7570 · Cited by 2
“However, how to deal with big data to provide big insights and make a big impact is still an open question [39].”
BackgroundKoplenig, A., & Müller-Spitzer, C. (2015) · arXiv (Cornell University) · Cited by 1
Can Big Data Forecast North Korean Military AggressionPreprint
Kim, Y. H., Kang, H.-G., & Lee, J. K. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2490693
News outlets in social media: aggression in comments
Holt, L. G. (2016) · Rowan Digitals Works (Rowan University) · Cited by 2
Anonymity and Online Search: Measuring the Privacy Impact Of Google’s 2012 Privacy Policy ChangePreprint
Cooper, J. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2909148 · Cited by 1
News Analytics for Global Infectious Disease Surveillance
Ghosh, S. (2017) · VTechWorks (Virginia Tech) · Cited by 1
Big Data in Political Science Research
Anastasopoulos, L. J., & Poole, K. T. (2017) · Oxford University Press eBooks · doi:10.1093/obo/9780199756223-0232 · Cited by 1
How Should One Study Clandestine Activities: Crimes, Tax Fraud, and Other “Dark” Economic Behavior?
Prinz, A. L. (2018) · doi:10.1002/9781119155713.ch2 · Cited by 1
Brown, N. J. L., & Coyne, J. (2018) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/dursw · Cited by 1
Modeling Zika Virus Spread in Colombia Using Google Search Queries and Logistic Power ModelsPreprint
Brown, M., Cain, C., Whitfield, J., Ding, E., Valle, S. Y. D., & Manore, C. A. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/365155 · Cited by 1
Kusiak, C. (2021) · ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · doi:10.7275/12681214 · Cited by 1
Same Influenza, Different Responses: Social Media Can Sense a Regional Spectrum of SymptomsPreprint
Shan, S., Jia, Y., & Zhao, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.01778 · Cited by 1
Big Data for Sociological Research
Radford, J. P., & Lazer, D. M. J. (2019) · Wiley eBooks · doi:10.1002/9781119429333.ch24 · Cited by 4
Self-citationMachine Learning in Population and Public HealthPreprint
Mhasawade, V., Zhao, Y., & Chunara, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07278 · Cited by 3
“For example, recent work on how individual-level syndromic reports or passive data from individuals relates to microbiologic confirmation [14], aims to address early challenges highlighted in using data such as Google searches to predict influenza [29].”
BackgroundLearning from pandemics: using extraordinary events can improve disease now-casting modelsPreprint
Mesquita, S., Vieira, C. H., Perfeito, L., & Gonçalves-Sá, J. (2021) · medRxiv · doi:10.1101/2021.01.18.21250056 · Cited by 1
Nicodemo, C., & Satorra, A. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3648799 · Cited by 1
Conditional Meta-Network for Blind Super-Resolution with Multiple Degradations.Preprint
Yin, G., Wang, W., Yuan, Z., et al. (2021) · arXiv (Cornell University) · Cited by 1
Predicting Latent Constructs from Passive Datasets – Significance and OpportunitiesPreprint
Sekar, A., & Rai, V. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3285417 · Cited by 1
Addressing Fairness, Bias and Class Imbalance in Machine Learning: the FBI-lossPreprint
Ferrari, E., & Bacciu, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.06345 · Cited by 12
“…the possibility to collect large volumes of data, this concern fade out giving way to what is a posteriori known as the ”big data hubris”, i.e. ’the implicit assumption that big data are a substitute for, rather than a supplement to, traditional data collection and analysis’ (Lazer et al., 2014).”
BackgroundInferring untrained complex dynamics of delay systems using an adapted echo state network.Preprint
Goldmann, M., Mirasso, C. R., Fischer, I., et al. (2021) · arXiv (Cornell University) · Cited by 1
Jiménez-Pérez, G., Acosta, J., Alcaine, A., & Cámara, Ó. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12996 · Cited by 15
“Finally, the black-box nature of the models, where most works cannot guarantee that their predictions are not the result from spurious or acausal relations within the data or data leakage [42].”
BackgroundThe Evolution of the COVID-19 Pandemic Through the Lens of Google SearchesPreprint
Marty, R., Maqueda, M. R., Khan, N., & Reichert, A. R. (2022) · Research Square · doi:10.21203/rs.3.rs-1551521/v1 · Cited by 1
Position Papers of the 2018 Federated Conference on Computer Science and Information Systems
(2018) · Annals of Computer Science and Information Systems · doi:10.15439/978-83-949419-8-7 · Cited by 1
Okunoye, B. (2020) · doi:10.31219/osf.io/5j4nk · Cited by 1
A proposal to improve research in AI algorithm and data governancePreprint
Braga, J., Regateiro, F. S., Stiubiener, I., & Braga, J. C. (2022) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/sr7kt · Cited by 1
Untapped Potential: Designed Digital Trace Data in Online Survey ExperimentsPreprint
Macke, E., Daviss, C., & Williams-Baron, E. (2022) · doi:10.31235/osf.io/frhj6 · Cited by 1
Metadata
- Authors
- Lazer, D., Kennedy, R., King, G., & Vespignani, A.
- Affiliations
- Northeastern University (US)
- Harvard University (US)
- Corresponding author
- Lazer, David
- Venue
- Science
- Published
- 2014-03-14
- Volume
- 343
- Issue
- 6176
- Pages
- 1203-1205
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2068181924
- Semantic Scholar
- 00d681cfff7f0224ace607e515f4ed510e792df9
- Cited by
- 2,523
- References
- 30
- Citations by year
- 2026: 113 · 2025: 101 · 2024: 121 · 2023: 150 · 2022: 168 · 2021: 229 · 2020: 273 · 2019: 261 · 2018: 272 · 2017: 249 · 2016: 271 · 2015: 219 · 2014: 91
- FWCI
- 208.16
- Type
- journal article
- Language
- en
- Open access
- green
- dash.harvard.edu
- Licence
- cc-by
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Epidemiology › Data-Driven Disease Surveillance
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- Google Flu Trends, influenza surveillance, big data, prediction error, data quality
- MeSH terms
- Centers for Disease Control and Prevention, U.S., Data Collection, Data Interpretation, Statistical, Disease Outbreaks, Humans, Influenza, Human, Office Visits, Prognosis, United States, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.