The Parable of Google Flu: Traps in Big Data Analysis
Open accessThe DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index; 5 of the 8 sources its claims cite were checked for support.
Abstract
Large errors in flu prediction were largely avoidable, which offers lessons for the use of big data.
Overview
CoChat Summary
TL;DR
The paper uses the failure of Google Flu Trends (GFT) to illustrate two traps in big data analysis: “big data hubris” and algorithm dynamics. It argues that GFT’s large flu-prediction errors were largely avoidable and that big data should supplement, not replace, traditional data and methods.
Key findings
- In February 2013, GFT predicted more than double the proportion of doctor visits for influenza-like illness (ILI) than the CDC; from 21 August 2011 to 1 September 2013, GFT reported overly high flu prevalence in 100 out of 108 weeks, and it overshot the 2011–2012 season by more than 50%.
- GFT errors were not random: they showed temporal autocorrelation and seasonality, meaning last week’s errors predicted this week’s errors and error direction/magnitude varied with time of year.
- Simple alternatives outperformed GFT: out-of-sample mean absolute error was 0.486 for GFT, 0.311 for lagged CDC, and 0.232 for combined GFT and CDC; all differences were statistically significant at P < 0.05. Even 3-week-old CDC data did better than GFT at projecting current flu prevalence.
- The authors attribute GFT’s problems to overfitting (initial GFT matched 50 million search terms to 1152 data points and missed the nonseasonal 2009 H1N1 pandemic) and to “algorithm dynamics,” including Google search algorithm changes and user behavior changes.
Methods
- Design: policy forum/case study comparing GFT estimates with CDC ILI surveillance data and alternative forecasting models.
- Data/sample: GFT and CDC flu prevalence estimates, including the period from 21 August 2011 to 1 September 2013; the initial GFT methodology reportedly fit 50 million search terms to 1152 data points.
- Main analysis: comparison of GFT alone, lagged CDC data with 52-week seasonality variables, and a combined GFT + CDC model using lagged CDC estimates, lagged GFT error, and seasonality variables; reported mean absolute error and statistical significance; Google Correlate used to examine correlated search terms. Supplementary materials contain data and methods.
Limitations
- The paper states that GFT never documented the 45 search terms used, released examples appear misleading, and Google Correlate is limited to national-level data whereas GFT was developed using regional correlations; therefore the original GFT analyses cannot be replicated from the information provided.
- The paper states that Google cannot make its full data available to outsiders because of privacy issues, though it argues there is no such constraint on derivative, aggregated data.
- Evident from the text: this is a commentary/policy forum rather than a full empirical article, with key data and methods placed in supplementary materials; the quantitative comparison depends on GFT and CDC data and model specifications not fully detailed in the main text.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 2 Not valid · 2 Not checked · 8 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“This ad hoc method of throwing out peculiar search terms failed when GFT completely missed the nonseasonal 2009 infl uenza A–H1N1 pandemic ( 2, 14).”
What the sources say
- Partly supportedcites [2] matched to “Reassessing Google Flu Trends Data for Detection of Seasonal and Pandemic Influenza: A Comparative Epidemiological Study at Three Geographic Scales” (2013) sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says the original GFT model missed the spring 2009 pandemic wave, supporting the claim about its failure to detect the nonseasonal outbreak. It does not say GFT completely missed the entire 2009 pandemic. The Introduction describes automated selection of the original model’s search terms, not an ad hoc method of throwing out peculiar terms. (LLM, conf=0.86)
- Disputedcites [14] S. Cook et al., PLOS ONE 6, e23610 (2011). sourceExperimental — accuracy not yet measured
The Abstract (Methodology/Principal Findings) says the original GFT model underestimated the magnitude of illness during the 2009 H1N1 pandemic, not that it completely missed it. The Abstract (Conclusions) says both models performed well during the pandemic, although the updated model performed better. The Discussion notes that pandemic-specific queries were excluded, but does not describe this as an ad hoc method of throwing out peculiar terms. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Oddly, the few search terms offered in the papers ( 14) do not seem to be strongly related with either GFT or the CDC data (SM)—we surmise that the authors felt an unarticulated need to cloak the actual search terms identifi ed.”
What the sources say
- Disputedcites [14] S. Cook et al., PLOS ONE 6, e23610 (2011). sourceExperimental — accuracy not yet measured
The Results section (‘Search Behavior During pH1N1’; page unavailable) says that all three search queries shown in Figure 3 closely tracked CDC ILINet data before the pandemic, and ‘symptoms of flu’ continued to do so during it. The paper does report that the bronchitis and pneumonia queries underestimated ILINet data during the pandemic, but that does not support the broad claim that the offered terms were not strongly related to the data. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The Internet has opened the way for improving standard surveys, experiments, and health reporting ( 35).”
What the sources say
- Partly supportedcites [35] A. J. Berinsky et al., Polit. Anal. 20, 351–368 (2012). sourceExperimental — accuracy not yet measured
Section 2 describes using an Internet platform to run surveys with embedded experiments, and the Conclusion says MTurk can make experimental research easier and less expensive. The paper does not address improvements to health reporting; the disease scenario in section 4.3.2 is a framing experiment, not health reporting. Page numbers are unavailable in our copy. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although not widely reported until 2013, the new GFT has been persistently overestimating flu prevalence for a much longer time.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“From 21 August 2011 to 1 September 2013, GFT reported overly high fl u prevalence 100 out of 108 weeks.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
3 more claims
Create a free account to see all 14 claims and every source verdict.
- It may state two separate claims as one. 5
- Probably not a substantial claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Google Trends predicted more than twice the number of influenza-like illnesses than the CDC in February 2013 46 .”
From Unrepresentative big surveys significantly overestimated US vaccine uptake (2021) · cites this paper as [46]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was one of the main reasons behind the failure of Google Flu Trends, since the data used for training the model in the first few years was not representative of the trends in subsequent years [15].”
From Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data (2017) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Social data research often relies on counting entities. These entities can be users, links, or messages, and the description is a summary of these counts (Lazer et al., 2014; Salganik, 2017). Yet, simple counts can mislead if it is unclear what is counted and how.”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This unintelligibility is also subject to functional biases (section 4.1), as it depends on the mechanisms available on each social platform (e.g., having a like button, but not a dislike one), as well as to variations in platform algorithms and mechanisms in response to users actions (Lazer et al., 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It also includes temporal validity, which captures to what extent constructs change over time (Howison et al., 2011) and may invalidate previous conclusions about societal and/or platform phenomena; e.g., see the case of Google Flu Trends (Lazer et al., 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 14 MARCH 2014 VOL 343 SCIENCE www.sciencemag.org 1204
“The 2009 version of GFT has weathered other media panics related to the fl u, including the 2005–2006 influenza A/H5N1 ("bird flu") outbreak”
“GFT engineers updated the algorithm in 2009, and this model has run ever since”
Citations
Cites
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Twitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206
Detecting influenza epidemics using search engine query data
Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009) · Nature · doi:10.1038/nature07634 · Cited by 4,295
Cohen, D. (1998) · The Women's Review of Books · doi:10.2307/4022859 · Cited by 4,261
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Big Data : The Management Revolution
マカフィー, ア., ブリニョルフソン, エ., & 有賀, 裕. (2013) · Harvard business review · Cited by 3,549
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
Self-citationPredicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment
Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14009 · Cited by 2,649
Multiscale mobility networks and the spatial spreading of infectious diseases
Balcan, D., Colizza, V., Gonçalves, B., Hu, H., Ramasco, J. J., & Vespignani, A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0906910106 · Cited by 1,380
Self-citationPredicting consumer behavior with web search
Goel, S., Hofman, J. M., Lahaie, S., Pennock, D. M., & Watts, D. J. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1005962107 · Cited by 636
Research works printed but not linked: 7
- A. McAfee, E. Brynjolfsson, Harv. Bus. Rev. 90, 60 (2012).
- AAAI Conference on Weblogs and Social Media, Atlanta, Georgia, 11 to 15 July 2010 (Association for Advancement of Artifi cial Intelligence, 2010), pp. 178.
- P. Copeland et al., Int. Soc. Negl. Trop. Dis. 2013, 3
- I. M. Hall et al., Epidemiol. Infect. 135, 372 (2007).
- E. Mustafaraj, P. Metaxas, in Proceedings of the Web- Sci10, Raleigh, NC, 26 and 27 April 2010 (Web Science Trust, 2010); http://journal.webscience.org/317/.
- J. Ratkiewicz et al., in Proceedings of 5th International AAAI Conference on Weblogs and Social Media, San Francisco, CA, 7 to 11 August 2011 (AAAI, 2011), p. 297–304.
- 2013; http://chronicle.com/article/Researchers-Struggle- to-Secure/141591/.
Cited by
Items 1,601–1,700 of 2,513
Computational PsychologyPreprint
Kosiński, M. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/tkhg4 · Cited by 3
Braga, J., Regateiro, F. S., Stiubiener, I., & Braga, J. C. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/wexjr · Cited by 6
Ai for the People? Embedding Ai Ethics in Hr and People Analytics ProjectsPreprint
Bar-Gil, O., Ron, T. H., & Czerniak, O. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4491697 · Cited by 4
Gonçalves, A. R., Pico, J. C., Hu, Y., Schlessinger, D., Greene, J., O’suilleabhain, L., Clancy, H., Vollmer, M., Liu, V. X., Bates, T., et al. (2025) · Research Square · doi:10.1101/2025.05.14.25327606 · Cited by 6
Tracking Protests Using Geotagged Flickr Photographs
Alanyali, M., Preis, T., & Moat, H. S. (2016) · PLoS ONE · doi:10.1371/journal.pone.0150466 · Cited by 8
Incorporating connectivity among Internet search data for enhanced influenza-like illness tracking
Ning, S., Hussain, A., & Wang, Q. (2024) · PLoS ONE · doi:10.1371/journal.pone.0305579 · Cited by 3
“In particular, the ARGO framework (AutoRegression with GOogle search data) [61] provides robust and highly accurate ILI estimates at the national level by directly addressing the limitations of GFT.”
Patient-Oriented Questionnaires and Machine Learning for Rare Disease Diagnosis: A Systematic Review
Brauner, L. E., Yao, Y., Grigull, L., & Klawonn, F. (2024) · Journal of Clinical Medicine · doi:10.3390/jcm13175132 · Cited by 4
From data to decisions: The vital role of digital epidemiology for strengthening public health
Hussain, S. S. A., Khwairakpam, P., Sharma, A. K., & Yadav, C. P. (2026) · Digital Health · doi:10.1177/20552076261436279 · Cited by 4
The Multiplicity of Algorithms
Bucher, T. (2018) · Oxford University Press eBooks · doi:10.1093/oso/9780190493028.003.0002 · Cited by 4
Metrics and Evaluation Tools for Communicable and Non-communicable Diseases
Lu, L. (2020) · Springer eBooks · doi:10.1007/978-3-030-59687-3_12 · Cited by 3
Predicting the Results of the 2019 Indonesian Presidential Election with Google Trends
Harkan, A. A., & Eriyanto (2021) · Advances in Social Science, Education and Humanities Research/Advances in social science, education and humanities research · doi:10.2991/assehr.k.210531.001 · Cited by 3
Behavioural OR: Recent developments and future perspectives
Kunc, M. H., & Katsikopoulos, K. V. (2022) · Springer eBooks · doi:10.1007/978-3-030-96935-6_22 · Cited by 3
‘Big data’ and policy learning
Dunleavy, P. (2016) · Policy Press eBooks · doi:10.51952/9781447329381.ch008 · Cited by 4
Factors Affecting the Reliability of Information: The Case of ChatGPT
Morato, J., Nafría, J. M. D., & Sánchez-Cuadrado, S. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-48930-3_12 · Cited by 4
Role of artificial intelligence and machine learning in women’s health
Rawat, S., Joshi, P., Praveen, G., & Saxena, J. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-21889-7.00006-3 · Cited by 3
Mitigating biases in big mobility data: a case study of monitoring large-scale transit systems
Wang, F., Ban, X. J., Chen, P., Liu, C., & Zhao, R. (2024) · Transportation Letters · doi:10.1080/19427867.2024.2379703 · Cited by 4
Big Data (R)evolution in Geography: Complexity Modelling in the Last Two Decades
Pérez, L., & Sengupta, R. (2024) · Geography Compass · doi:10.1111/gec3.70009 · Cited by 3
Moneva, A., & Leukfeldt, R. (2024) · European Journal of Criminology · doi:10.1177/14773708241275424 · Cited by 3
“Google ’ s algorithms are constantly under development, which can affect the data generation process (Lazer et al., 2014).”
Ogorek, B. A., Rhoads, T. P., Finkelman, E. S., & Rodriguez-Chavez, I. R. (2025) · JMIR AI · doi:10.2196/65481 · Cited by 6
Xu, Y., Gong, Z., Fang, T., Zhang, H., & Tang, G. (2025) · International Journal of Geographical Information Systems · doi:10.1080/13658816.2025.2515200 · Cited by 3
“Therefore, people’s search behaviors in virtual space are often used to predict their travel in physical space (Lazer et al .”
Manideep, A. S., Pasupuleti, R. S., Tulimelli, J., Dokku, Y., Chen, W.-K., & Nalluri, V. (2025) · IEEE International Conference on Social Sciences and Intelligence Management (SSIM) · doi:10.3390/engproc2025098012 · Cited by 4
Dades massives i estadística: La perspectiva d'un estadístic
Rossell, D. (2014) · Mètode: Revista de difusió de la investigació de la Universitat de Valencia · doi:10.7203/metode.83.3590 · Cited by 3
“However, Lazer et al. (2014), amongst others, reported that GFT predictions had not been that good.”
The fourth V, as in evolution: How evolutionary linguistics can contribute to data science
Pokornowski, M. (2015) · Theoria et Historia Scientiarum · doi:10.12775/ths-2014-003 · Cited by 2
“The accuracy of GTF has been recently questioned by Lazer et al. (2014) in a paper that reports significant discrepancies between the algorithm’s predictions and factual data obtained from public health care institutions.”
BackgroundLake, P., & Drake, R. G. (2014) · Advanced information and knowledge processing · doi:10.1007/978-3-319-13503-8_1 · Cited by 3
Promise and Perils of Big Data Science for Intelligence Community
Jani, K., & Soni, A. (2018) · Advanced sciences and technologies for security applications · doi:10.1007/978-3-319-75232-7_11 · Cited by 3
Datenschutz im Internet: Akteure, Regulierungspraktiken und Interessenlagen
Schütz, P., & Karaboga, M. (2018) · Springer eBooks · doi:10.1007/978-3-658-02033-0_11 · Cited by 3
Why Employees Leave Russian Companies? Analyzing Online Job Reviews Using Text Mining
Sokolov, D. N., Selivanovskikh, L. V., Zavyalova, E. K., & Latukha, M. O. (2018) · Russian Management Journal · doi:10.21638/spbu18.2018.402 · Cited by 4
“As a caution, no matter how big “big data” is, the infor mation and knowledge derived from it may still be rather misleading [Lazer et al., 2014; Lin, Lucas, Shmueli, 2013].”
BackgroundThe “War” on Expertise: Five Communities that Seek to Discredit Experts
Klein, G. A., Shneiderman, B., Hoffman, R. R., & Wears, R. L. (2018) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198795872.013.50 · Cited by 5
Big Data− Análisis informétrico de documentos indexados en Scopus y Web of Science
Musiño, C. M. (2020) · Investigación Bibliotecológica Archivonomía Bibliotecología e Información · doi:10.22201/iibi.24488321xe.2020.82.58035 · Cited by 8
“Estos autores publicaron en Science (Fung, Tse y Fu, 2015; Broniatowski, Paul y Dredze, 2014; Lazer et al., 2014b; Shneiderman, 2014) y Nature (Gijzen, 2013; Huberman, 2012).”
BackgroundHow Can Big Data Science Transform the Psychological Sciences?
Albritton, B. H., & Tonidandel, S. (2020) · The Spanish Journal of Psychology · doi:10.1017/sjp.2020.45 · Cited by 4
A Social Media Time-Series Data Analytics Approach for Digital Epidemiology
Parwez, M. A., Abulaish, M., & Jahiruddin, J. (2020) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wiiat50758.2020.00131 · Cited by 4
“Using social media data for surveillance, however, is a subject of major concerns as limitations have been pointed out by researchers [35], [36], especially, after the failure of Google Flu Trend [37].”
BackgroundHarnessing big data to strengthen evidence-informed precise public health response
Asokan, G. V., & Mohammed, M. Y. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-822884-5.00003-9 · Cited by 11
Big Data-Anwendungen in der Gesundheitsförderung und Prävention
Hocke-Bolte, Z., Peters, B., & Haunit, T. (2021) · Springer eBooks · doi:10.1007/978-3-658-31434-7_27 · Cited by 4
Single Model for Influenza Forecasting of Multiple Countries by Multi-task Learning
Murayama, T., Wakamiya, S., & Aramaki, E. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86514-6_21 · Cited by 5
“Also, the training of a single model using various flu-related data can capture the nature of flu epidemics in each country by escaping from overfitting, which is caused by a lesser degree of historical data in one country for training [27,26].”
BackgroundBeckhaus, J., Becher, H., & Belau, M. H. (2022) · One Health & Implementation Research · doi:10.20517/ohir.2022.01 · Cited by 3
Singleton, A. D., & Spielman, S. E. (2021) · The urban book series · doi:10.1007/978-981-15-8983-6_15 · Cited by 3
How reported outbreak data can shape individual behavior in a social world
Pritchard, A. J., Silk, M. J., Carrignon, S., Bentley, R. A., & Fefferman, N. H. (2022) · Journal of Public Health Policy · doi:10.1057/s41271-022-00357-7 · Cited by 3
“For instance, search results can overestimate disease prevalence [27] because searches, themselves, spread through social learning, algorithms that uprank popular searches, or both processes [28].”
BackgroundRepar, L. A., & Bogue, J. (2022) · Elsevier eBooks · doi:10.1016/b978-0-12-821408-4.00005-5 · Cited by 5
A Survey on Digital Epidemiology: Networks and Models
Pastor-Escuredo, D., & Olmos, L. E. (2022) · Medical Research Archives · doi:10.18103/mra.v10i10.3175 · Cited by 5
Sekara, V., Karsai, M., Moro, E., Kim, D., Delamónica, E., Cebrián, M., Luengo-Oroz, M. A., Jiménez, R. M., & García–Herranz, M. (2024) · Advances in Complex Systems · doi:10.48550/arxiv.2307.01891 · Cited by 12
Air pollution dispersion models
Vallero, D. A. (2023) · Elsevier eBooks · doi:10.1016/b978-0-443-13987-1.00022-3 · Cited by 3
Graph Neural Network Modeling of Web Search Activity for Real-time Pandemic Forecasting
Chen, L., Zhou, J., Zhang, J., Yang, C., & Agichtein, E. (2023) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi57859.2023.00027 · Cited by 6
“This led to data bias in the predictive model, as the correlation with actual flu outbreaks was no longer strong [25].”
Computational social science with confidence
Mattsson, C. E. S. (2024) · EPJ Data Science · doi:10.1140/epjds/s13688-023-00435-0 · Cited by 3
“While changes in platform governance are especially dramatic, there are also more pernicious threats to the validity of our tools over time [5].”
Walhidayat, W., Sadar, M., Yuhelmi, Y., & Samsidar, R. (2024) · E3S Web of Conferences · doi:10.1051/e3sconf/202459306001 · Cited by 5
Machine learning uncertainty framework applied to gas-liquid horizontal pipe flow
Quintino, A. M., & Stanko, M. (2025) · International Journal of Multiphase Flow · doi:10.1016/j.ijmultiphaseflow.2025.105184 · Cited by 5
Research on the Data Elements Enabling New Quality Productivity Mechanisms, Challenges, and Pathways
Xue, Y. (2025) · SHS Web of Conferences · doi:10.1051/shsconf/202521804011 · Cited by 3
Google Trends (GT) related to influenza
Wiwanitkit, V. (2015) · Cadernos de Saúde Pública · doi:10.1590/0102-311xca020615 · Cited by 3
Nowcasting in real time using popularity priors
Monokroussos, G., & Zhao, Y. (2020) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2020.03.004 · Cited by 3
Development Impact of Social Media
Ackland, R., & Tanaka, K. (2015) · World Bank, Washington, DC eBooks · doi:10.1596/23642 · Cited by 7
“provides an additional tool for monitoring crises such as pandemics, although the concerns raised by Lazer et al. (2014) in the context of using Google search data to monitor flu trends are pertinent here.”
InfluentialEngineering management: new advances and three open questions
Xu, J. (2016) · International Journal of Management Science and Engineering Management · doi:10.1080/17509653.2016.1162968 · Cited by 3
Potencjał Big Data w badaniach społecznych
Żulicki, R. (2017) · Studia Socjologiczne · Cited by 10
Mattingly-Jordan, S. (2018) · Administration & Society · doi:10.1177/0095399718770392 · Cited by 3
Programming the News: When Algorithms Come to Matter
Bucher, T. (2018) · Oxford University Press eBooks · doi:10.1093/oso/9780190493028.003.0006 · Cited by 9
Bonenfant, M., & Meurs, M. (2018) · Springer eBooks · doi:10.1007/978-94-024-1202-4_39-1 · Cited by 3
(re)Médiations numériques et perturbations des sciences sociales contemporaines
Beaude, B. (2018) · Sociologie et sociétés · doi:10.7202/1054275ar · Cited by 7
“…majeur, se révéla finalement être un échec dès l’année suivante, pour des raisons inhérentes à la faiblesse de la démarche adoptée (Butler, 2013 ; Lazer et al., 2014 ; Salzberg, 2014). voir le plus possible tout en divulguant le moins possible, soulignant le traitement différentiel des vertus…”
BackgroundThe Study of Patient Prediction Models on Flu, Pneumonia and HFMD Using Big Data
우종필, Hwang, J., 이지은, et al. (2018) · The Korea Journal of BigData · doi:10.36498/kbigdt.2018.3.1.55 · Cited by 3
Event Detection in Noisy Streaming Data with Combination of Corroborative and Probabilistic Sources
Suprem, A., & Pu, C. (2019) · IEEE International Conference on Collaboration and Internet Computing (CIC) · doi:10.1109/cic48465.2019.00029 · Cited by 3
Societal Verification for Nuclear Nonproliferation and Arms Control
Gastelum, Z. (2020) · Springer eBooks · doi:10.1007/978-3-030-29537-0_12 · Cited by 3
ThreatPredict: From Global Social and Technical Big Data to Cyber Threat Forecast
François, J., Beck, F., Mezzour, G., Carley, K. M., Lahmadi, A., Ghogho, M., Houmz, A., Hammouchi, H., Zakroum, M., Nejjari, N., et al. (2020) · NATO science for peace and security series. B, Physics and biophysics · doi:10.1007/978-94-024-2021-0_5 · Cited by 3
Nouira, K., & Njima, N. B. (2020) · International Multi-Conference on Organization of Knowledge and Advanced Technologies (OCTA) · doi:10.1109/octa49274.2020.9151707 · Cited by 3
“Many updates were achieved in GFT model, but it doesn’t lead for accuracy results and it ends with his quiet death [2].”
BackgroundBetcheva, L., Erhun, F., & Jiang, H. (2020) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780190066727.013.13 · Cited by 5
Rejoinder: A Nonparametric Superefficient Estimator of the Average Treatment Effect
Benkeser, D. C., Cai, W., & van der Laan, M. J. (2020) · Statistical Science · doi:10.1214/20-sts789 · Cited by 5
Innovating at the human–technology interface in disasters and disease outbreaks
Bernardo, T. M., Gutiérrez, E. P., Hachborn, G., Forrest, R. O., & Sobkowich, K. E. (2020) · Scientific and Technical Review (World Organisation for Animal Health) · doi:10.20506/rst.39.2.3100 · Cited by 3
INTEREST RATES SENSITIVITY ARBITRAGE – THEORY AND PRACTICAL ASSESMENT FOR FINANCIAL MARKET TRADING
Stádník, B. (2021) · Business Management and Economics Engineering · doi:10.3846/bmee.2021.12658 · Cited by 3
“, 2015; Lintilhac & Tourin, 2016), big data (McAfee et al., 2012; Lazer et al., 2014; Nardo et al., 2016) and factor investing (Maeso & Martellini, 2017).”
Nieograniczone programy inwigilacji elektronicznej a koncepcja państwa autorytarnego
Rojszczak, M. (2021) · Studia nad Autorytaryzmem i Totalitaryzmem · doi:10.19195/2300-7249.42.2.11 · Cited by 3
Conceptualizing an Ecological Model of Google Search and Twitter Data in Public Health
Liang, B., & Wang, Y. (2021) · Human Dynamics in Smart Cities · doi:10.1007/978-3-030-83010-6_10 · Cited by 3
Smart Visualization of Medical Images as a Tool in the Function of Education in Neuroradiology
Simović, A., Lutovac, M., Lekić, S., & Kuleto, V. (2022) · Diagnostics · doi:10.3390/diagnostics12123208 · Cited by 4
“the results obtained by Big Data analysis may result in the fall of perspective models [13].”
ResultThe Data Revolution: A Comprehensive Survey on Datafication
Donald, A. D., Srinivas, T. A. S., Rekha, K., Anjali, D., & Srihith, I. V. D. (2023) · International Journal of Advanced Research in Science Communication and Technology · doi:10.48175/ijarsct-9007 · Cited by 5
Pateras, K., Meletis, E., Denwood, M. J., Eusebi, P., & Kostoulas, P. (2023) · Infectious Disease Modelling · doi:10.1016/j.idm.2023.05.001 · Cited by 2
Ramanathan, A., Pullum, L., Hobson, T. C., Stahl, C. G., Steed, C. A., Quinn, S. P., Chennubhotla, C. S., & Valkova, S. (2015) · Frontiers in Public Health · doi:10.3389/fpubh.2015.00182 · Cited by 3
“Unlikeaggregatingweb-basedsearchpatternsbyusers(33,35), ortheuseofsocialmedia(39–42),wheretheuseofsuchdata cansignificantlyoverestimatethefluincidencepatterns(43,44), usingeHRCsprovidesamoreaccurateindicationandpotential predictors.”
Tanti, M. (2015) · doi:10.1109/isei.2015.7358716 · Cited by 3
“It would requires a recalibration of data from field data (Lazer, 2014).”
MethodMacPhail, T. (2015) · Public Culture · doi:10.1215/08992363-2841820 · Cited by 3
The creation of facts in the cloud
Gotterbarn, D. (2016) · ACM SIGCAS Computers and Society · doi:10.1145/2874239.2874248 · Cited by 3
Vertical and horizontal approaches to the making of racial statistics in Britain
Brown, L. (2016) · Ethnic and Racial Studies · doi:10.1080/01419870.2015.1124127 · Cited by 3
“Transforming official statistics into Big data raises a series of technical and ethical challenges around data quality, volatility over time and individual privacy (Lazer et al. 2014; Struijs, Braaksma, and Daas 2014).”
BackgroundUniversal knowledge discovery from big data using combined dual-cycle
Shen, B. (2015) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-015-0376-z · Cited by 3
Big data: Risks and regulatory strategies
Sun, M., Yang, W., & Jiang, J. (2015) · IEEE International Conference on Progress in Informatics and Computing · doi:10.1109/pic.2015.7489869 · Cited by 5
Ma, Y., Liu, Z., Zheng, N., Wang, J., & Chen, S. (2017) · Scientia Sinica Informationis · doi:10.1360/n112016-00120 · Cited by 11
Au-delà de la critique big data
Rogers, R. (2015) · Presses des Mines eBooks · doi:10.4000/books.pressesmines.1990 · Cited by 3
Kleibrink, A., & Mateos, J. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-409548-9.09674-3 · Cited by 3
Integrating social media and rainfall data to understand the impacts of severe weather in Argentina
Choi, S. (2016) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 3
An Insight into Theory-Guided Climate Data Science—A Literature Review
Sheikh, R., & Jahirabadkar, S. A. (2018) · Lecture notes in networks and systems · doi:10.1007/978-981-10-8360-0_11 · Cited by 4
À la recherche de nouvelles traces sociales: L’exemple des conspirationnistes
Bronner, G., David, P., & Buono, L. D. (2018) · Revue européenne des sciences sociales · doi:10.4000/ress.3988 · Cited by 4
Nandy, A., De, A., Roy, P., Dutta, M., Roy, M., Sen, D., & Basak, S. C. (2018) · Springer eBooks · doi:10.1007/978-981-10-8693-9_9 · Cited by 4
“Google Flu Trends, which is the recent surveillance tracking arm of the search engine giant, came into the spotlight for over-calculating the doctor visits for in fl uenza-like illness than the reports generated by CDC which relies on records procured from laboratories (Lazer et al. 2014).”
BackgroundMethodResearch on corporate finance: history, challenge and prospect in China
Lan, Y., Wang, J., Wu, S., & Yang, J. (2019) · Nankai Business Review International · doi:10.1108/nbri-02-2019-0006 · Cited by 3
A Framework for Intelligent Policy Decision Making Based on a Government Data Hub
Al-Lawati, A., & Barbosa, L. S. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-37858-5_8 · Cited by 4
A FOUNDATION FOR INDUCTIVE REASONING IN HARNESSING THE POTENTIAL OF BIG DATA
Hassad, R. A. (2020) · Statistics Education Research Journal · doi:10.52041/serj.v19i1.133 · Cited by 13
“The “natural history of disease refers to the progression of a disease process in an individual over time, in the absence of treatment” (CDC, 2006, p. 59), whereas biology relates to pathogenesis or the underlying disease mechanism.”
BackgroundHirschheim, R. (2020) · Journal of Information Technology · doi:10.1177/0268396220967677 · Cited by 4
“One exception is the study by Lazer et al. (2014) that examined the infamous “Google Flu” fiasco where Google Flu Trends (GFT) data were erroneously used to predict an outbreak of influenza in 2013, which never occurred.”
BackgroundGhozia, A., Attiya, G., Adly, E., & El-Fishawy, N. A. (2020) · Computational Intelligence and Neuroscience · doi:10.1155/2020/8813089 · Cited by 5
Jungherr, A. M., Rivero, G., & Gayo-Avello, D. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108297820.008 · Cited by 5
Big Data … Why (Oh Why?) This Computational Social Science?
O’Sullivan, D. (2018) · UNP - Nebraska eBooks · doi:10.2307/j.ctt21h4z6m.7 · Cited by 6
沈, 焕., & 张, 良. (2022) · SCIENTIA SINICA Terrae · doi:10.1360/sste-2022-0089 · Cited by 4
Jungherr, A. M., & Posegga, O. (2023) · Springer eBooks · doi:10.1007/978-3-658-31480-4_54-1 · Cited by 4
A Comparative Study of Symptom Clustering On Clinical and Social Media Data
Yang, C. C., Ip, E. H., Avis, N. E., et al. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16268-3_23 · Cited by 3
Ratushny, V., & Smith, G. P. (2015) · Journal of Investigative Dermatology · doi:10.1038/jid.2015.93 · Cited by 3
“Initial flaws in methodology resulted in an overestimation of influenza incidence due to search queries being overly influenced by media publicity rather than disease activity (Lazer et al., 2014).”
BackgroundMethodInfluentialMesurer l’activité durant la crise sanitaire
Blanchet, D., & Tavernier, J.-L. (2020) · Revue de l'OFCE/La Revue de l'OFCE · doi:10.3917/reof.166.0023 · Cited by 3
Construction of Economic Indicators Using Internet SearchesPreprint
Popescu, M. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2679256 · Cited by 3
Stockmann, D. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2607998 · Cited by 3
ARGO: a model for accurate estimation of influenza epidemics using Google search dataPreprint
Yang, S., Santillana, M., & Kou, S. C. (2015) · arXiv (Cornell University) · Cited by 4
Alves, D. S. (2015) · RIUnB Institutional Repository (University of Brasília) · doi:10.26512/2015.10.t.19345 · Cited by 3
“Mesmo sendo de grande valor, pesquisas como a descrita no parágrafo anterior e refutadas por [115] mostram que a tecnologia é algo em evolução e ainda apresenta grandes desafios.”
BackgroundDiscussion of ‘Analyzing Behavioral Big Data: Methodological, Practical, Ethical, and Moral Issues'
Tsui, K., & Zhao, Y. (2016) · Quality Engineering · doi:10.1080/08982112.2016.1210981 · Cited by 3
“However, as reported in Butler (2013) and Lazer et al. (2014), GFT failed by predicting more than double the proportion of doctor visits for ILI than the CDC report in the 2012–2013 season.”
MethodResult
Metadata
- Authors
- Lazer, D., Kennedy, R., King, G., & Vespignani, A.
- Affiliations
- Northeastern University (US)
- Harvard University (US)
- Corresponding author
- Lazer, David
- Venue
- Science
- Published
- 2014-03-14
- Volume
- 343
- Issue
- 6176
- Pages
- 1203-1205
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2068181924
- Semantic Scholar
- 00d681cfff7f0224ace607e515f4ed510e792df9
- Cited by
- 2,523
- References
- 30
- Citations by year
- 2026: 113 · 2025: 101 · 2024: 121 · 2023: 150 · 2022: 168 · 2021: 229 · 2020: 273 · 2019: 261 · 2018: 272 · 2017: 249 · 2016: 271 · 2015: 219 · 2014: 91
- FWCI
- 208.16
- Type
- journal article
- Language
- en
- Open access
- green
- dash.harvard.edu
- Licence
- cc-by
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Epidemiology › Data-Driven Disease Surveillance
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- Google Flu Trends, influenza surveillance, big data, prediction error, data quality
- MeSH terms
- Centers for Disease Control and Prevention, U.S., Data Collection, Data Interpretation, Statistical, Disease Outbreaks, Humans, Influenza, Human, Office Visits, Prognosis, United States, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.