The Parable of Google Flu: Traps in Big Data Analysis
Open accessThe DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index; 5 of the 8 sources its claims cite were checked for support.
Abstract
Large errors in flu prediction were largely avoidable, which offers lessons for the use of big data.
Overview
CoChat Summary
TL;DR
The paper uses the failure of Google Flu Trends (GFT) to illustrate two traps in big data analysis: “big data hubris” and algorithm dynamics. It argues that GFT’s large flu-prediction errors were largely avoidable and that big data should supplement, not replace, traditional data and methods.
Key findings
- In February 2013, GFT predicted more than double the proportion of doctor visits for influenza-like illness (ILI) than the CDC; from 21 August 2011 to 1 September 2013, GFT reported overly high flu prevalence in 100 out of 108 weeks, and it overshot the 2011–2012 season by more than 50%.
- GFT errors were not random: they showed temporal autocorrelation and seasonality, meaning last week’s errors predicted this week’s errors and error direction/magnitude varied with time of year.
- Simple alternatives outperformed GFT: out-of-sample mean absolute error was 0.486 for GFT, 0.311 for lagged CDC, and 0.232 for combined GFT and CDC; all differences were statistically significant at P < 0.05. Even 3-week-old CDC data did better than GFT at projecting current flu prevalence.
- The authors attribute GFT’s problems to overfitting (initial GFT matched 50 million search terms to 1152 data points and missed the nonseasonal 2009 H1N1 pandemic) and to “algorithm dynamics,” including Google search algorithm changes and user behavior changes.
Methods
- Design: policy forum/case study comparing GFT estimates with CDC ILI surveillance data and alternative forecasting models.
- Data/sample: GFT and CDC flu prevalence estimates, including the period from 21 August 2011 to 1 September 2013; the initial GFT methodology reportedly fit 50 million search terms to 1152 data points.
- Main analysis: comparison of GFT alone, lagged CDC data with 52-week seasonality variables, and a combined GFT + CDC model using lagged CDC estimates, lagged GFT error, and seasonality variables; reported mean absolute error and statistical significance; Google Correlate used to examine correlated search terms. Supplementary materials contain data and methods.
Limitations
- The paper states that GFT never documented the 45 search terms used, released examples appear misleading, and Google Correlate is limited to national-level data whereas GFT was developed using regional correlations; therefore the original GFT analyses cannot be replicated from the information provided.
- The paper states that Google cannot make its full data available to outsiders because of privacy issues, though it argues there is no such constraint on derivative, aggregated data.
- Evident from the text: this is a commentary/policy forum rather than a full empirical article, with key data and methods placed in supplementary materials; the quantitative comparison depends on GFT and CDC data and model specifications not fully detailed in the main text.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 2 Not valid · 2 Not checked · 8 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“This ad hoc method of throwing out peculiar search terms failed when GFT completely missed the nonseasonal 2009 infl uenza A–H1N1 pandemic ( 2, 14).”
What the sources say
- Partly supportedcites [2] matched to “Reassessing Google Flu Trends Data for Detection of Seasonal and Pandemic Influenza: A Comparative Epidemiological Study at Three Geographic Scales” (2013) sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says the original GFT model missed the spring 2009 pandemic wave, supporting the claim about its failure to detect the nonseasonal outbreak. It does not say GFT completely missed the entire 2009 pandemic. The Introduction describes automated selection of the original model’s search terms, not an ad hoc method of throwing out peculiar terms. (LLM, conf=0.86)
- Disputedcites [14] S. Cook et al., PLOS ONE 6, e23610 (2011). sourceExperimental — accuracy not yet measured
The Abstract (Methodology/Principal Findings) says the original GFT model underestimated the magnitude of illness during the 2009 H1N1 pandemic, not that it completely missed it. The Abstract (Conclusions) says both models performed well during the pandemic, although the updated model performed better. The Discussion notes that pandemic-specific queries were excluded, but does not describe this as an ad hoc method of throwing out peculiar terms. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Oddly, the few search terms offered in the papers ( 14) do not seem to be strongly related with either GFT or the CDC data (SM)—we surmise that the authors felt an unarticulated need to cloak the actual search terms identifi ed.”
What the sources say
- Disputedcites [14] S. Cook et al., PLOS ONE 6, e23610 (2011). sourceExperimental — accuracy not yet measured
The Results section (‘Search Behavior During pH1N1’; page unavailable) says that all three search queries shown in Figure 3 closely tracked CDC ILINet data before the pandemic, and ‘symptoms of flu’ continued to do so during it. The paper does report that the bronchitis and pneumonia queries underestimated ILINet data during the pandemic, but that does not support the broad claim that the offered terms were not strongly related to the data. (LLM, conf=0.87)
Checked by a reasoning model — may be wrong.
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The Internet has opened the way for improving standard surveys, experiments, and health reporting ( 35).”
What the sources say
- Partly supportedcites [35] A. J. Berinsky et al., Polit. Anal. 20, 351–368 (2012). sourceExperimental — accuracy not yet measured
Section 2 describes using an Internet platform to run surveys with embedded experiments, and the Conclusion says MTurk can make experimental research easier and less expensive. The paper does not address improvements to health reporting; the disease scenario in section 4.3.2 is a framing experiment, not health reporting. Page numbers are unavailable in our copy. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although not widely reported until 2013, the new GFT has been persistently overestimating flu prevalence for a much longer time.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“From 21 August 2011 to 1 September 2013, GFT reported overly high fl u prevalence 100 out of 108 weeks.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
3 more claims
Create a free account to see all 14 claims and every source verdict.
- It may state two separate claims as one. 5
- Probably not a substantial claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Google Trends predicted more than twice the number of influenza-like illnesses than the CDC in February 2013 46 .”
From Unrepresentative big surveys significantly overestimated US vaccine uptake (2021) · cites this paper as [46]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This was one of the main reasons behind the failure of Google Flu Trends, since the data used for training the model in the first few years was not representative of the trends in subsequent years [15].”
From Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data (2017) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Social data research often relies on counting entities. These entities can be users, links, or messages, and the description is a summary of these counts (Lazer et al., 2014; Salganik, 2017). Yet, simple counts can mislead if it is unclear what is counted and how.”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This unintelligibility is also subject to functional biases (section 4.1), as it depends on the mechanisms available on each social platform (e.g., having a like button, but not a dislike one), as well as to variations in platform algorithms and mechanisms in response to users actions (Lazer et al., 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“It also includes temporal validity, which captures to what extent constructs change over time (Howison et al., 2011) and may invalidate previous conclusions about societal and/or platform phenomena; e.g., see the case of Google Flu Trends (Lazer et al., 2014).”
From Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries (2019) · cites this paper as Lazer et al., 2014
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 14 MARCH 2014 VOL 343 SCIENCE www.sciencemag.org 1204
“The 2009 version of GFT has weathered other media panics related to the fl u, including the 2005–2006 influenza A/H5N1 ("bird flu") outbreak”
“GFT engineers updated the algorithm in 2009, and this model has run ever since”
Citations
Cites
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Twitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206
Detecting influenza epidemics using search engine query data
Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009) · Nature · doi:10.1038/nature07634 · Cited by 4,295
Cohen, D. (1998) · The Women's Review of Books · doi:10.2307/4022859 · Cited by 4,261
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Big Data : The Management Revolution
マカフィー, ア., ブリニョルフソン, エ., & 有賀, 裕. (2013) · Harvard business review · Cited by 3,549
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., Christakis, N., Contractor, N., Fowler, J., Gutmann, M., et al. (2009) · Science · doi:10.1126/science.1167742 · Cited by 3,365
Self-citationPredicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment
Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v4i1.14009 · Cited by 2,649
Multiscale mobility networks and the spatial spreading of infectious diseases
Balcan, D., Colizza, V., Gonçalves, B., Hu, H., Ramasco, J. J., & Vespignani, A. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0906910106 · Cited by 1,380
Self-citationPredicting consumer behavior with web search
Goel, S., Hofman, J. M., Lahaie, S., Pennock, D. M., & Watts, D. J. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1005962107 · Cited by 636
Research works printed but not linked: 7
- A. McAfee, E. Brynjolfsson, Harv. Bus. Rev. 90, 60 (2012).
- AAAI Conference on Weblogs and Social Media, Atlanta, Georgia, 11 to 15 July 2010 (Association for Advancement of Artifi cial Intelligence, 2010), pp. 178.
- P. Copeland et al., Int. Soc. Negl. Trop. Dis. 2013, 3
- I. M. Hall et al., Epidemiol. Infect. 135, 372 (2007).
- E. Mustafaraj, P. Metaxas, in Proceedings of the Web- Sci10, Raleigh, NC, 26 and 27 April 2010 (Web Science Trust, 2010); http://journal.webscience.org/317/.
- J. Ratkiewicz et al., in Proceedings of 5th International AAAI Conference on Weblogs and Social Media, San Francisco, CA, 7 to 11 August 2011 (AAAI, 2011), p. 297–304.
- 2013; http://chronicle.com/article/Researchers-Struggle- to-Secure/141591/.
Cited by
Items 2,001–2,100 of 2,513
Data science for epidemiology: a case study of dengue in Brazil.
Roster, K. I. O. (2022) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.55.2022.tde-27022023-142607 · Cited by 2
Observer Effect in Social Media UsePreprint
Saha, K., Gupta, P., Mark, G. J., Kıcıman, E., & Choudhury, M. D. (2024) · Research Square · doi:10.21203/rs.3.rs-2492994/v2 · Cited by 1
Google search data for social scientists: A tutorial and best practicesPreprint
Moon, J. W., & Barlev, M. (2024) · doi:10.31234/osf.io/af7b2 · Cited by 1
Words Worth Millions: Decrypting Financial Text with GeminiPreprint
PAL, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4861479 · Cited by 4
Lasisi, M., Kolade, K., & Rotimi, O. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-95689-5.00269-8 · Cited by 2
Jibril, M., Ekwere, E. O., O., E. S., Mohammed Bello, M., & Ishaq, S. A. (2024) · Preprints.org · doi:10.20944/preprints202408.1105.v1
O Funcionamento Diferencial dos Itens na Indução de Classificadores Mais Justos
Minatel, D. (2024) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.55.2024.tde-06012025-175603 · Cited by 2
Battling Antibiotic Resistance: Can Machine Learning Improve Prescribing?Preprint
Ribers, M. A., & Ullrich, H. (2019) · Research at the University of Copenhagen (University of Copenhagen) · doi:10.2139/ssrn.3392196 · Cited by 3
“One core lesson from this low performance is that collecting and combining relevant types of data is crucial (Lazer et al. 2014).”
BackgroundSocial Genome Mining for Crisis PredictionPreprint
Wlodarczak, P., Qian, S., Ally, M., & Soar, J. (2015) · SSRN Electronic Journal
Murray Valley encephalitis virus infection
Combs, B. (2013)
Visualizing Big Data amounts : The Case of Google Flu Trends (GFT)
Goysdotter, M., & Grönberg-Hernández, J. (2015) · Lund University Publications (Lund University)
Template for preparation of papers for IEEE sponsored conferences & symposia
Sacchi, L., Dagliati, A., Tibollo, V., Leporati, P., Cata, P. D., Cerra, C., Chiovato, L., & Bellazzi, R. (2015) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7318808
Does the Cloud of Surveillance Have a Silver Lining?
Lesk, M. (2014) · IEEE Security & Privacy · doi:10.1109/msp.2014.41
Only Time Will Tell. Data-driven Approaches to Phenotyping Critical Illness
Liu, V. (2014) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201404-141ed
Analysing “Long Data” on Collective Violence in Indonesia
MEYER, D. A., & Stein, A. A. (2015) · Asian journal of social science · doi:10.1163/15685314-04305006
“This analysis was initially very successful compared with the us Center for Disease Control’s observations, but then deteriorated, bringing to light a set of novel problems with this kind of data (Lazer et al., 2014).”
ResultSantillana, M., Wendong, D., Althouse, B. M., & Ayers, J. W. (2014)
13th ERS Lung Science Conference. The most important take home messages: News from the Underground
Bikov, A., Boots, A. W., Bjerg, A., Jacinto, T., Olland, A., & Skoczyński, S. (2015) · Breathe · doi:10.1183/20734735.04015
Prediksjon i Internasjonal politikk
Nygård, H. M. (2015) · Internasjonal politikk · doi:10.18261/issn1891-1757-2015-04-02
Predicting UK Unemployment with Internet Search and Survey DataPreprint
Smith, P. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2597082
“able? What is the motivation of the user to enter a search term? Lazer et al. (2014) highlight that there needs to be careful consideration of social and”
Big Data in the Social Sciencespreprint.pdf
Youtie, J., & Porter, A. L. (2016) · Science and Public Policy
A response to “Transaction Cost Economics on Trial Again”
Schermann, M., Yetton, P., & Krcmar, H. (2016) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2016.02.003
“In other data-driven research, it is very unlikely that data collected by practitioners, business associations and government bodies, for example, would explicate their theoretical frameworks (Lazer et al., 2014).”
BackgroundIs Big Data the New ?God? on Earth? [Opinion]
Pileggi, S. F. (2016) · IEEE Technology and Society Magazine · doi:10.1109/mts.2015.2494258
Reviewers should ask the right questions - but is InfoQ the answer?
Karlberg, M. (2016) · Statistical Journal of the IAOS · doi:10.3233/sji-160980
“…potential benefits” of us-ing InfoQ, there isn’t much in the way of critical discussion in [1] (this is reminiscent of the authors’ references to “Google’s ability to predict the prevalence of flu”, which would have benefitted from the type of critical discussion on the topic conducted elsewhere [7]).”
Bhattacharya, P., Phan, T. Q., & Goh, K. Y. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2391002
Leone, M. I., Dindalini, S., & Rullani, F. (2014)
Big Data: Google-haut ennustavat työttömyyttä Suomessa
Tuhkuri, J. (2014) · Econstor (Econstor)
McInerney, T. F. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781316105757.008
e-Science for Digital Development: ICT4ICT4D
Hilbert, M. (2015) · eScholarship (California Digital Library) · Cited by 2
Scanning the Science-Society Horizon
Moon, B. (2016) · ANU Open Research (Australian National University) · doi:10.25911/5d6664e8354b8
“1.4 Outline of chapters In Chapter 2, ‘Literature Review’ I begin by looking at the history of science communication and the growing focus on understanding public attitudes to science and technology issues as a key to effective science communication.”
BackgroundMethodResultInfluentialBig Data – et nyt humanvidenskabeligt forskningsfelt
Finnemann, N. O. (2015) · Magasin fra Det Kongelige Bibliotek · doi:10.7146/mag.v28i1.66808
Vertriebliche Implikationen und Anwendungsfelder von Big Data
Luck, L. (2015) · Springer eBooks · doi:10.1007/978-3-658-05054-2_22
Chasing yesterday : nowcasting economic activity with timely indicators
Smith, P. (2016) · STAX · doi:10.48730/j51r-5r95
Big Data? Big Issues Degradation in Longitudinal Data and Implications for Social Sciences
Weber, M. S., & Nguyen, T. H. H. (2015) · ACM Web Science Conference (WebSci) · doi:10.1145/2786451.2786482 · Cited by 2
SOCIOLOGY | RESEARCH ARTICLE Big data business models: Challenges and opportunities
Schroeder, R., & Halsall, J. P. (2016)
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0013
The Future Directions of Health Intelligence
Hill, A., & Flowers, J. (2016) · Springer eBooks · doi:10.1007/978-3-319-28326-5_12
What Engineering Contributes: Devotion to Innovative Technologies
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0005
Smith, G. A. (2018) · Oxford University Press eBooks · doi:10.1093/oso/9780198824305.003.0010
What Design Contributes: Fresh Thinking to Serve Human Needs
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0006
Combining Applied and Basic Research: The ABC Principle
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0002 · Cited by 2
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0012
Choose Actionable Problems that Address Civic, Business, and Global Priorities
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0007
Apply Observation, Intervention, and Controlled Experiments
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0008
What Science Contributes: Persistence in Understanding the World
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0004
Form Teams with Diverse Individuals and Organizations
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0009
Test Ideas and Prototypes with Realistic Interventions
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0010
Blending Science, Engineering, and Design: The SED Principle
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0003
Some guidance for the use of Big Data in macroeconomic nowcasting
Mazzi, G. L. (2016) · RiuNet (Universitat Politècnica de València) · doi:10.4995/carma2016.2015.4243
“For some critics about the usefulness of Big Data see Hartford (2014) and Lazer et al. (2014).”
Background‘Find love in Canada’: Distributed selves, abstraction, and the problem of privacy and autonomy
Miller, V. (2020) · Bloomsbury Academic eBooks · doi:10.5040/9781501356216.ch-001
Promote Adoption and Assess Impact
Shneiderman, B. (2016) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198758839.003.0011
Comparison study of SARIMA and ARGO models for in influenza epidemics prediction
Jung, J., & Lee, S. (2016) · Journal of the Korean Data and Information Science Society · doi:10.7465/jkdi.2016.27.4.1075 · Cited by 1
“Lazer et al .”
ResultThe Writing Is on the Wall: Epidemiology and the Anticipated Ends of Social Media
Hall, K. (2016) · Palgrave Macmillan UK eBooks · doi:10.1057/978-1-137-52141-5_6
A Method of Economic Indicator Nowcasting Using Baidu Searches
Li, F., & Li, G. (2015) · International Conference on Intelligent Systems and Knowledge Engineering (ISKE) · doi:10.1109/iske.2015.27
“Wu and Brynjolfsson [4] also did some research about predicting indicators related to housing market and obtained outstanding results, but they pointed out that search query data can be easily manipulated, which reduced the robustness of the predicting method.”
BackgroundGlobal Health Surveillance Approaches: New Opportunities
Song, I., Hayes, D., Mandal, P., et al. (2016) · Managing the Asian century · doi:10.1007/978-981-10-2281-4_5
Research on the Computing Framework in Big Data Environment
Liu, Y., Zhang, J., Han, S., et al. (2016) · International Conference on Information Science and Control Engineering · doi:10.1109/icisce.2016.125
An Alarm System for Flu Outbreaks Using Google Flu Trend Data
Vaughan, G., Aseltine, R. H., Chiou, S. H., & Yan, J. (2016) · ICSA book series in statistics · doi:10.1007/978-3-319-42568-9_22
Big Data Critical Thinking Skills for Analysts—Learning to Ask the Right Questions Full Paper
McKinney, E. H., & Niese, B. (2016)
Démons et merveilles du « prédictif » : une bonne fois pour toutes...
Raufer, X. (2016) · Sécurité globale · doi:10.3917/secug.164.0107
Bayesian models and inferential methods for forecasting disease outbreak severity
Michaud, N. (2016) · Iowa State University Digital Repository (Iowa State University) · doi:10.31274/etd-180810-5401
Business Applications of Big Data
Vidal-García, J., Vidal, M., & Barros, R. H. (2017) · Advances in public policy and administration (APPA) book series · doi:10.4018/978-1-5225-2537-0.ch007
Biography of Data: A Societal Level Perspective On Data Quality
Eshraghian, F., Lloyd, A., & Harwood, S. (2014) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.13140/rg.2.1.5026.4802
“A contemporary example of this is related by Lazer et al. (2014) who explained Google’s over-estimate of influenza diffusion in terms of a failure to consider the context of this data and how this introduces biases that might be controlled for.”
BackgroundBig data para el análisis de las necesidades traductológicas en cinco capitales de Europa
Fernández, A. G. (1970) · SKOPOS Revista Internacional de Traducción e Interpretación · doi:10.21071/skopos.v7i.6901
An Introduction to Computational Social Science for Organizational Communication
Pilny, A. N., & Poole, M. S. (2017) · Advances in business strategy and competitive advantage book series · doi:10.4018/978-1-5225-2823-4.ch011
Räumliches Data-Mining und Big Geospatial Data
Meng, L. (2019) · Springer Reference Naturwissenschaften · doi:10.1007/978-3-662-47096-1_70
Conversational Services for Multi-Agency Situational Understanding.
Preece, A. D., & Braines, D. (2017) · ORCA Online Research @Cardiff (Cardiff University)
Leading data analytics transformations
Moreno, C., & Mauricio, J. (2017)
Learning in the context of distribution drift
Webb, G. I. (2017)
Senses of “Selfie” Around the World From Web Search Patterns Over Extended Time
Hai-Jew, S. (2017) · Advances in media, entertainment and the arts (AMEA) book series · doi:10.4018/978-1-5225-3373-3.ch008
Big Data: why (oh why?) this computational social science?
O’Sullivan, D. (2017) · eScholarship (California Digital Library)
Big Data-Revolution oder Datenhybris?
Gramelsberger, G. (2017) · NTM Zeitschrift für Geschichte der Wissenschaften Technik und Medizin · doi:10.1007/s00048-017-0179-2
Über Wahrscheinlichkeiten und die (Un-)Fähigkeit, die Zukunft vorhersagen zu können
Hilbig, H. (2017) · Springer eBooks · doi:10.1007/978-3-658-19616-5_1
Nelimarkka, M., & Sund, R. (2017) · ProComma academic · doi:10.31885/2018.00030 · Cited by 5
E-santé, réseaux sociaux et rhumatologie : compatibles ou pas ?
Berenbaum, F. (2017) · Revue du Rhumatisme · doi:10.1016/j.rhum.2017.10.022
Applications of stochastic modeling and data analytics techniques in healthcare decision making
Dalgic, O. O. (2017) · UWSpace (University of Waterloo)
A Review on Importance of Smart City Indexes for Citizens’ Health Case Study: Esfahan City
Shahbazi, M., Massoud, M., Nastaran, M., Mohamadi, M., & Noe, M. G. (2017) · Global Journal of Health Science · doi:10.5539/gjhs.v10n2p70 · Cited by 1
Digital traces, sociology and Twitter : between false promises and real potential
Philippe, O. (2016) · ePrints Soton (University of Southampton)
Mazari, A. A. (2018) · Advances in healthcare information systems and administration book series · doi:10.4018/978-1-5225-3203-3.ch004 · Cited by 1
Robin, N., Klein, T., & Jütting, J. (2018) · doi:10.1787/f21ff556-ko
Research on Intelligent Sales Platform of Automobile Industry Based on Large Data Mining
Lee, J. (2018) · Springer proceedings in business and economics · doi:10.1007/978-3-319-72745-5_33
Data policies for big health data and personal health data
Chitondo, P. D. J. (2016)
Big Data Analysis Techniques for Visualization of Genomics in Medicinal Plants
Kumar, H., Chandramohan, V., Simon, S. M., Yadav, R. S., & Kumar, S. B. (2018) · Advances in data mining and database management book series · doi:10.4018/978-1-5225-3142-5.ch026
Sun, D. F. (2022) · Digital Archive @ GSU · doi:10.57709/12052350
Aiello, M. (2018) · Springer eBooks · doi:10.1007/978-3-319-90008-7_3
Social Network Analysis in an Age of Digital Information
Hardey, M. (2018) · Springer eBooks · doi:10.1007/978-1-4939-7131-2_403
Correlating regional emergency epistaxis visits with internet search activity
Griffith, S., Archbold, R., & Schell, S. (2018) · American Journal of Otolaryngology · doi:10.1016/j.amjoto.2018.07.012
Life at the Top: Engineering Participation
Bucher, T. (2018) · Oxford University Press eBooks · doi:10.1093/oso/9780190493028.003.0004 · Cited by 3
How to Train a Data Scientist for the Global South
Nygren, E., Sutinen, M., Westerlund, T., & Sutinen, E. (2018) · IST-Africa Week Conference · Cited by 2
Social Media Data Into Performance Measurement Systems: Methodologies, Opportunities, and Risks
Agostino, D., Arnaboldi, M., & AZZONE, G. F. (2018) · Advances in electronic government, digital divide, and regional development book series · doi:10.4018/978-1-5225-3731-1.ch012
Managing Enterprise Data and Providing a Personalized Employee Data Visualization
Manishankar, S., & Gokuldev, S. (2017) · International Conference on Current Trends in Computer, Electrical, Electronics and Communication (CTCEEC) · doi:10.1109/ctceec.2017.8455203
Segmenting, Summarizing and Predicting Data Sequences
Chen, L. (2018) · VTechWorks (Virginia Tech)
Traps in Multisource Heterogeneous Big Data Processing
Liu, Y. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-99695-0_28
Tavsiye Sistemlerinde Büyük Verinin Kullanımı Üzerine Kapsamlı Bir İnceleme
Utku, A., & Akcayol, M. A. (2018) · International Journal of Advances in Engineering and Pure Sciences · doi:10.7240/marufbd.440095 · Cited by 4
Exploration of Biostatistics Research in the New Century
Xiao, X. (2018) · Proceedings of the 2018 2nd International Conference on Education Innovation and Social Science (ICEISS 2018) · doi:10.2991/iceiss-18.2018.34
“They are the most interesting research topics in recent years [11-12].”
BackgroundFrom Mechatronics to the Cloud
Bradley, D., Russell, D., Hehenberger, P., Azorín-López, J., Watt, S., & Milne, C. (2020) · Springer eBooks · doi:10.1007/978-3-030-29131-0_3 · Cited by 6
Detecting Influenza Outbreaks Based on Spatiotemporal Information from Urban Systems
Grottenberg, L. O., Njå, O., Tøssebro, E., Braut, G. S., Bragstad, K., & Grøneng, G. M. (2019) · Lecture notes in geoinformation and cartography · doi:10.1007/978-3-030-05330-7_4
Applications of Big Spatial Data: Health
Waller, L. A. (2019) · Springer eBooks · doi:10.1007/978-3-319-77525-8_72
Rejoinder: Response to Discussions and a Look Ahead
Dorie, V., Hill, J., Shalit, U., et al. (2019) · Statistical Science · doi:10.1214/18-sts688
“In fact, we have been dismayed to observe how abandoning these important tenets has led to some cautionary tales in the past years (e.g., Lazer et al., 2014).”
BackgroundBig Data-Revolution oder Datenhybris? : Überlegungen zum Datenpositivismus der Molekularbiologie
Gramelsberger, G. (2017) · RWTH Publications (RWTH Aachen)
Metadata
- Authors
- Lazer, D., Kennedy, R., King, G., & Vespignani, A.
- Affiliations
- Northeastern University (US)
- Harvard University (US)
- Corresponding author
- Lazer, David
- Venue
- Science
- Published
- 2014-03-14
- Volume
- 343
- Issue
- 6176
- Pages
- 1203-1205
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2068181924
- Semantic Scholar
- 00d681cfff7f0224ace607e515f4ed510e792df9
- Cited by
- 2,523
- References
- 30
- Citations by year
- 2026: 113 · 2025: 101 · 2024: 121 · 2023: 150 · 2022: 168 · 2021: 229 · 2020: 273 · 2019: 261 · 2018: 272 · 2017: 249 · 2016: 271 · 2015: 219 · 2014: 91
- FWCI
- 208.16
- Type
- journal article
- Language
- en
- Open access
- green
- dash.harvard.edu
- Licence
- cc-by
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Epidemiology › Data-Driven Disease Surveillance
- Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- Google Flu Trends, influenza surveillance, big data, prediction error, data quality
- MeSH terms
- Centers for Disease Control and Prevention, U.S., Data Collection, Data Interpretation, Statistical, Disease Outbreaks, Humans, Influenza, Human, Office Visits, Prognosis, United States, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.