Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index; 5 of the 8 sources its claims cite were checked for support.

The Parable of Google Flu: Traps in Big Data Analysis

Open access

Abstract

Large errors in flu prediction were largely avoidable, which offers lessons for the use of big data.

Overview

CoChat Summary

TL;DR

The paper uses the failure of Google Flu Trends (GFT) to illustrate two traps in big data analysis: “big data hubris” and algorithm dynamics. It argues that GFT’s large flu-prediction errors were largely avoidable and that big data should supplement, not replace, traditional data and methods.

Key findings

  • In February 2013, GFT predicted more than double the proportion of doctor visits for influenza-like illness (ILI) than the CDC; from 21 August 2011 to 1 September 2013, GFT reported overly high flu prevalence in 100 out of 108 weeks, and it overshot the 2011–2012 season by more than 50%.
  • GFT errors were not random: they showed temporal autocorrelation and seasonality, meaning last week’s errors predicted this week’s errors and error direction/magnitude varied with time of year.
  • Simple alternatives outperformed GFT: out-of-sample mean absolute error was 0.486 for GFT, 0.311 for lagged CDC, and 0.232 for combined GFT and CDC; all differences were statistically significant at P < 0.05. Even 3-week-old CDC data did better than GFT at projecting current flu prevalence.
  • The authors attribute GFT’s problems to overfitting (initial GFT matched 50 million search terms to 1152 data points and missed the nonseasonal 2009 H1N1 pandemic) and to “algorithm dynamics,” including Google search algorithm changes and user behavior changes.

Methods

  • Design: policy forum/case study comparing GFT estimates with CDC ILI surveillance data and alternative forecasting models.
  • Data/sample: GFT and CDC flu prevalence estimates, including the period from 21 August 2011 to 1 September 2013; the initial GFT methodology reportedly fit 50 million search terms to 1152 data points.
  • Main analysis: comparison of GFT alone, lagged CDC data with 52-week seasonality variables, and a combined GFT + CDC model using lagged CDC estimates, lagged GFT error, and seasonality variables; reported mean absolute error and statistical significance; Google Correlate used to examine correlated search terms. Supplementary materials contain data and methods.

Limitations

  • The paper states that GFT never documented the 45 search terms used, released examples appear misleading, and Google Correlate is limited to national-level data whereas GFT was developed using regional correlations; therefore the original GFT analyses cannot be replicated from the information provided.
  • The paper states that Google cannot make its full data available to outsiders because of privacy issues, though it argues there is no such constraint on derivative, aggregated data.
  • Evident from the text: this is a commentary/policy forum rather than a full empirical article, with key data and methods placed in supplementary materials; the quantitative comparison depends on GFT and CDC data and model specifications not fully detailed in the main text.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1126/science.1248506

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

2 Valid · 2 Not valid · 2 Not checked · 8 Own results and interpretations

3 more claims

Create a free account to see all 14 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 5
  • Probably not a substantial claim. 2

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

6 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 30 references

Research works printed but not linked: 4

  1. McAfee A., Brynjolfsson E., Harv. Bus. Rev. 90, 60 (2012).
  2. Copeland P., et al.., Int. Soc. Negl. Trop. Dis. 2013, 3 (2013).
  3. Mustafaraj E., Metaxas P., in Proceedings of the Web-Sci10, Raleigh, NC, 26 and 27 April 2010 (Web Science Trust, 2010); http://journal.webscience.org/317/.
  4. Ratkiewicz J., et al.., in Proceedings of 5th International AAAI Conference on Weblogs and Social Media, San Francisco, CA, 7 to 11 August 2011 (AAAI, 2011), p. 297–304.

Cited by

See all 2,523

Metadata

Authors
Lazer, D., Kennedy, R., King, G., & Vespignani, A.
Affiliations
  • Northeastern University (US)
  • Harvard University (US)
Corresponding author
Lazer, David
Venue
Science
Published
2014-03-14
Volume
343
Issue
6176
Pages
1203-1205
ISSN
  • 0036-8075 (ISSN-L)
  • 1095-9203
OpenAlex
W2068181924
Cited by
2,523
References
30
Citations by year
2026: 113 · 2025: 101 · 2024: 121 · 2023: 150 · 2022: 168 · 2021: 229 · 2020: 273 · 2019: 261 · 2018: 272 · 2017: 249 · 2016: 271 · 2015: 219 · 2014: 91
FWCI
208.16
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Science (publishedVersion)
  • PubMed (publishedVersion)
  • Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
Topics
  • Health Sciences › Medicine › Epidemiology › Data-Driven Disease Surveillance
  • Social Sciences › Decision Sciences › Information Systems and Management › Big Data Technologies and Applications
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
Google Flu Trends, influenza surveillance, big data, prediction error, data quality
MeSH terms
Centers for Disease Control and Prevention, U.S., Data Collection, Data Interpretation, Statistical, Disease Outbreaks, Humans, Influenza, Human, Office Visits, Prognosis, United States, Social Media

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Parable of Google Flu: Traps in Big Data Analysis. https://app.cochat.ai/papers/10.1126/science.1248506
The paper
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The Parable of Google Flu: Traps in Big Data Analysis. Science. https://doi.org/10.1126/science.1248506

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.