The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Combining Search, Social Media, and Traditional Data Sources to Improve Influenza Surveillance
Open accessAbstract
We present a machine learning-based methodology capable of providing real-time ("nowcast") and forecast estimates of influenza activity in the US by leveraging data from multiple data sources including: Google searches, Twitter microblogs, nearly real-time hospital visit records, and data from a participatory surveillance system. Our main contribution consists of combining multiple influenza-like illnesses (ILI) activity estimates, generated independently with each data source, into a single prediction of ILI utilizing machine learning ensemble approaches. Our methodology exploits the information in each data source and produces accurate weekly ILI predictions for up to four weeks ahead of the release of CDC's ILI reports. We evaluate the predictive ability of our ensemble approach during the 2013-2014 (retrospective) and 2014-2015 (live) flu seasons for each of the four weekly time horizons. Our ensemble approach demonstrates several advantages: (1) our ensemble method's predictions outperform every prediction using each data source independently, (2) our methodology can produce predictions one week ahead of GFT's real-time estimates with comparable accuracy, and (3) our two and three week forecast estimates have comparable accuracy to real-time predictions using an autoregressive model. Moreover, our results show that considerable insight is gained from incorporating disparate data streams, in the form of social media and crowd sourced data, into influenza predictions in all time horizons.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Support Vector Regression Machines
Drucker, H., Burges, C. J. C., Kaufman, L., Smola, A. J., & Vapnik, V. N. (1996) · Cited by 4,658
Detecting influenza epidemics using search engine query data
Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009) · Nature · doi:10.1038/nature07634 · Cited by 4,295
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00117832 · Cited by 1,912
Signorini, A., Segre, A. M., & Polgreen, P. M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0019467 · Cited by 1,235
Using Internet Searches for Influenza Surveillance
Polgreen, P. M., Chen, Y., Pennock, D. M., & Nelson, F. D. (2008) · Clinical Infectious Diseases · doi:10.1086/593098 · Cited by 566
Butler, D. (2013) · Nature · doi:10.1038/494155a · Cited by 514
Broniatowski, D. A., Paul, M. J., & Dredze, M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0083672 · Cited by 442
Self-citationCook, S., Conrad, C., Fowlkes, A. L., & Mohebbi, M. H. (2011) · PLoS ONE · doi:10.1371/journal.pone.0023610 · Cited by 381
Accurate estimation of influenza epidemics using Google search data via ARGO
Yang, S., Santillana, M., & Kou, S. C. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1515373112 · Cited by 373
Self-citationOlson, D. R., Konty, K. J., Paladini, M., Viboud, C., & Simonsen, L. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003256 · Cited by 320
Chan, E. H., Sahai, V., Conrad, C., & Brownstein, J. S. (2011) · PLoS Neglected Tropical Diseases · doi:10.1371/journal.pntd.0001206 · Cited by 284
Self-citationSeparating Fact from Fear: Tracking Flu Infections on Twitter
Lamb, A., Paul, M. J., & Dredze, M. H. (2013) · Cited by 247
Self-citationTwitter Improves Influenza Forecasting
Paul, M. J., Dredze, M., & Broniatowski, D. (2014) · PLoS Currents · doi:10.1371/currents.outbreaks.90b9ed0f59bae4ccaa683a39865d9117 · Cited by 238
Self-citationMonitoring Influenza Epidemics in China with Search Query from Baidu
Yuan, Q., Nsoesie, E. O., Lv, B., Peng, G., Chunara, R., & Brownstein, J. S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0064323 · Cited by 233
Self-citationYang, W., Karspeck, A., & Shaman, J. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003583 · Cited by 201
Olson, D. R., Heffernan, R. T., Paladini, M., Konty, K., Weiss, D., & Mostashari, F. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040247 · Cited by 198
McIver, D. J., & Brownstein, J. S. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003581 · Cited by 197
Self-citationFlu Near You: Crowdsourced Symptom Reporting Spanning 2 Influenza Seasons
Smolinski, M. S., Crawley, A. W., Baltrusaitis, K., Chunara, R., Olsen, J. M., Wójcik, O., Santillana, M., Nguyen, A., & Brownstein, J. S. (2015) · American Journal of Public Health · doi:10.2105/ajph.2015.302696 · Cited by 192
Self-citationWhat Can Digital Disease Detection Learn from (an External Revision to) Google Flu Trends?
Santillana, M., Zhang, D. W., Althouse, B. M., & Ayers, J. W. (2014) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2014.05.020 · Cited by 183
Self-citationGlobal Disease Monitoring and Forecasting with Wikipedia
Generous, N., Fairchild, G., Deshpande, A., Del Valle, S. Y., & Priedhorsky, R. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003892 · Cited by 183
Paolotti, D., Carnahan, A., Colizza, V., Eames, K., Edmunds, J., Gomes, G., Koppeschaar, C., Rehn, M., Smallenburg, R., Turbelin, C., et al. (2014) · Clinical Microbiology and Infection · doi:10.1111/1469-0691.12477 · Cited by 177
Nagar, R., Yuan, Q., Freifeld, C. C., Santillana, M., Nojima, A., Chunara, R., & Brownstein, J. S. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3416 · Cited by 160
Self-citationInference of seasonal and pandemic influenza transmission dynamics
Yang, W., Lipsitch, M., & Shaman, J. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1415012112 · Cited by 159
Evaluation of Internet-Based Dengue Query Data: Google Dengue Trends
Gluskin, R. T., Johansson, M. A., Santillana, M., & Brownstein, J. S. (2014) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0002713 · Cited by 154
Self-citationLipsitch, M., Finelli, L., Heffernan, R. T., Leung, G. M., & Redd, S. C. (2011) · Biosecurity and Bioterrorism Biodefense Strategy Practice and Science · doi:10.1089/bsp.2011.0007 · Cited by 144
Dalton, C. B., Dürrheim, D. N., Fejsa, J., Francis, L., Carlson, S., d’Espaignet, E. T., & Tuyl, F. (2009) · Communicable Diseases Intelligence · doi:10.33321/cdi.2009.33.34 · Cited by 90
Using Clinicians’ Search Query Data to Monitor Influenza Epidemics
Santillana, M., Nsoesie, E., Mekaru, S. R., Scales, D., & Brownstein, J. (2014) · Clinical Infectious Diseases · doi:10.1093/cid/ciu647 · Cited by 81
Self-citationUsing Networks to Combine “Big Data” and Traditional Surveillance to Improve Influenza Predictions
Davidson, M. W., Haim, D. A., & Radin, J. M. (2015) · Scientific Reports · doi:10.1038/srep08154 · Cited by 68
Using search queries for malaria surveillance, Thailand
Ocampo, A., Chunara, R., & Brownstein, J. S. (2013) · Malaria Journal · doi:10.1186/1475-2875-12-390 · Cited by 65
Self-citationViboud, C. G., Charu, V., Olson, D. R., Ballesteros, S. P., Gog, J. R., Khan, F. L., Grenfell, B. T., & Simonsen, L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0102429 · Cited by 63
Flu Gone Viral: Syndromic Surveillance of Flu on Twitter Using Temporal Topic Models
Chen, L., Hossain, K. S. M. T., Butler, P., Ramakrishnan, N., & Prakash, B. A. (2014) · Proceedings · doi:10.1109/icdm.2014.137 · Cited by 61
Optimizing Provider Recruitment for Influenza Surveillance Networks
Scarpino, S. V., Dimitrov, N. B., & Ancel, L. W. (2012) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002472 · Cited by 47
Global Disease Monitoring and Forecasting with Wikipedia
Generous, N., Fairchild, G. C., Deshpande, A., Valle, S. Y. D., & Priedhorsky, R. (2016) · Online Journal of Public Health Informatics · doi:10.5210/ojphi.v8i1.6530 · Cited by 41
A New Approach to Monitoring Dengue Activity
Madoff, L. C., Fisman, D. N., & Kass‐Hout, T. (2011) · PLoS neglected tropical diseases · doi:10.1371/journal.pntd.0001215 · Cited by 35
Reengineering real time outbreak detection systems for influenza epidemic monitoring.
Brownstein, J. S., & Mandl, K. D. (2006) · PubMed · Cited by 20
Self-citationARGO: a model for accurate estimation of influenza epidemics using Google search dataPreprint
Yang, S., Santillana, M., & Kou, S. C. (2015) · arXiv (Cornell University) · Cited by 3
Self-citation
Cited by
Statistical physics of vaccination
Wang, Z., Bauch, C. T., Bhattacharyya, S., d'Onofrio, A., Manfredi, P., Perc, M., Perra, N., Salathé, M., & Zhao, D. (2016) · Physics Reports · doi:10.1016/j.physrep.2016.10.006 · Cited by 872
Accurate estimation of influenza epidemics using Google search data via ARGO
Yang, S., Santillana, M., & Kou, S. C. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1515373112 · Cited by 373
Self-citationTracking COVID-19 using online search
Lampos, V., Majumder, M. S., Yom-Tov, E., Edelstein, M., Moura, S., Hamada, Y., Rangaka, M. X., McKendry, R. A., & Cox, I. J. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00384-w · Cited by 164
Srinivas, S., & Ravindran, A. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.02.022 · Cited by 147
Johansson, M. A., Reich, N. G., Hota, A., Brownstein, J. S., & Santillana, M. (2016) · Scientific Reports · doi:10.1038/srep33707 · Cited by 120
Self-citation
Metadata
- Authors
- Santillana, M., Nguyen, A. T., Dredze, M., Paul, M. J., Nsoesie, E. O., & Brownstein, J. S.
- Venue
- PLOS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-10-29
- Volume
- 11
- Issue
- 10
- Pages
- e1004513
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2148330046
- Cited by
- 420
- References
- 41
- FWCI
- 26.15
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.