Phenomenological forecasting of disease incidence using heteroskedastic Gaussian processes: A dengue case study
Open accessThe DOI matches its Crossref record; 36 of the 38 references it cites were found in a registry or an index, and 2 are not checked yet.
Abstract
of peak week, peak incidence during that week, and total season incidence across each of several seasons. our team was one of the winners of that competition, outperforming other teams in multiple targets/locales. In this paper we report on our methodology, a large component of which, surprisingly, ignores the known biology of epidemics at large-for example, relationships between dengue transmission and environmental factors-and instead relies on flexible nonparametric nonlinear Gaussian process (GP) regression fits that "memorize" the trajectories of past seasons, and then "match" the dynamics of the unfolding season to past ones in real-time. Our phenomenological approach has advantages in situations where disease dynamics are less well understood, or where measurements and forecasts of ancillary covariates like precipitation are unavailable, and/or where the strength of association with cases are as yet unknown. In particular, we show that the GP approach generally outperforms a more classical generalized linear (autoregressive) model (GLM) that we developed to utilize abundant covariate information. We illustrate variations of our method(s) on the two benchmark locales alongside a full summary of results submitted by other contest competitors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Variance models were developed for heteroscedastic data scenarios and have been used for decades [13, 14, 18, 40] in fields as diverse as macroeconomics [13], epidemiology [23] and ecology [37].”
From A Bayesian framework to model variance of grain yield response to plant density (2025) · cites this paper as [23]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,919
Cressie, N. A. C. (1993) · Wiley Series in Probability and Statistics · doi:10.1002/9781119115151 · Cited by 6,507
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793 · cited 2×
“le range of values { for each of thirteen weeks evenly spanning the season (i.e., one forecast every 4 weeks, starting from week zero). Competition evaluation focused on logarithmic scores (following Gneiting and Raftery, 2007, Example 3) over just the rst 24 weeks. As an example of forecasts for the three targets, re…”
BackgroundMethodMatheron, G. (1963) · Economic Geology · doi:10.2113/gsecongeo.58.8.1246 · Cited by 4,845 · cited 2×
“Gaussian process (GP) regression is an established nonparametric modeling apparatus originating in the spatial statistics literature, where it is also known as kriging [Cressie (1993), Matheron (1963)].”
BackgroundMethodAn Improved In Situ and Satellite SST Analysis for Climate
Reynolds, R. W., Rayner, N. A., Smith, T. M., Stokes, D. C., & Wang, W. (2002) · Journal of Climate · doi:10.1175/1520-0442(2002)015<1609:aiisas>2.0.co;2 · Cited by 4,559 · cited 2×
“Our team obtained current monthly SOI and sea surface temperatures [Reynolds et al. (2002)] and aligned these with the weekly case data to augment our data set.”
MethodMaking and Evaluating Point Forecasts
Gneiting, T. (2011) · Journal of the American Statistical Association · doi:10.1198/jasa.2011.r10138 · Cited by 1,154 · cited 3×
“For example, median week, median highest observed weekly incidence, and median sum over the weeks, respectively, are appropriate under the mean-absolute error metric used for contest evaluation [Gneiting (2011)]. However, medians may not be appropriate, for example, when incidences are multimodal (see, e.g, 1998/1999…”
BackgroundMethodModern Applied Statistics with S-Plus
Venables, W. N., & Ripley, B. D. (1994) · Statistics and Computing · doi:10.1007/978-1-4899-2819-1 · Cited by 806
Mordecai, E. A., Cohen, J. M., Evans, M. V., Gudapati, P., Johnson, L. R., Lippi, C. A., Miazgowicz, K., Murdock, C. C., Rohr, J. R., Ryan, S. J., et al. (2017) · PLOS Neglected Tropical Diseases · doi:10.1371/journal.pntd.0005568 · Cited by 785 · cited 2×
“For vector-borne disease, the value of R0 depends on vector (here mosquito) traits such as mortality rates, biting rates, reproduction, etc. Full details of parameterized versions of R0 from data are given by Mordecai et al. (2017).”
BackgroundSelf-citationImpact of daily temperature fluctuations on dengue virus transmission by Aedes aegypti
Lambrechts, L., Paaijmans, K. P., Fansiri, T., Carrington, L. B., Kramer, L. D., Thomas, M. B., & Scott, T. W. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1101377108 · Cited by 710 · cited 1×
“…covariates, as it is widely known that environmental factors impact the dynamics of mosquito vectors and thus transmission [e.g., Barrera, Amador and MacKay (2011), Johansson, Cummings and Glass (2009), Lambrechts et al. (2011), Moore et al. (1978), Stewart-Ibarra et al. (2013), Xu et al. (2016)].”
BackgroundOptimal temperature for malaria transmission is dramatically lower than previously predicted
Mordecai, E. A., Paaijmans, K. P., Johnson, L. R., Balzer, C., Ben‐Horin, T., de Moor, E., McNally, A., Pawar, S., Ryan, S. J., Smith, T. C., et al. (2013) · Ecology Letters · doi:10.1111/ele.12015 · Cited by 680 · cited 3×
“This measure was derived and parameterized using previously published data on how mosquito traits depend on temperature, following methods developed in Mordecai et al. (2013) and Johnson et al. (2015).”
MethodSelf-citationStochastic Kriging for Simulation Metamodeling
Ankenman, B., Nelson, B. L., & Staum, J. (2010) · Operations Research · doi:10.1287/opre.1090.0754 · Cited by 622 · cited 1×
“r version clearly arises as a special case. However a dierent approach may be desired for cases where severity is likely to smoothly vary with the other inputs. In that case, stochastic Kriging (SK, Ankenman et al., 2010) may present an attractive alternative. However, note that SK requires replication of observations…”
BackgroundJohansson, M. A., Cummings, D. A. T., & Glass, G. E. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000168 · Cited by 292 · cited 3×
“Previous studies have identified correlations between the Southern Oscillation Index (SOI) and other El Niño sea surface temperature indices and dengue (e.g., Gagnon et al., 2001; Johansson et al., 2009).”
BackgroundBarrera, R., Amador, M., & MacKay, A. J. (2011) · PLoS Neglected Tropical Diseases · doi:10.1371/journal.pntd.0001378 · Cited by 292
Using climate to predict infectious disease epidemics.
Kuhn, K. G., Campbell-Lendrum, D. H., Haines, A., & Cox, J. S. H. (2005) · LSHTM Research Online (London School of Hygiene and Tropical Medicine) · Cited by 252
Gneiting, T., Larson, K., Westrick, K., Genton, M. G., & Aldrich, E. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000456 · Cited by 246
Climate variation drives dengue dynamics
Xu, L., Stige, L. C., Chan, K.-S., Zhou, J., Yang, J., Sang, S., Wang, M., Yang, Z., Yan, Z., Jiang, T., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1618558114 · Cited by 233 · cited 2×
“e indicated with blue error bars. vectors and thus transmission dynamics (e.g., Moore et al., 1978; Johansson et al., 2009; Barrera et al., 2011; Lambrechts et al., 2011; Stewart-Ibarra et al., 2013; Xu et al., 2016). The data provided included weekly precipitation, temperatures (min, max, and average), habitat indice…”
BackgroundInfluentialStewart Ibarra, A. M., Ryan, S. J., Beltrán, E., Mejía, R., Silva, M., & Muñoz, Á. (2013) · PLoS ONE · doi:10.1371/journal.pone.0078263 · Cited by 229 · cited 2×
“…covariates, as it is widely known that environmental factors impact the dynamics of mosquito vectors and thus transmission [e.g., Barrera, Amador and MacKay (2011), Johansson, Cummings and Glass (2009), Lambrechts et al. (2011), Moore et al. (1978), Stewart-Ibarra et al. (2013), Xu et al. (2016)].”
BackgroundResultSelf-citationDengue vaccine: WHO position paper – July 2016.
(2016) · PubMed · Cited by 217
Forecasting seasonal influenza with a state-space SIR model
Osthus, D., Hickmann, K. S., Caragea, P. C., Higdon, D., & Valle, S. Y. D. (2017) · The Annals of Applied Statistics · doi:10.1214/16-aoas1000 · Cited by 187 · cited 1×
“For instance, our approach is likely well suited to modeling influenza cases in the US, as approached by Osthus et al. (2017) using a state-space version of an SIR model.”
MethodDengue epidemics and the El Niño Southern Oscillation
Gagnon, A. S., Bush, A., & Smoyer-Tomic, K. E. (2001) · Climate Research · doi:10.3354/cr019035 · Cited by 176
Climate and Non-Climate Drivers of Dengue Epidemics in Southern Coastal Ecuador
Stewart-Ibarra, A. M., & Lowe, R. (2013) · The American Society of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.12-0478 · Cited by 162 · cited 2×
“Other previous studies indicate similar long lags between environmental covariates and incidence measures for dengue [Johansson, Cummings and Glass (2009), Xu et al. (2016), Stewart-Ibarra and Lowe (2013), Stewart-Ibarra et al. (2013)].”
ResultlaGP: Large-Scale Spatial Modeling via Local Approximate Gaussian Processes in R
Gramacy, R. B. (2016) · Journal of Statistical Software · doi:10.18637/jss.v072.i01 · Cited by 162
Self-citationMoore, C. G., Cline, B. L., Ruiz-Tibén, E., Lee, D., Romney-Joseph, H., & Rivera-Correa, E. (1978) · The American Journal of Tropical Medicine and Hygiene · doi:10.4269/ajtmh.1978.27.1225 · Cited by 159
Seasonal Forecasts, Climatic Change and Human Health
Thomson, M. C., García‐Herrera, R., & Beniston, M. (2008) · Springer eBooks · doi:10.1007/978-1-4020-6877-5 · Cited by 137 · cited 1×
“For example, knowing well in advance the numbers of cases that are expected and when they will occur allows preparation via education and community mobilization campaigns, reallocation of resources to highrisk areas (people, insecticide, diagnostic reagents), or re-training of physicians to recognize symptoms and to t…”
Background[Design and Analysis of Computer Experiments]: Rejoinder
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012420 · Cited by 132
Modeling Nonstationary Processes Through Dimension Expansion
Bornn, L., Shaddick, G., & Zidek, J. V. (2012) · Journal of the American Statistical Association · doi:10.1080/01621459.2011.646919 · Cited by 118 · cited 1×
“The introduction of a latent coordinate has recently been proposed as a remedy for adapting a stationary GP to nonstationary data (Bornn et al., 2012).”
BackgroundSuperensemble forecasts of dengue outbreaks
Yamana, T. K., Kandula, S., & Shaman, J. L. (2016) · Journal of The Royal Society Interface · doi:10.1098/rsif.2016.0410 · Cited by 110 · cited 2×
“39 F3 37.60 6.71 954.43 SEF12 21.74 5.70 630.82 SEF123 20.76 6.30 733.05 Table 2: MAE results for San Juan from forecasts provided by Yamana et al., comparable to the right panel of Table 1. Recently Yamana et al. (2016) built predictive model(s) for the San Juan data, including so-called superensembles in a spirit si…”
MethodInfluentialElderd, B. D., Dukić, V., & Dwyer, G. (2006) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0600816103 · Cited by 100 · cited 1×
“It has preciously been noted that uncertainty in parameter values and in model structure for SIR-type models can result poor prediction and in sub-optimal and more costly interventions (Elderd et al., 2006).”
BackgroundUnderstanding uncertainty in temperature effects on vector‐borne disease: a Bayesian approach
Johnson, L. R., Ben-Horin, T., Lafferty, K. D., McNally, A., Mordecai, E., Paaijmans, K. P., Pawar, S., & Ryan, S. J. (2015) · Ecology · doi:10.1890/13-1964.1 · Cited by 94 · cited 2×
“This measure was derived and parameterized using previously published data on how mosquito traits depend on temperature, following methods developed in Mordecai et al. (2013) and Johnson et al. (2015).”
MethodSelf-citationToward an early warning system for dengue prevention: modeling climate impact on dengue transmission
Dégallier, N., Favier, C., Menkès, C., Lengaigne, M., Ramalho, W. M., Souza, R., Servain, J., & Boulanger, J. (2009) · Climatic Change · doi:10.1007/s10584-009-9747-3 · Cited by 80 · cited 3×
“For example, knowing well in advance the numbers of cases that are expected and when they will occur allows preparation via education and community mobilization campaigns, reallocation of resources (people, insecticide, diagnostic reagents) to high-risk areas, or retraining of physicians to recognize symptoms and to t…”
BackgroundInfectious disease prediction with kernel conditional density estimation
Ray, E. L., Sakrejda, K., Lauer, S. A., Johansson, M. A., & Reich, N. G. (2017) · Statistics in Medicine · doi:10.1002/sim.7488 · Cited by 69 · cited 2×
“involve direct modeling of conditional distributions (e.g., Gneiting et al., 2006). A disadvantage here is bespoke implementation | simple lm and glm commands no longer suce. We have been pointed to Ray et al. (2017) who are perusing a promising, related approach in a disease modeling context. Their method utilizes ke…”
BackgroundMethodA Statistical Framework for the Adaptive Management of Epidemiological Interventions
Merl, D., Johnson, L. R., Gramacy, R. B., & Mangel, M. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005807 · Cited by 54 · cited 2×
“Turning things around, hetGP-based simulated forecasts could be fed into mechanistic models of interventions and their costs, as entertained in the setups of Ludkovski and Niemi (2010), Merl et al. (2009) and Hu and Ludkovsk (2017).”
BackgroundSelf-citationBayesian Emulation and Calibration of a Dynamic Epidemic Model for A/H1N1 Influenza
Farah, M., Birrell, P., Conti, S., & Angelis, D. D. (2014) · Journal of the American Statistical Association · doi:10.1080/01621459.2014.934453 · Cited by 53 · cited 1×
“For example Farah et al. (2014) use GPs to emulate an SIR-type computer model in a forecasting framework for influenza, and Hu and Ludkovsk (2017) deploy GPs within a stochastic control framework involving a continuous time Markov process inspired by SIR models.”
MethodSequential Design for Ranking Response Surfaces
Hu, R., & Ludkovski, M. (2017) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/15m1045168 · Cited by 40 · cited 2×
“Turning things around, hetGP-based simulated forecasts could be fed into mechanistic models of interventions and their costs, as entertained in the setups of Ludkovski and Niemi (2010), Merl et al. (2009) and Hu and Ludkovsk (2017).”
BackgroundMethodOptimal Dynamic Policies for Influenza Management
Ludkovski, M., & Niemi, J. (2010) · Statistical Communications in Infectious Diseases · doi:10.2202/1948-4690.1020 · Cited by 34 · cited 1×
“Turning things around, hetGP-based simulated forecasts could be fed into mechanistic models of interventions and their costs, as entertained in the setups of Ludkovski and Niemi (2010), Merl et al. (2009) and Hu and Ludkovsk (2017).”
BackgroundPredictive modeling of cholera outbreaks in Bangladesh
Koepke, A. A., Longini, I. M., Halloran, M. E., Wakefield, J., & Minin, V. N. (2016) · The Annals of Applied Statistics · doi:10.1214/16-aoas908 · Cited by 27 · cited 1×
“Applications such as cholera in Bangladesh, as described by Koepke et al. (2016), would be more challenging as only a single season of data is available.”
BackgroundWhen is the mode functional the Bayes classifier?
Gneiting, T. (2017) · Stat · doi:10.1002/sta4.148 · Cited by 12
Research works printed but not linked: 3
- BINOIS, M., GRAMACY, R. B. and LUDKOVSKI, M. (2016). Practical heteroskedastic Gaussian process modeling for large simulation experiments. arXiv preprint, arXiv:1611.05902. · arXiv
- JOHNSON, L. R., GRAMACY, R. B., COHEN, J., MORDECAI, E., MURDOCK, C., ROHR, J., RYAN, S. J., STEWART-IBARRA, A. M. and WEIKEL, D. (2018). Supplement to 'Phenomenological forecasting of disease incidence using heteroskedastic Gaussian processes: A dengue case study.' DOI:10.1214/17-AOAS1090SUPPA, DOI:10.1214/17-AOAS1090SUPPB. · doi:10.1214/17-aoas1090suppb
- GRAMACY, R. B. (2014). laGP : Local approximate Gaussian process regression. R package version 1.1-4. · doi:10.32614/cran.package.lagp
Cited by
An open challenge to advance probabilistic forecasting for dengue epidemics
Johansson, M. A., Apfeldorf, K. M., Dobson, S., Devita, J., Buczak, A. L., Baugher, B., Moniz, L. J., Bagley, T., Babin, S. M., Guven, E., et al. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1909865116 · Cited by 218
“Ensemble approaches were used by almost half the teams (7 of 16) (20, 21, 30) and on average, had better calibration and higher forecast skill than forecasts generated from single models.”
Self-citationBartlow, A. W., Manore, C., Xu, C., Kaufeld, K. A., Del Valle, S., Ziemann, A., Fairchild, G., & Fair, J. M. (2019) · Veterinary Sciences · doi:10.3390/vetsci6020040 · Cited by 162
Understanding how temperature shifts could impact infectious disease
Rohr, J. R., & Cohen, J. M. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000938 · Cited by 144
Self-citationForecasting dengue epidemics using a hybrid methodology
Chakraborty, T., Chattopadhyay, S., & Ghosh, I. (2019) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2019.121266 · Cited by 136
The Role of Vector Trait Variation in Vector-Borne Disease Dynamics
Cator, L. J., Johnson, L. R., Mordecai, E. A., Moustaid, F. E., Smallwood, T. R. C., LaDeau, S. L., Johansson, M. A., Hudson, P. J., Boots, M., Thomas, M. B., et al. (2020) · Frontiers in Ecology and Evolution · doi:10.3389/fevo.2020.00189 · Cited by 137
“Under the previous approach, abundance (V) is directly associated with temperature by fitting a time-series model or where abundance is assumed, a priori, to follow a sinusoidal function (Bacaër and Guernaoui, 2006; Bacaër, 2007; Bacaër and Ouifki, 2007; Bacaër and Ait Dads, 2012; Hoshi et al., 2014; Johnson et al., 2…”
BackgroundMethodSelf-citation
Metadata
- Authors
- Johnson, L. R., Gramacy, R. B., Cohen, J., Mordecai, E., Murdock, C., Rohr, J., Ryan, S. J., Stewart-Ibarra, A. M., & Weikel, D.
- Affiliations
- Virginia Tech (US)
- University of South Florida (US)
- Stanford University (US)
- University of Georgia (US)
- University of Florida (US)
- University of Michigan (US)
- ORCID
- Venue
- The Annals of Applied Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2018-03-01
- Volume
- 12
- Issue
- 1
- Pages
- 27-66
- ISSN
- 1932-6157 (ISSN-L)
- 1941-7330
- PMC
- PMC11017302
- OpenAlex
- W2585052966
- Cited by
- 42
- References
- 37
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 2 · 2023: 2 · 2022: 5 · 2021: 6 · 2020: 9 · 2019: 7 · 2018: 3
- FWCI
- 5.96
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- The Annals of Applied Statistics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- Project Euclid (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (R01GM109499, 1640780, 1241889, R01TW010286-01, 1518681, 1521702)
- U.S. Department of Defense
- U.S. Department of Health and Human Services
- U.S. Department of Homeland Security
- U.S. Department of Commerce
- Australian Government
- National Science and Technology Council
- National Institutes of Health (EF-1241889)
- National Oceanic and Atmospheric Administration
- Stanford Woods Institute for the Environment
- National Institute of General Medical Sciences (R01GM109499)
- Fogarty International Center (R01TW010286)
- Stanford University Center for Innovation in Global Health
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Mosquito-borne diseases and control
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Health Sciences › Medicine › Epidemiology › Data-Driven Disease Surveillance
- Keywords
- disease incidence forecasting, dengue, nonparametric regression, phenological forecasting, epidemic dynamics, seasonal occurrence, generalized linear models, forecasting competitions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.