Development and validation of a machine learning algorithm for predicting the risk of postpartum depression among pregnant women
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
OBJECTIVE: There is a scarcity in tools to predict postpartum depression (PPD). We propose a machine learning framework for PPD risk prediction using data extracted from electronic health records (EHRs). METHODS: Two EHR datasets containing data on 15,197 women from 2015 to 2018 at a single site, and 53,972 women from 2004 to 2017 at multiple sites were used as development and validation sets, respectively, to construct the PPD risk prediction model. The primary outcome was a diagnosis of PPD within 1 year following childbirth. A framework of data extraction, processing, and machine learning was implemented to select a minimal list of features from the EHR datasets to ensure model performance and to enable future point-of-care risk prediction. RESULTS: The best-performing model uses from clinical features related to mental health history, medical comorbidity, obstetric complications, medication prescription orders, and patient demographic characteristics. The model performances as measured by area under the receiver operating characteristic curve (AUC) are 0.937 (95% CI 0.912 - 0.962) and 0.886 (95% CI 0.879-0.893) in the development and validation datasets, respectively. The model performances were consistent when tested using data ending at multiple time periods during pregnancy and at childbirth. LIMITATIONS: The prevalence of PPD in the study data represented a treatment prevalence and is likely lower than the illness prevalence. CONCLUSIONS: EHRs and machine learning offer the ability to identify women at risk for PPD early in their pregnancy. This may facilitate scalable and timely prevention and intervention, reducing negative outcomes and the associated burden.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 1×
“The evaluation was performed using the held-out data set at WCM and the entire dataset from NYC-CDRN using AUC, sensitivity, specificity, and the Brier score (Hanley and McNeil, 1982).”
MethodDetection of Postnatal Depression: Development of the 10-item Edinburgh Postnatal Depression Scale
Cox, J. L., Holden, J. M., & Sagovsky, R. (1987) · British Journal of Psychiatry · doi:10.1192/bjp.150.6.782 · Cited by 14,183
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648 · cited 1×
“…scale using machine learning, as demonstrated by successful data-driven clinical decision support (Shortliffe and Sepúlveda, 2018) applications that assist with decision making across clinical conditions (Goldstein et al., 2017; Liang et al., 2019; Rajkomar et al., 2018; Tomašev et al., 2019).”
MethodSelf-citationEffects of perinatal mental disorders on the fetus and child
Stein, A., Pearson, R. M., Goodman, S. H., Rapa, E., Rahman, A., McCallum, M., Howard, L. M., & Pariante, C. M. (2014) · The Lancet · doi:10.1016/s0140-6736(14)61277-0 · Cited by 2,166 · cited 1×
“Long-term health effects of PPD to mothers, children, and family include increased maternal and infant mortality, increased hospitalizations, impaired mother-child bonding, and impaired long-term child development (Field, 2010; Jacques et al., 2019; Stein et al., 2014; Weobong et al., 2015).”
BackgroundIdentifying the women at risk of antenatal anxiety and depression: a systematic review
Biaggi, A., Conroy, S., Pawlby, S., & Pariante, C. M. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2015.11.014 · Cited by 1,657 · cited 2×
“Social determinants of health (SDoH), including poor marital relationship, low socioeconomic status, and stressful life events are also known contributors to increased PPD risk (Biaggi et al., 2016).”
BackgroundPostpartum depression effects on early interactions, parenting, and safety practices: a review
Field, T. (2010) · Infant Behavior and Development · doi:10.1016/j.infbeh.2009.10.005 · Cited by 1,493 · cited 1×
“Long-term health effects of PPD to mothers, children, and family include increased maternal and infant mortality, increased hospitalizations, impaired mother-child bonding, and impaired long-term child development (Field, 2010; Jacques et al., 2019; Stein et al., 2014; Weobong et al., 2015).”
BackgroundMonitoring Depression Treatment Outcomes With the Patient Health Questionnaire-9
L??we, B., Un??tzer, J., Callahan, C. M., Perkins, A. J., & Kroenke, K. (2004) · Medical Care · doi:10.1097/00005650-200412000-00006 · Cited by 1,354 · cited 3×
“We also did not compare effectiveness of primary prevention via the prediction algorithm to current widely recommended secondary prevention efforts via EPDS or PHQ-9 screening.”
BackgroundWisner, K. L., Sit, D. K. Y., McShea, M. C., Rizzo, D. M., Zoretich, R. A., Hughes, C. L., Eng, H. F., Luther, J. F., Wisniewski, S. R., Costantino, M. L., et al. (2013) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2013.87 · Cited by 1,091 · cited 1×
“The prevalence of PPD is estimated to affect as many as 1 in 7 mothers in the US (Hahn-Holbrook et al., 2017; Wisner et al., 2013), but underdiagnosis and lack of treatment for PPD are common, especially among women with low socioeconomic status (Biaggi et al., 2016; O’Connor et al., 2019).”
BackgroundA clinically applicable approach to continuous prediction of future acute kidney injury
Tomašev, N., Glorot, X., Rae, J. W., Zielinski, M., Askham, H., Saraiva, A., Mottram, A., Meyer, C., Ravuri, S., Protsyuk, I., et al. (2019) · Nature · doi:10.1038/s41586-019-1390-1 · Cited by 1,086 · cited 1×
“…scale using machine learning, as demonstrated by successful data-driven clinical decision support (Shortliffe and Sepúlveda, 2018) applications that assist with decision making across clinical conditions (Goldstein et al., 2017; Liang et al., 2019; Rajkomar et al., 2018; Tomašev et al., 2019).”
MethodCohen, L. S. (2006) · JAMA · doi:10.1001/jama.295.5.499 · Cited by 1,031
Goldstein, B. A., Navar, A. M., Pencina, M. J., & Ioannidis, J. P. A. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw042 · Cited by 936
Clinical decision support in the era of artificial intelligence
Shortliffe, E. H., & Sepúlveda, M. J. (2018) · JAMA · doi:10.1001/jama.2018.17163 · Cited by 920 · cited 1×
“…be collected and analyzed routinely on a large scale using machine learning, as demonstrated by successful data-driven clinical decision support (Shortliffe and Sepúlveda, 2018) applications that assist with decision making across clinical conditions (Goldstein et al., 2017; Liang et al., 2019;…”
MethodACOG Committee Opinion No. 757: Screening for Perinatal Depression
(2018) · Obstetrics & Gynecology · doi:10.1097/aog.0000000000002927 · Cited by 771
Hahn-Holbrook, J., Cornwell-Hinrichs, T., & Anaya, I. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00248 · Cited by 667 · cited 2×
“First, our study cohort as derived from the EHR in an urban academic medical center is not representative of the general US population suffering from PPD and differs from cohorts reported in previous studies with respect to PPD prevalence (Hahn-Holbrook et al., 2017).”
BackgroundResultValidation of a common data model for active safety surveillance research
Overhage, J. M., Ryan, P. B., Reich, C. G., Hartzema, A. G., & Stang, P. E. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000376 · Cited by 548
Stewart, D. E., & Vigod, S. (2016) · New England Journal of Medicine · doi:10.1056/nejmcp1607649 · Cited by 500
Postpartum psychiatric disorders
Meltzer-Brody, S., Howard, L. M., Bergink, V., Vigod, S., Jones, I., Munk-Olsen, T., Honikman, S., & Milgrom, J. (2018) · Nature Reviews Disease Primers · doi:10.1038/nrdp.2018.22 · Cited by 357
Postpartum depression screening: importance, methods, barriers, and recommendations for practice
Gjerdingen, D. K., & Yawn, B. P. (2007) · The Journal of the American Board of Family Medicine · doi:10.3122/jabfm.2007.03.060171 · Cited by 344 · cited 1×
“In addition to being “too little, too late” from a clinical perspective, these screening tools present major feasibility problems for both large and smaller health systems (Beck and Gable, 2000; Gjerdingen and Yawn, 2007).”
BackgroundIncorporating Recognition and Management of Perinatal Depression Into Pediatric Practice
Earls, M. F., Yogman, M. W., Mattson, G., Rafferty, J., COMMITTEE ON PSYCHOSOCIAL ASPECTS OF CHILD AND FAMILY HEALTH, Baum, R., Gambon, T., Lavin, A., & Wissow, L. (2019) · Pediatrics · doi:10.1542/peds.2018-3259 · Cited by 327 · cited 3×
“( Committee, 2018; Earls et al., 2019) With these automatically extractable features, an EHR-based prediction tool may assist with existing EHR interventions for screening to minimize variations across clinical practices in screening and information collection.”
BackgroundMethodO’Connor, E., Senger, C. A., Henninger, M. L., Coppola, E., & Gaynes, B. N. (2019) · JAMA · doi:10.1001/jama.2018.20865 · Cited by 316
Detection, Treatment, and Referral of Perinatal Depression and Anxiety by Obstetrical Providers
Goodman, J. H., & Tyer-Viola, L. (2010) · Journal of Women's Health · doi:10.1089/jwh.2008.1352 · Cited by 240 · cited 1×
“, 2019)Previous studies have reported that while the rates of screening and referrals for mental health care can be high when obstetricians recognize a risk for PPD, but they are low if symptoms are unnoticed by the care provider.(Goodman and Tyer-Viola, 2010) Our risk prediction model, by identifying women with eleva…”
BackgroundJacques, N., de Mola, C. L., Joseph, G., Mesenburg, M. A., & da Silveira, M. F. (2019) · Journal of Affective Disorders · doi:10.1016/j.jad.2018.09.055 · Cited by 107 · cited 1×
“Long-term health effects of PPD to mothers, children, and family include increased maternal and infant mortality, increased hospitalizations, impaired mother-child bonding, and impaired long-term child development (Field, 2010; Jacques et al., 2019; Stein et al., 2014; Weobong et al., 2015).”
BackgroundUsing Electronic Health Records and Machine Learning to Predict Postpartum Depression
Shuojia, W., Jyotishman, P., & Yiye, Z. (2019) · Studies in Health Technology and Informatics · doi:10.3233/shti190351 · Cited by 99
Sholle, E. T., Pinheiro, L. C., Adekkanattu, P., Davila, M. A., Johnson, S. B., Pathak, J., Sinha, S., Li, C., Lubansky, S. A., Safford, M. M., et al. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz040 · Cited by 89
Self-citationWeobong, B., ten Asbroek, A. H. A., Soremekun, S., Gram, L., Amenga-Etego, S., Danso, S., Owusu-Agyei, S., Prince, M., & Kirkwood, B. R. (2015) · BMJ Open · doi:10.1136/bmjopen-2014-006509 · Cited by 56 · cited 1×
“Long-term health effects of PPD to mothers, children, and family include increased maternal and infant mortality, increased hospitalizations, impaired mother-child bonding, and impaired long-term child development (Field, 2010; Jacques et al., 2019; Stein et al., 2014; Weobong et al., 2015).”
BackgroundLong, M. M., Cramer, R. J., Jenkins, J., Bennington, L., & Paulson, J. F. (2019) · Archives of Women's Mental Health · doi:10.1007/s00737-018-0876-4 · Cited by 49 · cited 1×
“( Long et al., 2019)Previous studies have reported that while the rates of screening and referrals for mental health care can be high when obstetricians recognize a risk for PPD, but they are low if symptoms are unnoticed by the care provider.”
Background
Cited by
Kim, J. Y., Hasan, A., Kellogg, K. C., Ratliff, W., Murray, S. G., Suresh, H., Valladares, A., Shaw, K., Tobey, D., Vidal, D. E., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000390 · Cited by 61
Richter, T., Fishbain, B., Richter-Levin, G., & Okon-Singer, H. (2021) · Journal of Personalized Medicine · doi:10.3390/jpm11100957 · Cited by 52
Postpartum depression: a developed and validated model predicting individual risk in new mothers
Munk-Olsen, T., Liu, X., Madsen, K. B., Kjeldsen, M.-M. Z., Petersen, L. V., Bergink, V., Skalkidou, A., Vigod, S. N., Frokjaer, V. G., Pedersen, C. B., et al. (2022) · Translational Psychiatry · doi:10.1038/s41398-022-02190-8 · Cited by 36
Metadata
- Authors
- Zhang, Y., Wang, S., Hermann, A., Joly, R., & Pathak, J.
- Venue
- Journal of Affective Disorders
- Publisher
- Elsevier BV
- Published
- 2021-01
- Volume
- 279
- Pages
- 1-8
- ISSN
- 0165-0327 (ISSN-L)
- OpenAlex
- W3090927654
- Cited by
- 164
- References
- 27
- FWCI
- 10.50
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.