Errors, Omissions, and Outliers in Hourly Vital Signs Measurements in Intensive Care
The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Objective: To empirically examine the prevalence of errors, omissions, and outliers in hourly vital signs recorded in the ICU. Design: Retrospective analysis of vital signs measurements from a large-scale clinical data warehouse (Multiparameter Intelligent Monitoring in Intensive Care III). Setting: Data were collected from the medical, surgical, cardiac, and cardiac surgery ICUs of a tertiary medical center in the United States. Patients: We analyzed data from approximately 48,000 ICU stays including approximately 28 million vital signs measurements. Interventions: None. Measurements and Main Results: We used the vital sign day as our unit of measurement, defined as all the recordings from a single patient for a specific vital sign over a single 24-hour period. Approximately 30–40% of vital sign days included at least one gap of greater than 70 minutes between measurements. Between 3% and 10% of blood pressure measurements included logical inconsistencies. With the exception of pulse oximetry vital sign days, the readings in most vital sign days were normally distributed. We found that 15–38% of vital sign days contained at least one statistical outlier, of which 6–19% occurred simultaneously with outliers in other vital signs. Conclusions: We found a significant number of missing, erroneous, and outlying vital signs measurements in a large ICU database. Our results provide empirical evidence of the nonrepresentativeness of hourly vital signs. Additional studies should focus on determining optimal sampling frequencies for recording vital signs in the ICU.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Especially raw data, collected through continuous patient monitoring in the ICU environment, is prone to measurement errors [10,39].”
From Moving from bytes to bedside: a systematic review on the use of artificial intelligence in the intensive care unit (2021) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480
Sample criteria for testing outlying observations
Grubbs, F. E. (1950) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729885 · Cited by 1,631 · cited 1×
“For normal or log-normal VSDs, Grubbs’ test (55) and the generalized extreme studentized deviate test (56) were applied to raw and log-transformed measurements, respectively.”
MethodSaeed, M., Villarroel, M., Reisner, A. T., Clifford, G., Lehman, L.-W., Moody, G., Heldt, T., Kyaw, T. H., Moody, B., & Mark, R. G. (2011) · Critical Care Medicine · doi:10.1097/ccm.0b013e31820a92c6 · Cited by 1,162
Pirracchio, R., Petersen, M. L., Carone, M., Rigon, M. R., Chevret, S., & van der Laan, M. J. (2015) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(14)70239-5 · Cited by 374
Diagnosis code assignment: models and evaluation metrics
Perotte, A., Pivovarov, R., Natarajan, K., Weiskopf, N., Wood, F., & Elhadad, N. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002159 · Cited by 244
Celi, L. A., Hinske Christian, L., Alterovitz, G., & Szolovits, P. (2008) · Critical Care · doi:10.1186/cc7140 · Cited by 52
Research works printed but not linked: 55
- Verduijn, 2008, Individual and joint expert judgments as reference standards in artifact detection., J Am Med Inform Assoc, 15, 227, 10.1197/jamia.M2493 · DOI
- Cunningham, 1994, The practical management of artifact in computerised physiological data., Int J Clin Monit Comput, 11, 211, 10.1007/BF01139872 · doi:10.1007/bf01139872
- Saeed, 2002, MIMIC II: A massive temporal ICU patient database to support research in intelligent patient monitoring., Comput Cardiol, 29, 641, 10.1109/CIC.2002.1166854 · DOI · Paper page
- Eshelman, 2008, Development and evaluation of predictive alerts for hemodynamic instability in ICU patients., AMIA Annu Symp Proc, 2008, 379
- Ennett, 2008, Predicting respiratory instability in the ICU., Conf Proc IEEE Eng Med Biol Soc, 2008, 2848
- Cao, 2008, Predicting ICU hemodynamic instability using continuous multiparameter trends., Conf Proc IEEE Eng Med Biol Soc, 2008, 3803
- Jia, 2008, Risk factors for ARDS in patients receiving mechanical ventilation for > 48 h., Chest, 133, 853, 10.1378/chest.07-1121 · DOI · Paper page
- Aboukhalil, 2008, Reducing false alarm rates for critical arrhythmias using the arterial blood pressure waveform., J Biomed Inform, 41, 442, 10.1016/j.jbi.2008.03.003 · DOI · Paper page
- Sun, 2009, The cardiac output from blood pressure algorithms trial., Crit Care Med, 37, 72, 10.1097/CCM.0b013e3181930174 · DOI
- Lee, 2010, An investigation of patterns in hemodynamic data indicative of impending hypotension in intensive care., Biomed Eng Online, 9, 62, 10.1186/1475-925X-9-62 · DOI
- Ghaffari, 2010, Parallel processing of ECG and blood pressure waveforms for detection of acute hypotensive episodes: A simulation study using a risk scoring model., Comput Methods Biomech Biomed Engin, 13, 197, 10.1080/10255840903099711 · DOI
- Rocha, 2010, Wavelet based time series forecast with application to acute hypotensive episodes prediction., Conf Proc IEEE Eng Med Biol Soc, 2010, 2403
- Sow, 2010, Real-time prognosis of ICU physiological data streams., Conf Proc IEEE Eng Med Biol Soc, 2010, 6785
- Cao, 2010, Heuristics to determine ventilation times of ICU patients from the MIMIC-II database., Conf Proc IEEE Eng Med Biol Soc, 2010, 5213
- Zhang, 2010, Evaluation of monitoring cardiac output by long time interval analysis of a radial arterial blood pressure waveform using the MIMIC II database., Conf Proc IEEE Eng Med Biol Soc, 2010, 5217
- Li, 2010, Heartbeat-driven medium-access control for body sensor networks., IEEE Trans Inf Technol Biomed, 14, 44, 10.1109/TITB.2009.2028136 · DOI · Paper page
- Asgari, 2010, A robust approach toward recognizing valid arterial-blood-pressure pulses., IEEE Trans Inf Technol Biomed, 14, 166, 10.1109/TITB.2009.2034845 · DOI
- Ghaffari, 2010, A methodology for prediction of acute hypotensive episodes in ICU via a risk scoring model including analysis of ST-segment variations., Cardiovasc Eng, 10, 12, 10.1007/s10558-009-9088-x · DOI
- Lee, 2011, Open-access MIMIC-II database for intensive care research., Conf Proc IEEE Eng Med Biol Soc, 2011, 8315
- Rocha, 2011, Prediction of acute hypotensive episodes by means of neural network multi-models., Comput Biol Med, 41, 881, 10.1016/j.compbiomed.2011.07.006 · DOI
- Mandelbaum, 2011, Outcome of critically ill patients with acute kidney injury using the Acute Kidney Injury Network criteria., Crit Care Med, 39, 2659, 10.1097/CCM.0b013e3182281f1b · DOI · Paper page
- Lehman, 2012, Risk stratification of ICU patients using topic models inferred from unstructured progress notes., AMIA Annu Symp Proc, 2012, 505
- Warner, 2012, Phenome based analysis as a means for discovering context dependent clinical reference ranges., AMIA Annu Symp Proc, 2012, 1441
- Mikhno, 2012, Prediction of extubation failure for neonates with respiratory distress syndrome using the MIMIC-II clinical database., Conf Proc IEEE Eng Med Biol Soc, 2012, 5094
- Baumgartner, 2012, A data mining approach to reduce the false alarm rate of patient monitors., Conf Proc IEEE Eng Med Biol Soc, 2012, 5935
- Abhyankar, 2012, Lower short- and long-term mortality associated with overweight and obesity in a large cohort study of adult intensive care unit patients., Crit Care, 16, R235, 10.1186/cc11903 · DOI
- Abhyankar, 2012, Standardizing clinical laboratory data for secondary use., J Biomed Inform, 45, 642, 10.1016/j.jbi.2012.04.012 · DOI
- Abhyankar, 2013, A simple method to extract key maternal data from neonatal clinical notes., AMIA Annu Symp Proc, 2013, 2
- Henriksson, 2013, Identifying synonymy between SNOMED clinical terms of varying length using distributional analysis of electronic health records., AMIA Annu Symp Proc, 2013, 600
- Mandelbaum, 2013, Empirical relationships among oliguria, creatinine, mortality, and renal replacement therapy in the critically ill., Intensive Care Med, 39, 414, 10.1007/s00134-012-2767-x · DOI
- Behar, 2013, ECG signal quality during arrhythmia and its application to false alarm reduction., IEEE Trans Biomed Eng, 60, 1660, 10.1109/TBME.2013.2240452 · DOI · Paper page
- Warner, 2013, External phenome analysis enables a rational federated query strategy to detect changing rates of treatment-related complications associated with multiple myeloma., J Am Med Inform Assoc, 20, 696, 10.1136/amiajnl-2012-001355 · DOI
- Warner, 2013, Temporal phenome analysis of a large electronic health record cohort enables identification of hospital-acquired complications., J Am Med Inform Assoc, 20, e281, 10.1136/amiajnl-2013-001861 · DOI
- Dejam, 2014, The effect of age and clinical circumstances on the outcome of red blood cell transfusion in critically ill patients., Crit Care, 18, 487, 10.1186/s13054-014-0487-z · doi:10.1186/s13054-014-0487-z
- Barletta, 2014, Proton pump inhibitors increase the risk for hospital-acquired Clostridium difficile infection in critically ill patients., Crit Care, 18, 714, 10.1186/s13054-014-0714-7 · DOI
- Zhang, 2014, Predictive value of ionized calcium in critically ill patients: An analysis of a large clinical database MIMIC II., PLoS One, 9, e95204, 10.1371/journal.pone.0095204 · DOI
- Zhang, 2014, Urine output on ICU entry is associated with hospital mortality in unselected critically ill patients., J Nephrol, 27, 65, 10.1007/s40620-013-0024-1 · DOI
- Ghassemi, 2014, Leveraging a critical care database: Selective serotonin reuptake inhibitor use prior to ICU admission is associated with increased hospital mortality., Chest, 145, 745, 10.1378/chest.13-1722 · DOI · Paper page
- Velasquez, 2014, Long-term outcomes of minor troponin elevations in the intensive care unit., Anaesth Intensive Care, 42, 356, 10.1177/0310057X1404200313 · DOI
- Zhang, 2014, Predictive value of lactate in unselected critically ill patients: An analysis using fractional polynomials., J Thorac Dis, 6, 995
- Ghassemi, 2014, A data-driven approach to optimized medication dosing: A focus on heparin., Intensive Care Med, 40, 1332, 10.1007/s00134-014-3406-5 · DOI
- Abhyankar, 2014, Combining structured and unstructured data to identify a cohort of ICU patients who received dialysis., J Am Med Inform Assoc, 21, 801, 10.1136/amiajnl-2013-001915 · DOI · Paper page
- Chasin, 2014, Word sense disambiguation in the clinical domain: A comparison of knowledge-rich and knowledge-poor unsupervised methods., J Am Med Inform Assoc, 21, 842, 10.1136/amiajnl-2013-002133 · DOI
- Marafino, 2014, N-gram support vector machines for scalable procedure and diagnosis classification, with applications to clinical free text data from the intensive care unit., J Am Med Inform Assoc, 21, 871, 10.1136/amiajnl-2014-002694 · DOI · Paper page
- Barletta, 2014, Histamine-2-receptor antagonist administration and gastrointestinal bleeding when used for stress ulcer prophylaxis in patients with severe sepsis or septic shock., Ann Pharmacother, 48, 1276, 10.1177/1060028014540513 · DOI
- Bromuri, 2014, Multi-label classification of chronically ill patients with bag of words and supervised dimensionality reduction algorithms., J Biomed Inform, 51, 165, 10.1016/j.jbi.2014.05.010 · DOI
- Salas-Boni, 2014, False ventricular tachycardia alarm suppression in the ICU based on the discrete wavelet transform in the ECG signal., J Electrocardiol, 47, 775, 10.1016/j.jelectrocard.2014.07.016 · DOI
- Zhang, 2015, Antipyretic therapy in critically ill patients with sepsis: An interaction with body temperature., PLoS One, 10, e0121919, 10.1371/journal.pone.0121919 · DOI
- Marafino, 2015, Efficient and sparse feature selection for biomedical text classification via the elastic net: Application to ICU risk stratification from nursing notes., J Biomed Inform, 54, 114, 10.1016/j.jbi.2015.02.003 · DOI
- Rosner, 1983, Percentage points for a generalized ESD many-outlier procedure., Technometrics, 25, 165, 10.1080/00401706.1983.10487848 · DOI · Paper page
- Ester, 1996, A density-based algorithm for discovering clusters in large spatial databases with noise., Kdd, 96, 226
- Moorman, 2011, Mortality reduction by heart rate characteristic monitoring in very low birth weight neonates: A randomized trial., J Pediatr, 159, 900, 10.1016/j.jpeds.2011.06.044 · DOI · Paper page
- Buchman, 2002, Heart rate variability in critical illness and critical care., Curr Opin Crit Care, 8, 311, 10.1097/00075198-200208000-00007 · DOI · Paper page
- Hug, 2007, An analysis of the errors in recorded heart rate and blood pressure in the ICU using a complex set of signal quality metrics., Comput Cardiol, 641
- Buhl, 2013, Big Data., Wirtschaftsinf, 55, 63, 10.1007/s11576-013-0350-x · DOI
Cited by
van de Sande, D., van Genderen, M. E., Huiskens, J., Gommers, D., & van Bommel, J. (2021) · Intensive Care Medicine · doi:10.1007/s00134-021-06446-7 · Cited by 279
van de Sande, D., Van Genderen, M. E., Smit, J. M., Huiskens, J., Visser, J. J., Veen, R. E. R., van Unen, E., BA, O. H., Gommers, D., & Bommel, J. V. (2022) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2021-100495 · Cited by 135
Metadata
- Authors
- Maslove, D. M., Dubin, J. A., Shrivats, A., & Lee, J.
- Affiliations
- Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Ontario, Canada.
- School of Public Health and Health Systems, University of Waterloo, Waterloo, Ontario, Canada.
- Venue
- Critical Care Medicine
- Published
- 2016-11
- Volume
- 44
- Issue
- 11
- Pages
- e1021-e1030
- ISSN
- 0090-3493
- Cited by
- 32
- References
- 6
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Errors, Omissions, and Outliers in Hourly Vital Signs Measurements in Intensive Care. https://app.cochat.ai/papers/10.1097/ccm.0000000000001862
- The paper
- Maslove, D. M., Dubin, J. A., Shrivats, A., & Lee, J. (2016). Errors, Omissions, and Outliers in Hourly Vital Signs Measurements in Intensive Care. Critical Care Medicine. https://doi.org/10.1097/ccm.0000000000001862
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.