Development of a Real-time Force-based Algorithm for Infusion Failure Detection
Open accessAbstract
Background: Continuous subcutaneous insulin infusion (CSII) is a common treatment option for people with diabetes (PWD), but insulin infusion failures pose a significant challenge, leading to hyperglycemia, diabetes burnout, and increased hospitalizations. Current CSII pumps’ occlusion alarm systems are limited in detecting infusion failures; therefore, a more effective detection method is needed. Methods: We conducted five preclinical animal studies to collect data on infusion failures, utilizing both insulin and non-insulin boluses. Data were captured using in-line pressure and flow rate sensors, with additional force data from CSII pumps’ onboard sensors in one study. A novel classifier model was developed using this dataset, aimed at detecting different types of infusion failures through direct utilization of force sensor data. Performance was compared against various occlusion alarm thresholds from commercially available CSII pumps. Results: The testing dataset included 251 boluses. The Bagging classifier model showed the highest performance metrics among the models tested, exhibiting high accuracy (96%), sensitivity (94%), and specificity (98%), with lower false-positive and false-negative rate compared with traditional occlusion alarm pressure thresholds. Conclusions: Our study developed a novel non-threshold classifier that outperforms current occlusion alarm systems in CSII pumps in detecting infusion failures. This advancement has the potential to reduce the risk of hyperglycemia and hospitalizations due to undetected infusion failures, offering a more reliable and effective CSII therapy for PWD. Further studies involving human participants are recommended to validate these findings and assess the classifier’s performance in a real-world setting.
AI snapshot
Written from the paper’s full text.
TL;DR
The paper develops a non-threshold machine-learning classifier using CSII pump onboard force sensor data to detect infusion failures, trained and tested on preclinical swine bolus data. The Bagging classifier outperformed commercial occlusion alarm thresholds in retrospective testing, but human validation is still needed.
Key findings
- In the 251-bolus testing set (169 functional, 82 dysfunctional), the Bagging classifier with 29 features achieved 96% accuracy, 94% sensitivity, 97% specificity, 3% FPR, 6% FNR, and MCC 0.909; all five final models had accuracy ≥90%, but only BAG had MCC ≥0.9.
- Compared with force-converted commercial occlusion thresholds, BAG had fewer false positives and false negatives; for example, the 13.15 PSI/10,885 mN threshold had 60% accuracy, 30% sensitivity, and 74% specificity, while the 35 PSI/21,783 mN threshold had 67% accuracy, 0% sensitivity, and 100% specificity.
- All classifications were observed within 13 to 120 seconds following bolus completion and depended on bolus size and injection speed.
- During testing, LR had the highest PR-AUC (0.979) and ROC-AUC (0.988), followed by SVM, BAG, RF, and GBM.
Methods
- Five preclinical swine studies conducted between November 2020 and August 2023 yielded 866 boluses; 615 (71%) from studies 1–4 were used as the training set with pressure data only, and 251 (29%) from study 5 were used as the testing set with both onboard force and in-line pressure data.
- Boluses were manually labeled “Functional” or “Dysfunctional” by two trained individuals using flow rate, pressure, and force data, with a 70% volume-delivery cutoff and six testing-set label overwrites.
- A third-order polynomial pressure-to-force scaling function was fitted from synchronized study 5 data; models tested included LR, SVM, RF, LDA, GNB, GPC, BAG, and GBM, with feature elimination, hyperparameter tuning, SMOTE, 10-fold 3-repeat stratified k-fold cross-validation, and F2-score optimization.
Limitations
- The paper states reliance on non-human animal data for both training and testing; ideal validation would involve human data and glucose outcomes.
- The paper states that training included ambulatory insulin infusion set data but testing did not, and that extended basal delivery scenarios were not tested; pressure-to-force conversion meant training used processed rather than raw data, potentially introducing model bias.
- The paper states the model identifies individual bolus failures but does not yet provide a comprehensive infusion-site health assessment or actionable replacement recommendations; evident from the text, testing came from a single study with 82 dysfunctional boluses, and no human clinical validation was performed.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
10 of 16 cited sources checked · 3 supported · 0 flagged · accuracy not yet measured
- Premisecitation
Extended wear infusion sets present a wider timeframe for infusion failure.
“given the recent introduction of extended wear infusion sets, 32 which present a wider timeframe for infusion failure”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [32] source
- Premisecitation
Current occlusion alarm systems cannot detect other types of infusion failures such as leakages, dislodged cannulas, and silent occlusions.
“However, these detection thresholds are not standardized across manufacturers and cannot detect other types of infusion failures such as leakages, dislodged cannulas, and “silent” (ie, no alert) occlusions.15,16”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [15] source
The Discussion (page unavailable) says current pumps have no leakage-detection alarms and that most cannula malplacements may go unrecognized for hours. It also says some malplacements are eventually recognized by high-pressure alarms, so an absolute inability to detect them overstates the source. The paper does not address “silent” occlusions. (LLM, conf=0.86)
Experimental — accuracy not yet measured - cites [16] source
The Mechanical problems and Hyperglycemia sections identify leaks, displaced catheters, and blocked infusion sets as causes of unexplained high blood glucose; Mechanical problems also says leaks can be difficult to detect. Neither section says that occlusion alarms fail to detect these problems or silent occlusions, and the paper does not establish that alarm thresholds vary across manufacturers. No page numbers are available in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measured
- Premisecitation
Occlusion detection thresholds are not standardized across manufacturers.
“However, these detection thresholds are not standardized across manufacturers and cannot detect other types of infusion failures such as leakages, dislodged cannulas, and “silent” (ie, no alert) occlusions.15,16”
Quote in the paperSubstantial claim2 of 2 sources checked- cites [15] source
- cites [16] source
The Mechanical problems section discusses clogged infusion sets, leakage, and displaced sets, but does not address occlusion-detection thresholds or whether manufacturers standardize them. Searches for threshold standardization and occlusion-alarm pressure found no relevant passage in our copy; page numbers were not available. (LLM, conf=0.76)
Experimental — accuracy not yet measured
- Premisecitation
Testing of retrofitted pressure sensors in CSII pumps suggests that the incorporation of more precise fluid monitoring could improve infusion failure detection.
“Alternatively, the testing of retrofitted pressure sensors in CSII pumps suggests that the incorporation of more precise fluid monitoring could improve the infusion failure detection.15”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [15] source
- Premisecitation
Occlusion alarms can take time to trigger depending on the pump's infusion rate, with delays up to 40 hours post-failure.
“Furthermore, occlusion alarms can take time to trigger depending on the pump's infusion rate, with delays up to 40 hours post-failure.17”
Quote in the paperSubstantial claim1 of 1 sources checked- cites [17] source
The Results report that occlusion detection took substantially longer at 0.1 U/h than at 1.0 U/h, supporting the infusion-rate dependence. But “up to 40 hours” understates the observed delays: the longest median was 41 hours 21 minutes, and an individual alarm took 65 hours 40 minutes. (LLM, conf=0.87)
Experimental — accuracy not yet measured
- Findingown data
LR had the best ROC AUC (0.988) during testing.
“B-2: LR had the best ROC AUC (0.988) during testing.”
Quote in the paperSubstantial claim - Findingown data
A substantial portion of the training data originated from insulin infusion sets used in an ambulatory setting with insulin over more extended periods up to 2.02 (± 0.84) days.
“A substantial portion of the training data originated from insulin infusion sets used in an ambulatory setting with insulin over more extended periods up to 2.02 (± 0.84) days, which we consider a strength.”
Quote in the paperSubstantial claim - Findingown data
Best performance was achieved at 98 features for RF (F2-score 0.94).
“Best performance was achieved at 33 (0.93) for LR, 27 (0.92) for SVM, 98 (0.94) for RF, 44 (0.934) for BAG, and 29 (0.932) for GBM identified with the marker.”
Quote in the paperSubstantial claim - Findingown data
Best performance was achieved at 29 features for GBM (F2-score 0.932).
“Best performance was achieved at 33 (0.93) for LR, 27 (0.92) for SVM, 98 (0.94) for RF, 44 (0.934) for BAG, and 29 (0.932) for GBM identified with the marker.”
Quote in the paperSubstantial claim - Findingown data
Best performance was achieved at 27 features for SVM (F2-score 0.92).
“Best performance was achieved at 33 (0.93) for LR, 27 (0.92) for SVM, 98 (0.94) for RF, 44 (0.934) for BAG, and 29 (0.932) for GBM identified with the marker.”
Quote in the paperNot a substantial claim - Findingown data
ROC AUC shows a 0.006 difference between best (GBM) and worst model (RF).
“ROC AUC also shows a 0.006 difference between best (GBM) and worst model (RF).”
Quote in the paperNot a substantial claim - Findingown data
LR had the best PR AUC (0.979) during testing.
“B-1: LR has the best PR AUC (0.979) during testing”
Quote in the paperSubstantial claim - Findingown data
Best performance was achieved at 44 features for BAG (F2-score 0.934).
“Best performance was achieved at 33 (0.93) for LR, 27 (0.92) for SVM, 98 (0.94) for RF, 44 (0.934) for BAG, and 29 (0.932) for GBM identified with the marker.”
Quote in the paperNot a substantial claim - Findingown data
Data from 866 boluses were obtained from all studies using pressure and force sensors.
“Data from 866 boluses were obtained from all studies using pressure and force sensors.”
Quote in the paperSubstantial claim - Findingown data
Using a 99.5% confidence interval (P < .005), 104 features were identified to be statistically significant.
“Using a 99.5% confidence interval (P < .005),104 features were identified to be statistically significant.”
Quote in the paperNot a substantial claim - Findingown data
GBM had the best PR AUC (0.983) during tuning.
“A-1: GBM had the best PR AUC (0.983) during tuning”
Quote in the paperSubstantial claim - Findingcitation
Five preclinical studies were conducted using nine healthy swine without diabetes and three with streptozotocin-induced diabetes.
“Five preclinical studies were conducted (Table 1), using nine healthy swine without diabetes and three with streptozotocin-induced diabetes.22”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [22] source
- Findingown data
LDA, GNB, and GPC are not clustered with the other 5 models (Recall > 0.86, Precision > 0.80).
“Here, we see that LDA, GNB, and GPC are not clustered with the other 5 models (Recall > 0.86, Precision > 0.80) and can be eliminated going forward.”
Quote in the paperSubstantial claim - Findingown data
Best performance was achieved at 33 features for LR (F2-score 0.93).
“Best performance was achieved at 33 (0.93) for LR, 27 (0.92) for SVM, 98 (0.94) for RF, 44 (0.934) for BAG, and 29 (0.932) for GBM identified with the marker.”
Quote in the paperNot a substantial claim - Findingown data
No infusions met the 35 PSI threshold due to the study pump occlusion threshold being set at 20,000 mN for study 5.
“No infusions met this threshold due to the study pump occlusionthreshold being set at 20 000 mN for study 5.”
Quote in the paperSubstantial claim - Findingnone given
The developed model for detecting infusion failures outperforms traditional onboard occlusion pump alarms.
“We have developed an innovative model for detecting infusion failures that outperforms traditional onboard occlusion pump alarms.”
Quote in the paperNot a substantial claim- Rests on an assumption: The testing dataset is representative of the broader population of insulin pump users.
- Findingown data
GBM had the best ROC AUC (0.991) during tuning.
“A-2: BGM had best ROC AUC of (0.991) during tuning.”
Quote in the paperNot a substantial claim - Findingown data
The BAG model with 29 features achieved 97% specificity on the testing dataset.
“| BAG | 29 | 77 | 164 | 5 | 5 | 96% | 94% | 97% | 3% | 6% | 0.909 |”
Quote in the paperSubstantial claim - Findingown data
The BAG model with 29 features achieved 96% accuracy on the testing dataset.
“| BAG | 29 | 77 | 164 | 5 | 5 | 96% | 94% | 97% | 3% | 6% | 0.909 |”
Quote in the paperSubstantial claim - Findingown data
The BAG model with 29 features achieved 94% sensitivity on the testing dataset.
“| BAG | 29 | 77 | 164 | 5 | 5 | 96% | 94% | 97% | 3% | 6% | 0.909 |”
Quote in the paperSubstantial claim - Premisenone given
The efficacy of CGM-based infusion failure detection systems is reliant on the accuracy and connectivity of external CGM sensors.
“Yet, the efficacy of these systems is reliant on the accuracy and connectivity of external CGM sensors.”
Quote in the paperNot a substantial claim - Interpretationown data
The dynamic models trained on animal data effectively detect infusion set failures using data from an insulin pump's onboard force sensor.
“This study demonstrates that the dynamic models trained on our animal data effectively detects infusion set failures using data from an insulin pump's onboard force sensor.”
Quote in the paperSubstantial claim- Rests on an assumption: The manually assigned binary labels based on fluid dynamics data accurately represent functional versus dysfunctional boluses.
- Study design and conductown data
A limitation of this study is the reliance on non-human animal data used for both training and testing the algorithm.
“A limitation of this study is the reliance on non-human animal data used for both training and testing the algorithm.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The algorithm has the ability to identify site failures immediately, without the need for glucose data or hyperglycemia to trigger an alert.
“Its primary advantage lies in its ability to identify site failures immediately, without the need for glucose data or hyperglycemia to trigger an alert.”
Quote in the paperSubstantial claim - Study design and conductown data
Another notable limitation was the need to translate pressure measurements into force values in the preclinical training dataset.
“Another notable limitation was the need to translate pressure measurements into force values in our preclinical training dataset.”
Quote in the paperNot a substantial claim - Study design and conductreasoning
The lack of inclusion of ambulatory insulin data in the testing set is a limitation.
“However, the lack of inclusion of these data in the testing set is a limitation that opens avenues for future research.”
Quote in the paperNot a substantial claim - Premisecitation
The majority of commercially available CSII pumps have occlusion-only alarm systems that rely on an onboard sensor and algorithm to alert a user when the pressure in the infusion line exceeds a programmed threshold.
“The majority of commercially available pumps have occlusion-only alarm systems, which rely on an onboard sensor and algorithm to alert a user when the pressure in the infusion line exceeds a programmed threshold.[10][11][12][13][14]”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [10][11][12][13][14]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premiseown data
An increase in false negatives leaves users to rely on hyperglycemia to detect failures.
“On the other hand, an increase in FNs leaves users to rely on hyperglycemia to detect failures.”
Quote in the paperSubstantial claim - Interpretationown data
The performance metrics for the 35 PSI threshold are a good estimate of the performance metrics for not having an occlusion system at all.
“This is a good estimate of the performance metrics for not having an occlusion system at all.”
Quote in the paperSubstantial claim - Interpretationreasoning
The improvements over standard thresholds are primarily attributable to reductions in both false positives and false negatives.
“These improvements are primarily attributable to the reductions in both FPs and FNs.”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 17
- May combine two claims. 25
- The quote only partly states this claim. 2
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductcitationDuring training, model performance was evaluated using 10-fold, 3-repeats stratified k-fold cross-validation and F2-score.
“During training, model performance was evaluated using 10-fold, 3-repeats stratified k-fold cross-validation and F2-score (ie, instead of F1-score) as it penalizes false negatives.24,25”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [24] source
The Methods section (Model evaluation metrics; page unavailable) reports stratified tenfold cross-validation on the training set. It does not report three repeats or use of the F2-score; instead, it names accuracy, AUC, recall, and precision as its evaluation metrics. (LLM, conf=0.90)
Experimental — accuracy not yet measured - cites [25] source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationInfusion failures are the leading cause of hospitalizations for diabetes-related ketoacidosis among people with diabetes who use CSII therapy, in some cases resulting in fatal outcomes.
“Notably, infusion failures are the leading cause of hospitalizations for diabetes-related ketoacidosis (DKA) hospitalization among PWD who use CSII therapy, in some cases resulting in fatal outcomes.8,9”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites [8] source
The Results (Etiology of DKA; page not provided) report that pump/tubing defects were associated with 55% of DKA cases among the study’s CSII users, making them the leading cause in that hospitalized group. The Discussion describes DKA as potentially fatal but does not link any fatal outcome specifically to an infusion failure. The claim also generalizes the finding beyond the adults with type 1 diabetes studied. (LLM, conf=0.88)
Experimental — accuracy not yet measured - cites [9] source
May combine two claims.
Findingown dataDuring tuning, all models showcased strong PR-AUC and ROC-AUC which speaks to the model's robustness and ability to generalize past the training set.
“During tuning, all models showcased strong PR-AUC and ROC-AUC which speaks to the model's robustness and ability to generalize past the training set (Figure 5).24”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites [24] source
The Results sections on model performance and feature selection report ROC-AUC values and use cross-validation and external validation to assess robustness and generalization (page not provided). They do not report PR-AUC, and the decision tree performed poorly on external validation, so the claim that all models had strong results on both measures is not supported. (LLM, conf=0.88)
Experimental — accuracy not yet measured
May combine two claims.
Findingnone givenThe model provides immediate detection of failures without requiring glucose or other external input.
“This model provides immediate detection of failures without requiring glucose or other external input”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataPR AUC shows minimal difference between models, 0.013 difference between best (GBM) and worst model (RF).
“PR AUC highlights minimal difference between models, 0.013 difference between best (GBM) and worst model (RF).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAll models showed accuracies ≥0.9, but only BAG had an MCC ≥0.9.
“Overall, all models showed accuracies ≥0.9, but only BAG had an MCC ≥0.9.”
Quote in the paperSubstantial claim- Rests on an assumption: The manually assigned binary labels based on fluid dynamics data accurately represent functional versus dysfunctional boluses.
May combine two claims.
Findingown dataA total of 251 (29%) boluses (study 5) were used for the testing set and had data from both the study pump's onboard force sensor and the in-line pressure sensor.
“A total of 251 (29%) boluses (study 5) were used for the testing set and had data from both the study pump's onboard force sensor and the in-line pressure sensor.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataBAG (29 features) had the highest accuracy (96%), MCC (0.909), sensitivity (94%), specificity (97%), FPR (3.0%), and FNR (6.0%) among the five trained models.
“Of the five, BAG (29 features) had the highest accuracy (96%), MCC (0.909), sensitivity (94%), specificity (97%), FPR (3.0%), and FNR (6.0%).”
Quote in the paperSubstantial claim- Rests on an assumption: The manually assigned binary labels based on fluid dynamics data accurately represent functional versus dysfunctional boluses.
May combine two claims.
Findingown dataLogistic regression, SVM, RF, BAG, and GBM showed similar performance with Recall > 0.86, Precision > 0.80, and F2-score > 0.86.
“Logistic regression, SVM, RF, BAG, and GBM showed similar performance with Recall > 0.86, Precision > 0.80, and F2-score > 0.86 so LDA, GNB, and GPC were not considered for further testing (Figure 3, Table 2).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe other 5 models have Recall > 0.86 and Precision > 0.80.
“Here, we see that LDA, GNB, and GPC are not clustered with the other 5 models (Recall > 0.86, Precision > 0.80) and can be eliminated going forward.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataOf the 251 values included in the testing dataset, 169 (67.3%) were labeled as Functional and 82 (32.7%) were Dysfunctional.
“Of the 251 values included in the testing dataset, 169 (67.3%) were labeled as “Functional” and 82 (32.7%) were “Dysfunctional.””
Quote in the paperNot a substantial claim- Rests on an assumption: The manually assigned binary labels based on fluid dynamics data accurately represent functional versus dysfunctional boluses.
May combine two claims.
Findingown dataOf the models, BAG yielded the highest performance; however, even LR (the least complex model) performed nearly as well.
“Of the models, BAG yielded the highest performance; however, even LR (the least complex model) performed nearly as well.”
Quote in the paperNot a substantial claim- Rests on an assumption: The manually assigned binary labels based on fluid dynamics data accurately represent functional versus dysfunctional boluses.
May combine two claims.
Findingnone givenAll classifications were observed within 13 to 120 seconds following bolus completion and were dependent on bolus size and injection speed.
“All classifications were observed within 13 to 120 seconds following bolus completion and were dependent on bolus size and injection speed.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAll models showcased strong PR-AUC and ROC-AUC scores during testing with LR having the highest in both instances with 0.979 and 0.988, respectively, followed by SVM, BAG, RF, and GBM.
“All models showcased strong PR-AUC and ROC-AUC scores during testing with LR having the highest in both instances with 0.979 and 0.988, respectively, followed by SVM, BAG, RF, and GBM (Figure 5).”
Quote in the paperNot a substantial claim- Rests on an assumption: The manually assigned binary labels based on fluid dynamics data accurately represent functional versus dysfunctional boluses.
May combine two claims.
Findingown dataA total of 615 (71%) boluses (studies 1-4) were used as the training set and only contained pressure data.
“A total of 615 (71%) boluses (studies 1-4) were used as the training set and only contained pressure data.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe identified optimal feature subsets included 33, 27, 98, 44, and 29 features for LR, SVM, RF, BAG, and GBM, respectively, achieving F2-scores of 0.930, 0.923, 0.940, 0.934, and 0.932.
“The identified optimal subsets included 33, 27, 98, 44, and 29 features for LR, SVM, RF, BAG, and GBM, respectively, achieving F2-scores of 0.930, 0.923, 0.940, 0.934, and 0.932 (Figure 4).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataOverwriting of %VD occurred in six instances in the testing set.
“Overwriting of %VD occurred in six instances in the testing set.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingown dataThe output of the pressure-to-force preprocessing step was a third-order polynomial function used to transform the training set's pressure data to force.
“The output of this preprocessing step was a thirdorder polynomial function that we could use to transform the training set's pressure data to force (Supplement 3).”
Quote in the paperSubstantial claimMay combine two claims. The quote only partly states this claim.
Findingown dataThe BAG model achieved higher accuracy (96%) than the force-based occlusion thresholds (60-68%) on the testing dataset.
“| BAG | 29 | 77 | 164 | 5 | 5 | 96% | 94% | 97% | 3% | 6% | 0.909 |”
- Also stated in: § Conclusion · plain
“| 30 d | 19 084 | 4 | 167 | 2 | 78 | 68% | 5% | 99% | 1% | 95% | 0.113 |”
Quote in the paperSubstantial claim- Rests on an assumption: The testing dataset is representative of the broader population of insulin pump users.
- Also stated in: § Conclusion · plain
The quote only partly states this claim.
Findingown dataThe BAG model achieved higher sensitivity (94%) than the force-based occlusion thresholds (0-30%) on the testing dataset.
“| BAG | 29 | 77 | 164 | 5 | 5 | 96% | 94% | 97% | 3% | 6% | 0.909 |”
- Also stated in: § Conclusion · plain
“| 13.15 a | 10 885 | 25 | 125 | 44 | 57 | 60% | 30% | 74% | 26% | 70% | 0.047 |”
Quote in the paperSubstantial claim- Rests on an assumption: The testing dataset is representative of the broader population of insulin pump users.
- Also stated in: § Conclusion · plain
Classified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenAll study boluses were analyzed simultaneously by two trained individuals and manually assigned a binary label: Functional or Dysfunctional.
“All study boluses were analyzed simultaneously by two trained individuals and manually assigned a binary label: “Functional” or “Dysfunctional.””
Quote in the paperSubstantial claim- Rests on an assumption: The manually assigned binary labels based on fluid dynamics data accurately represent functional versus dysfunctional boluses.
Probably not a substantial claim: may not pass S4 (based).
InterpretationreasoningThe model is a robust tool that could be applied across a variety of system configurations and subcutaneous infusion set types.
“making it a robust tool that could be applied across a variety of system configurations and subcutaneous infusion set types.”
Quote in the paperNot a substantial claim- Rests on an assumption: The testing dataset is representative of the broader population of insulin pump users.
May combine two claims.
Premisenone givenCGM-based models require an elevation of glucose levels over time, which may produce false alarms if hyperglycemia develops that is not related to an infusion failure.
“These models also require an elevation of glucose levels over time, which may produce false alarms if hyperglycemia develops that is not related to an infusion failure.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationAlgorithms designed to detect infusion failure using continuous glucose monitoring and historical insulin infusion data have been developed but not implemented commercially.
“Algorithms designed to detect infusion failure using continuous glucose monitoring (CGM) and historical insulin infusion data have been developed but not implemented commercially.[19][20][21]”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [19][20][21]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
InterpretationreasoningThis potential universal compatibility could allow broad adoption, offering all users, regardless of their specific devices, the opportunity to enhance the safety and efficacy of their insulin therapy.
“This potential universal compatibility could allow board adoption, offering all users, regardless of their specific devices, the opportunity to enhance the safety and efficacy of their insulin therapy.”
Quote in the paperNot a substantial claim- Rests on an assumption: Onboard force sensors across different CSII pump systems and infusion set types provide comparable data suitable for the algorithm.
Probably not a substantial claim: may not pass S4 (based).
Premisenone givenStandard threshold methods are limited by an inverse trade-off between increasing false positive rate or false negative rate as the threshold is increased or decreased, respectively.
“Standard threshold methods are limited by an inverse trade-off between either increasing FPR or FNR as the threshold is increased or decreased, respectively.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenEach study received prior approval from either the Medical Education & Research Institute or University of Tennessee Health Science Center Institutional Animal Care and Use Committee.
“Each study received prior approval from either the Medical Education & Research Institute (MERI) or University of Tennessee Health Science Center (UTHSC) Institutional Animal Care and Use Committee (Memphis, Tennessee).”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningBecause the algorithm operates based on data from onboard force sensors, its implementation could be possible across a variety of systems and infusion set types.
“In addition, because the algorithm operates based on data from onboard force sensors, its implementation could be possible across a variety of systems and infusion set types.”
Quote in the paperSubstantial claim- Rests on an assumption: Onboard force sensors across different CSII pump systems and infusion set types provide comparable data suitable for the algorithm.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataThe testing included small-volume boluses, but did not test more extended basal delivery scenarios.
“In addition, our testing included smallvolume boluses, but did not test more extended basal delivery scenarios (this type of data was collected during study 4 and will be examined in later work).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenSite leakage was not separately identified or quantified.
“Site leakage was not separately identified or quantified.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoningThe conversion of pressure to force means training itself was conducted on processed rather than raw data, potentially introducing model bias.
“Despite effective dataset training, as evidenced by a strong MCC, the conversion means training itself was conducted on processed rather than raw data, potentially introducing model bias.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningThe ideal validation scenario would involve testing the classifier on human data.
“While swine models offer a reasonable approximation of human tissue and the use of imaging data allows for confirmation of bolus infusion, the ideal validation scenario would involve testing the classifier on human data.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
InterpretationreasoningThis capability could empower individuals to address issues with their infusion sites as soon as they occur, thereby reducing the risk of hyperglycemia and preventing more severe complications, such as DKA.
“This capability could empower individuals to address issues with their infusion sites as soon as they occur, thereby reducing the risk of hyperglycemia and preventing more severe complications, such as DKA.”
Quote in the paperNot a substantial claim- Rests on an assumption: Swine models offer a reasonable approximation of human tissue.
May combine two claims.
Interpretationown dataCompared to standard commercial pump occlusion detection thresholds, the models achieve higher accuracy, sensitivity, specificity, and MCC with fewer false positives and false negatives.
“Compared to standard commercial pump occlusion detection thresholds, we achieve higher accuracy, sensitivity, specificity, and MCC with fewer FPs and FNs.”
- Also stated in: § Abstract · plain
“Our study developed a novel non-threshold classifier that outperforms current occlusion alarm systems in CSII pumps in detecting infusion failures.”
Quote in the paperSubstantial claim- Rests on an assumption: The manually assigned binary labels based on fluid dynamics data accurately represent functional versus dysfunctional boluses.
- Also stated in: § Abstract · plain
Classified as probably not a substantial claim.
InterpretationreasoningExtended wear infusion sets are prime candidates for active monitoring.
“making them prime candidates for the benefits of an active monitoring system.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationInsulin infusion failures in CSII are associated with reduced insulin delivery, prolonged hyperglycemia, and increased likelihoods of diabetes burnout and pump discontinuation.
“Infusion failures are associated with reduced insulin delivery, prolonged hyperglycemia, and increased likelihoods of diabetes burnout and pump discontinuation.[3][4][5]”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [3][4][5]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningThis advancement has the potential to reduce the risk of hyperglycemia and hospitalizations due to undetected infusion failures, offering a more reliable and effective CSII therapy for people with diabetes.
“This advancement has the potential to reduce the risk of hyperglycemia and hospitalizations due to undetected infusion failures, offering a more reliable and effective CSII therapy for PWD.”
Quote in the paperNot a substantial claim- Rests on an assumption: Swine models offer a reasonable approximation of human tissue.
The paper relies on this without support.
Swine models offer a reasonable approximation of human tissue.
“While swine models offer a reasonable approximation of human tissue and the use of imaging data allows for confirmation of bolus infusion, the ideal validation scenario would involve testing the classifier on human data.”
The paper relies on this without support.
The manually assigned binary labels based on fluid dynamics data accurately represent functional versus dysfunctional boluses.
“All study boluses were analyzed simultaneously by two trained individuals and manually assigned a binary label: “Functional” or “Dysfunctional.””
The paper relies on this without support.
Onboard force sensors across different CSII pump systems and infusion set types provide comparable data suitable for the algorithm.
“In addition, because the algorithm operates based on data from onboard force sensors, its implementation could be possible across a variety of systems and infusion set types.”
The paper relies on this without support.
The testing dataset is representative of the broader population of insulin pump users.
“Performance of the Models and Occlusion Alarm Thresholds on the Testing Dataset (Total N = 251, 169 Functional, 82 Dysfunctional).”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
BAG does not have fewer false positives or higher specificity than every threshold
MajorThe discussion claims higher specificity and fewer false positives than commercial thresholds. But Table 3 gives BAG 97% specificity and 5 false positives, versus 100% specificity and 0 false positives at the 35 PSI threshold.
“Compared to standard commercial pump occlusion detection thresholds, we achieve higher accuracy, sensitivity, specificity, and MCC with fewer FPs and FNs.”
“| 35 e | 21 783 | 0 | 169 | 0 | 82 | 67% | 0% | 100% | 0% | 100% | 0.000 |”
Abstract overstates BAG specificity
MinorThe abstract reports 98% specificity for BAG, whereas the Results report 97%. Table 3 also reports 97% (164 true negatives among 169 functional boluses).
“The Bagging classifier model showed the highest performance metrics among the models tested, exhibiting high accuracy (96%), sensitivity (94%), and specificity (98%),”
“Of the five, BAG (29 features) had the highest accuracy (96%), MCC (0.909), sensitivity (94%), specificity (97%), FPR (3.0%), and FNR (6.0%).”
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
3D Slicer as an image computing platform for the Quantitative Imaging Network
Fedorov, A., Beichel, R., Kalpathy–Cramer, J., Finet, J., Fillion‐Robin, J., Pujol, S., Bauer, C., Jennings, D. L., Fennessy, F., Milan Sonka, A., et al. (2012) · Magnetic Resonance Imaging · doi:10.1016/j.mri.2012.05.001 · Cited by 9,095 · see citing works
Chicco, D., & Jurman, G. (2023) · BioData Mining · doi:10.1186/s13040-023-00322-4 · Cited by 667 · see citing works
Kumar, V., Lalotra, G. S., Sasikala, P., Rajput, D. S., Kaluri, R., Lakshmanna, K., Shorfuzzaman, M., Alsufyani, A., & Uddin, M. (2022) · Healthcare · doi:10.3390/healthcare10071293 · Cited by 142 · see citing works
Berget, C., Messer, L. H., & Forlenza, G. P. (2019) · Diabetes Spectrum · doi:10.2337/ds18-0091 · Cited by 136 · see citing works
A Review on Imbalanced Data Classification Techniques
Basha, S. J., Madala, S. R., Vivek, K., Kumar, E. S., & Ammannamma, T. (2022) · 2022 International Conference on Advanced Computing Technologies and Applications (ICACTA) · doi:10.1109/icacta54488.2022.9753392 · Cited by 67 · see citing works
Insulin Infusion Set Use: European Perspectives and Recommendations
Deiss, D., Adolfsson, P., Alkemade-van Zomeren, M., Bolli, G. B., Charpentier, G., Cobelli, C., Danne, T., Girelli, A., Mueller, H., Verderese, C. A., et al. (2016) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2016.07281.sf · Cited by 62 · see citing works
Graf, R., Zeldovich, M., & Friedrich, S. (2022) · Biometrical Journal · doi:10.1002/bimj.202200098 · Cited by 59 · see citing works
Early Detection of Infusion Set Failure During Insulin Pump Therapy in Type 1 Diabetes
Cescon, M., DeSalvo, D. J., Ly, T. T., Maahs, D. M., Messer, L. H., Buckingham, B. A., Doyle, F. J., & Dassau, E. (2016) · Journal of Diabetes Science and Technology · doi:10.1177/1932296816663962 · Cited by 51 · cited 1× · see citing works
“3,18 19-21 Yet, the efficacy of these systems is reliant on the accuracy and connectivity of external CGM sensors.”
Zisser, H. (2008) · Diabetes Care · doi:10.2337/dc07-1757 · Cited by 44 · see citing works
Flores, M., Amir, M., Ahmed, R., Alashi, S., Li, M., Wang, X., Lansang, M. C., & Al-Jaghbeer, M. J. (2020) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2020-001329 · Cited by 43 · see citing works
Insulin Pump Occlusions: For Patients Who Have Been Around the (Infusion) Block
Klonoff, D. C., Freckmann, G., & Heinemann, L. (2017) · Journal of Diabetes Science and Technology · doi:10.1177/1932296817700545 · Cited by 41 · see citing works
Davis, G. M., Hughes, M. S., Brown, S. A., Sibayan, J., Perez-Guzman, M. C., Stumpf, M., Thompson, Z., Basina, M., Patel, R. M., Hester, J., et al. (2023) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2023.0304 · Cited by 41 · see citing works
Self-citationEffects of streptozotocin-induced diabetes in domestic pigs with focus on the amino acid metabolism
Jensen-Waern, M., Andersson, M., Kruse, R., Nilsson, B., Larsson, R., Korsgren, O., & Essén-Gustavsson, B. (2009) · Laboratory Animals · doi:10.1258/la.2008.008069 · Cited by 39 · see citing works
Kanapka, L. G., Lum, J. W., & Beck, R. W. (2022) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2021.0548 · Cited by 35 · see citing works
Continuous subcutaneous insulin infusion: practical issues
Saboo, B., & Talaviya, P. (2012) · Indian Journal of Endocrinology and Metabolism · doi:10.4103/2230-8210.104055 · Cited by 32 · see citing works
Real-Time Detection of Infusion Site Failures in a Closed-Loop Artificial Pancreas
Howsmon, D. P., Baysal, N., Buckingham, B. A., Forlenza, G. P., Ly, T. T., Maahs, D. M., Marcal, T., Towers, L., Mauritzen, E., Deshpande, S., et al. (2018) · Journal of Diabetes Science and Technology · doi:10.1177/1932296818755173 · Cited by 32 · see citing works
Machine learning approaches for biomarker discovery to predict large-artery atherosclerosis
Sun, T.-H., Wang, C.-C., Wu, Y.-L., Hsu, K.-C., & Lee, T.-H. (2023) · Scientific Reports · doi:10.1038/s41598-023-42338-0 · Cited by 32 · see citing works
Regittnig, W., Tschaikner, M., Tuca, A., Simic, A., Feiel, J., Schaller‐Ammann, R., Licht, A. H., Jungklaus, M., & Pieber, T. R. (2022) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.14594 · Cited by 27 · see citing works
Tissue Response to Subcutaneous Infusion Catheter
Zhang, E., & Cao, Z. (2019) · Journal of Diabetes Science and Technology · doi:10.1177/1932296819837972 · Cited by 24 · see citing works
Longevity of the novel <scp>ConvaTec</scp> infusion set with Lantern technology
Lal, R. A., Hsu, L., Zhang, J., Schøndorff, P. K., Heschel, M., & Buckingham, B. (2021) · Diabetes, Obesity and Metabolism · doi:10.1111/dom.14395 · Cited by 22 · see citing works
Self-citationOcclusion Detection Time in Insulin Pumps at Two Different Basal Rates
Freckmann, G., Kamecke, U., Waldenmaier, D., Haug, C., & Ziegler, R. (2018) · Journal of Diabetes Science and Technology · doi:10.1177/1932296817750404 · Cited by 18 · see citing works
Hughes, M. S., Douvas, J. L., Layfield-Bryan, M., Blanco, L. E., Gray, J. C., Zapotoczny, G., Espinoza, J., Wilcox, J. H., & Lal, R. A. (2023) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2023.0005 · Cited by 16 · see citing works
Self-citationFatalities Due to Failure of Continuous Subcutaneous Insulin Infusion Devices: A Report of Six Cases
Ziegler, A., Williams, T., Yarid, N., Schultz, D. L., & Bundock, E. A. (2019) · Journal of Forensic Sciences · doi:10.1111/1556-4029.13841 · Cited by 14 · see citing works
Meneghetti, L., Dassau, E., Doyle, F. J., & Favero, S. D. (2021) · Journal of Diabetes Science and Technology · doi:10.1177/1932296821997854 · Cited by 11 · see citing works
Insulin infusion sets associated with adverse events: strategies for improved diabetes education
Neves, A. L. D., Martins, L. E. G., Gabbay, M. A. L., Pascali, P. M., de Oliveira, T., Martinazzo, A., Dib, S. A., Casarini, D. E., Arêdes, S. V., Tenório, F. S., et al. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1275394 · Cited by 6 · see citing works
Liu, Y., Heiselman, C. J., Quirk, J. G., et al. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414754 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
The Future of Automated Insulin Delivery Systems
Hughes, M. S., & Levy, C. J. (2025) · Endocrine Practice · doi:10.1016/j.eprac.2025.05.752 · Cited by 18 · see citing works
Self-citationHughes, M. S., Pasquel, F. J., Davis, G. M., Lal, R., Buckingham, B. A., Boughton, C. K., Brown, S. A., & Bally, L. (2024) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2024.0343 · Cited by 13 · see citing works
“74 To manage glucose excursions, insulin delivery issues, and other pump malfunctions effectively, standardized alarm settings and response protocols should be developed.”
Self-citation
- Authors
- Blanco, L. E., Wilcox, J. H., Hughes, M. S., & Lal, R. A.
- Affiliations
- Diatech Diabetes, Inc., Memphis, TN, USA
- Stanford University (US)
- Venue
- Journal of Diabetes Science and Technology
- Publisher
- SAGE Publications
- Published
- 2024-04-23
- Volume
- 18
- Issue
- 6
- Pages
- 1313-1323
- ISSN
- 1932-2968 (ISSN-L)
- PMC
- PMC11535327
- OpenAlex
- W4395078941
- Semantic Scholar
- b831013f71cd21039dd4064ec2c78f21337100be
- Cited by
- 2
- References
- 27
- Citations by year
- 2025: 1 · 2024: 1
- Type
- conference abstract
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Diabetes Science and Technology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (submittedVersion)
- Funders
- The Consortium for Technology & Innovation in Pediatrics (P50FD007963)
- National Institute of Diabetes and Digestive and Kidney Diseases (1K23DK122017, 1K23DK138267, 1R43DK130036-01, P30 DK116074)
- Ypsomed AG
- JDRF (3-IND-2023-1444-I-X)
- Topics
- Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Diabetes Management and Research
- Physical Sciences › Engineering › Biomedical Engineering › Intravenous Infusion Technology and Safety
- Health Sciences › Medicine › Surgery › Healthcare Technology and Patient Monitoring
- Keywords
- insulin pump therapy, occlusion detection, insulin pump, bagging classifier, diabetes mellitus, hyperglycemia prevention, real-time classification
- MeSH terms
- Algorithms, Animals, Equipment Failure, Humans, Hypoglycemic Agents, Insulin, Insulin Infusion Systems, Male, Clinical Alarms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-13.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-13.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review