Real-Time Identification of Look-Alike Medical Vials Using Mixed Reality-Enabled Deep Learning
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
The accurate identification of look-alike medical vials is essential for patient safety, particularly when similar vials contain different substances, volumes, or concentrations. Traditional methods, such as manual selection or barcode-based identification, are prone to human error or face reliability issues under varying lighting conditions. This study addresses these challenges by introducing a real-time deep learning-based vial identification system, leveraging a Lightweight YOLOv4 model optimized for edge devices. The system is integrated into a Mixed Reality (MR) environment, enabling the real-time detection and annotation of vials with immediate operator feedback. Compared to standard barcode-based methods and the baseline YOLOv4-Tiny model, the proposed approach improves identification accuracy while maintaining low computational overhead. The experimental evaluations demonstrate a mean average precision (mAP) of 98.76 percent, with an inference speed of 68 milliseconds per frame on HoloLens 2, achieving real-time performance. The results highlight the model’s robustness in diverse lighting conditions and its ability to mitigate misclassifications of visually similar vials. By combining deep learning with MR, this system offers a more reliable and efficient alternative for pharmaceutical and medical applications, paving the way for AI-driven MR-assisted workflows in critical healthcare environments.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Specifically, previous works have not systematically identified the batch size regimes at which GPU memory pressure leads to inference engine failures or runtime instability.”
From Benchmarking YOLOv8 Variants for Object Detection Efficiency on Jetson Orin NX for Edge Computing Applications (2026) · cites this paper as [52]
Citations
Cites
Lai, Z.-H., Tao, W., Leu, M. C., & Yin, Z. (2020) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2020.02.010 · Cited by 214 · cited 1×
“Lai et al. [12] used Faster R-CNN to track assembly tools in real time, allowing for operators to quickly locate the required tools, reducing errors and improving workflow efficiency.”
Strategies for reducing medication errors in the emergency department
Weant, K. A., Bailey, A., & Baker, S. (2014) · Open Access Emergency Medicine · doi:10.2147/oaem.s64174 · Cited by 104 · cited 1×
“The rates of drug errors in emergency departments (ED) vary from 4% to 14%, escalating to 39% in pediatric cases [1].”
A Novel Lightweight Real-Time Traffic Sign Detection Integration Framework Based on YOLOv4
Gu, Y., & Si, B. (2022) · Entropy · doi:10.3390/e24040487 · Cited by 60 · cited 1×
“Furthermore, Gu et al. [15] implemented YOLOv4 for real-time traffic sign detection, demonstrating that optimized deep learning models can improve detection speed and accuracy, even in complex and dynamic environments.”
Real-Time Onboard Object Detection for Augmented Reality: Enhancing Head-Mounted Display with YOLOv8
Łysakowski, M., Żywanowski, K., Banaszczyk, A., Nowicki, M., Skrzypczyński, P., & Tadeja, S. (2023) · IEEE International Conference on Edge Computing and Communications (EDGE) · doi:10.1109/edge60047.2023.00059 · Cited by 59 · cited 1×
“A key study by Farasin et al. [19] examined the feasibility of deploying real-time object detection models on HoloLens, emphasizing the trade-off between model complexity and real-time performance.”
Zheng, L., Liu, X., An, Z., Li, S., & Zhang, R. (2020) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2019.12.002 · Cited by 56 · cited 1×
“Similarly, Zheng et al. [13] developed an AI-powered cable assembly system that uses deep learning to detect and position brackets accurately, thus streamlining industrial automation tasks.”
Accurate Object Detection System on HoloLens Using YOLO Algorithm
Bahri, H., Krčmařík, D., & Kočí, J. (2019) · 2019 International Conference on Control, Artificial Intelligence, Robotics & Optimization (ICCAIRO) · doi:10.1109/iccairo47923.2019.00042 · Cited by 43 · cited 1×
“Bahri et al. [16] demonstrated how YOLOv3 can be integrated with HoloLens for object detection, achieving real-time classification with moderate performance limitations due to computational constraints.”
Eckert, M., Blex, M., & Friedrich, C. M. (2018) · International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC) · doi:10.5220/0006655605550561 · Cited by 33 · cited 1×
“Similarly, Eckert et al. [17] explored the use of YOLOv2 in HoloLens to aid visually impaired users, allowing for real-time object recognition with audio feedback.”
Real-time Object Detection and Tracking in Mixed Reality using Microsoft HoloLens
Farasin, A., Peciarolo, F., Grangetto, M., Gianaria, E., & Garza, P. (2020) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0008877901650172 · Cited by 31 · cited 1×
“Łysakowski et al. [18] implemented YOLOv8 on HoloLens 2 for real-time detection tasks, demonstrating that deep learning models can be successfully deployed on AR headsets, although with latency and processing limitations.”
Surgical Instrument Recognition Based on Improved YOLOv5
Jiang, K., Pan, S., Yang, L., Yu, J., Lin, Y., & Wang, H. (2023) · Applied Sciences · doi:10.3390/app132111709 · Cited by 30 · cited 1×
“Wang et al. [14] developed a YOLOv5-based model to identify medical instruments in surgical environments, highlighting the potential of CNNs to improve procedural precision.”
Does Colour-Coded Labelling Reduce the Risk of Medication Errors?
Filiatrault, P., & Hyland, S. (2009) · The Canadian Journal of Hospital Pharmacy · doi:10.4212/cjhp.v62i2.446 · Cited by 24 · cited 1×
“However, its effectiveness is limited by the small number of distinctly recognizable colors, making it inadequate to differentiate a wide range of medications [3].”
Kazi, R., Hoyle, J. D., Huffman, C., Ekblad, G., Ruffing, R. P., Dunwoody, S., Hover, T., Cody, S., & Fales, W. D. (2023) · Prehospital Emergency Care · doi:10.1080/10903127.2022.2162648 · Cited by 23 · cited 1×
“…ongoing prevalence of errors despite the availability of state-wide dosing resources such as the MI-MEDIC card, with a notable proportion of these errors arising from medications necessitating intricate calculations (e.g., dextrose, glucagon, intranasal fentanyl, midazolam, and epinephrine) [2].”
Virtual, augmented, and mixed reality: potential clinical and training applications in pediatrics
Yoo, S., & Son, M. H. (2023) · Korean Journal of Pediatrics · doi:10.3345/cep.2022.00731 · Cited by 21 · cited 2×
“Additionally, Yoo and Son [8] conducted a comprehensive review of MR applications in healthcare, emphasizing the growing role of Mixed Reality in medical training and procedural guidance.”
Home-based mirror therapy in phantom limb pain treatment: the augmented humans framework
Marullo, G., Innocente, C., Ulrich, L., Faro, A. L., Porcelli, A., Ruggieri, R., Vecchio, B., & Vezzetti, E. (2025) · Multimedia Tools and Applications · doi:10.1007/s11042-025-20628-1 · Cited by 12 · cited 1×
“Recent research [9] highlights the use of extended reality (XR) frameworks like home-based mirror therapy for phantom limb pain, showing the growing relevance of AI-assisted XR interfaces across diverse user groups, including patients, nurses, and physicians.”
Osteoporotic spinal burst fracture in a young adult as first presentation of systemic mastocytosis
Ble, C., Tsitsopoulos, P. P., Anestis, D. M., Hadjileontiadou, S., Κoletsa, T., Papaioannou, M., & Tsonidis, C. A. (2016) · Journal of Surgical Case Reports · doi:10.1093/jscr/rjw063 · Cited by 4 · cited 1×
“Manual verification, which involves healthcare professionals cross-checking medication labels before administration, remains fundamental but is highly prone to human error due to fatigue, distractions, and high workloads [6].”
Use of Artificial Intelligence in the Healthcare Sector
Koyuncu, M. (2024) · Lokman Hekim Health Sciences · doi:10.14744/lhhs.2024.27920 · Cited by 4 · cited 1×
“Object detection algorithms, particularly YOLO-based models, have shown exceptional performance in real-time medical applications, enabling the rapid and accurate identification of objects in complex environments [10].”
Accurate Medical Vial Identification Through Mixed Reality: A HoloLens 2 Implementation
Mahmud, B. U., Hong, G. Y., Sharmin, A., Asher, Z. D., & Hoyle, J. D. (2024) · Electronics · doi:10.3390/electronics13224420 · Cited by 3 · cited 1×
“In our previous research [11], we introduced a barcode-based medical vial detection system that utilizes mixed reality technology, specifically designed for HoloLens 2.”
Self-citationEnhanced Pre-processing for Robust Small Object Detection
Al-Hababi, M. A. M., & Liu, Y. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.08.241 · Cited by 2
Tiwari, E. A. A. (2023) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v11i9.9049 · Cited by 1 · cited 1×
“Tiwari et al. [20] proposed an enhanced YOLOv4-based object detection and tracking system in Mixed Reality environments, improving classification accuracy and localization.”
Cited by
Aljami, H. M., Alrowais, N. A., AlAwajy, A. M., Alhrgan, S. O., Aldwaani, R. A., Alsawadi, M. S., Saqib, N. U., Alam, S. S., & Alsubaie, R. (2026) · Computers · doi:10.3390/computers15020074 · Cited by 24
A Mixed-Reality Approach to Cardiovascular Anatomy Education
Patil, S., Lalwani, V. R., Mahmud, B. U., Carr, S., Woodcock, J., Temprine, K., & Hong, G. Y. (2026) · Future Internet · doi:10.3390/fi18060314 · Cited by 4
Self-citationSun, X., Yang, J., & Zhou, Y. F. (2026) · Information · doi:10.3390/info17010111 · Cited by 3
“However, existing models for recognising facial expressions generally suffer from high computational complexity and a large number of parameters, which make it a major challenge to deploy them in mobile terminals, embedded devices, and low-power environments [2].”
Barua, S., Bahre, R., & Hong, G. Y. (2026) · IEEE Communications Magazine · doi:10.1109/mcom.001.2500767 · Cited by 2
“Multimodal Intelligence: Integrating multiple sources improves diagnostics [9], but most systems require cloud-scale computing.”
Self-citationAdvanced analysis and detection of cervical cancer using NASNetLarge
Bilgaiyan, S., Swain, A., Das, S., Medha, Kumar, K., & Roy, S. (2026) · Scientific Reports · doi:10.1038/s41598-026-38341-w
Metadata
- Authors
- Mahmud, B. U., Hong, G., Lalwani, V. R., Brown, N., & Asher, Z. D.
- Affiliations
- Western Michigan University (US)
- Department of Computer Science, Western Michigan University, Kalamazoo, MI 49008, USA
- Venue
- Future Internet
- Publisher
- MDPI AG
- Published
- 2025-05-16
- Volume
- 17
- Issue
- 5
- Pages
- 223
- ISSN
- 1999-5903 (ISSN-L)
- OpenAlex
- W4410446071
- Cited by
- 8
- References
- 18
- Citations by year
- 2026: 8
- FWCI
- 2.83
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Future Internet (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Industrial Vision Systems and Defect Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Augmented Reality Applications
- Keywords
- drug identification, mixed reality, YOLOv4, edge computing, real-time object detection, HoloLens 2, patient safety, pharmaceutical packaging, illumination robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.