The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Automated HFrEF Diagnosis Using an Optimized TimeSformer Model in Echocardiography
Open accessAbstract
Heart failure with reduced ejection fraction (HFrEF) presents a significant diagnostic challenge, particularly in advanced stages. Deep learning (DL) models offer promising potential for automated HFrEF detection, but their effectiveness is often limited in small and imbalanced clinical datasets. In this study, we propose a novel approach to enhance HFrEF detection from echocardiographic videos. We adapt the TimeSformer architecture-a Transformer-based model optimized for spatiotemporal feature extraction in video data-and apply it for the first time in echocardiography. In addition, we introduce a domain-informed left ventricle (LV) masking method using image segmentation to guide model attention toward diagnostically critical regions, boosting overall performance. Our methodology is evaluated both directly on a large-scale benchmark dataset and on a smaller, specialized clinical dataset from our cardiology department after fine-tuning. Experimental results show that the proposed framework yields substantial performance gains: on the benchmark dataset, we observe a 3% improvement in both model accuracy and area under the curve (AUC), while on the specialized dataset, improvements reach 7% in accuracy and 30% in AUC values. Moreover, TimeSformer consistently outperforms conventional approaches after the implementation of fine-tuning and LV masking. These findings highlight a practical and generalizable strategy for improving automated HFrEF diagnosis, with strong implications for clinical decision support in data-scarce healthcare settings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Vision Transformer (ViT) [17], and video-specific variants such as TimeSformer [ 18, 19 ] and Video Swin Transformer [20] capture longrange dependencies and global spatiotemporal context that CNNs often miss.”
From Cross-dataset video deepfake detection using Transformer and CNN architectures (2026) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“…its performance on video tasks by learning from smaller spatial features, (iv) ResNet18, ResNet50 and ResNet101, which are part of the ResNet [37] family and are distinguished by their residual connections that facilitate deeper networks and address the vanishing gradient problem, providing…”
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“…providing strong spatial feature extraction but with limitations in capturing temporal information in their basic forms, (v) VGG16 and VGG19 [38], characterized by their uniform convolutional layers and simplicity, effective at spatial feature extraction but limited in their ability to…”
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Review of deep learning: concepts, CNN architectures, challenges, applications, future directions
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021) · Journal of Big Data · doi:10.1186/s40537-021-00444-8 · Cited by 7,906
Rethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517 · cited 1×
“To achieve this masking, we employed the Deeplabv3-ResNet50 model [41], a state-of-the-art DL architecture designed for semantic segmentation at multiple scales.”
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Authors/Task Force Members:, McDonagh, T. A., Metra, M., Adamo, M., Gardner, R. S., Baumbach, A., Böhm, M., Burri, H., Butler, J., Čelutkienė, J., et al. (2022) · European Journal of Heart Failure · doi:10.1002/ejhf.2333 · Cited by 2,171
Video-based AI for beat-to-beat assessment of cardiac function
Ouyang, D., He, B., Ghorbani, A., Yuan, N., Ebinger, J., Langlotz, C. P., Heidenreich, P. A., Harrington, R. A., Liang, D. H., Ashley, E. A., et al. (2020) · Nature · doi:10.1038/s41586-020-2145-8 · Cited by 1,153 · cited 3×
“Other studies, such as those by Tromp et al. [24] and Liu et al. [25], used CNN architectures on the EchoNet-Dynamic dataset [26] to classify patients with LVEF thresholds of 40% and 50%, respectively.”
InfluentialDeep learning modelling techniques: Current progress, applications, advantages, and challenges
Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079 · cited 1×
“Furthermore, DL models often face significant challenges when working with small datasets, particularly when the data consists of videos, such as echocardiograms [9].”
Legal and Ethical Consideration in Artificial Intelligence in Healthcare: Who Takes Responsibility?
Naik, N., Hameed, B. M. Z., Shetty, D. K., Swain, D., Shah, M., Paul, R., Aggarwal, K., Ibrahim, S., Patil, V., Smriti, K., et al. (2022) · Frontiers in Surgery · doi:10.3389/fsurg.2022.862322 · Cited by 931
Heart Failure With Reduced Ejection Fraction: A Review
Murphy, S. P., Ibrahim, N. E., & Januzzi, J. L. (2020) · JAMA · doi:10.1001/jama.2020.10262 · Cited by 769 · cited 1×
“Among its subtypes, HFrEF is particularly concerning [3].”
Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition
Hara, K., Kataoka, H., & Satoh, Y. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.373 · Cited by 726 · cited 1×
“…spatial feature extraction but limited in their ability to capture temporal dynamics when applied to video data, and (vi) 3D-CNN [39] and 3D-ResNet [40], which are specifically designed for video classification tasks by processing spatial and temporal dimensions simultaneously through 3D…”
A survey on active learning and human-in-the-loop deep learning for medical image analysis
Budd, S., Robinson, E. C., & Kainz, B. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102062 · Cited by 606 · cited 1×
“Additionally, active learning approaches, where the model actively queries clinicians for annotations only on the most uncertain or important samples, could help reduce the annotation burden [48].”
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond
Li, X., Xiong, H., Li, X., Wu, X., Zhang, X., Liu, J., Bian, J., & Dou, D. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01756-8 · Cited by 497 · cited 1×
“Interpretability and transparency remain critical challenges for DL models in medical applications [46]-[47].”
Pathophysiology of heart failure
Schwinger, R. H. G. (2021) · Cardiovascular Diagnosis and Therapy · doi:10.21037/cdt-20-302 · Cited by 428
Ethical implications of AI and robotics in healthcare: A review
Elendu, C., Amaechi, D. C., Elendu, T. C., Jingwa, K. A., Okoye, O. K., John Okah, M., Ladele, J. A., Farah, A. H., & Alimi, H. A. (2023) · Medicine · doi:10.1097/md.0000000000036671 · Cited by 370
Digitization of healthcare sector: A study on privacy and security concerns
Paul, M., Maglaras, L., Ferrag, M. A., & Almomani, I. (2023) · ICT Express · doi:10.1016/j.icte.2023.02.007 · Cited by 347
A survey on the interpretability of deep learning in medical diagnosis
Teng, Q., Liu, Z., Song, Y., Han, K., & Lu, Y. (2022) · Multimedia Systems · doi:10.1007/s00530-022-00960-4 · Cited by 198
Tromp, J., Seekings, P. J., Hung, C.-L., Iversen, M. B., Frost, M. J., Ouwerkerk, W., Jiang, Z., Eisenhaber, F., Goh, R. S. M., Zhao, H., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00235-1 · Cited by 195
Classification of Heart Failure According to Ejection Fraction
Lam, C. S. P., & Solomon, S. D. (2021) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2021.04.070 · Cited by 174
Bachtiger, P., Petri, C. F., Scott, F. E., Ri Park, S., Kelshiker, M. A., Sahemey, H. K., Dumea, B., Alquero, R., Padam, P. S., Hatrick, I. R., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00256-9 · Cited by 167
Akerman, A. P., Porumb, M., Scott, C. G., Beqiri, A., Chartsias, A., Ryu, A. J., Hawkes, W., Huntley, G. D., Arystan, A. Z., Kane, G. C., et al. (2023) · JACC Advances · doi:10.1016/j.jacadv.2023.100452 · Cited by 149
State-of-the-Art Deep Learning Methods on Electrocardiogram Data: Systematic Review
Πετμεζάς, Γ., Stefanopoulos, L., Kilintzis, V., Tzavelis, A., Rogers, J. A., Katsaggelos, A. K., & Maglaveras, N. (2022) · JMIR Medical Informatics · doi:10.2196/38454 · Cited by 147
Self-citationCho, J., Lee, B., Kwon, J., Lee, Y., Park, H., Oh, B., Jeon, K., Park, J., & Kim, K. (2020) · ASAIO Journal · doi:10.1097/mat.0000000000001218 · Cited by 102
Liu, X., Fan, Y., Li, S., Chen, M., Li, M., Hau, W. K., Zhang, H., Xu, L., & Lee, A. P. (2021) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.00416.2020 · Cited by 84
Kwon, J.-M., Kim, K.-H., Eisen, H. J., Cho, Y., Jeon, K.-H., Lee, S. Y., Park, J., & Oh, B.-H. (2021) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztaa015 · Cited by 82
Kwon, J., Jo, Y., Lee, S. Y., Kang, S., Lim, S.-Y., Lee, M. S., & Kim, K. (2022) · Diagnostics · doi:10.3390/diagnostics12030654 · Cited by 60
Gated recurrent unit-based heart sound analysis for heart failure screening
Gao, S., Zheng, Y., & Guo, X. (2020) · BioMedical Engineering OnLine · doi:10.1186/s12938-020-0747-x · Cited by 58 · cited 1×
“Gao et al. [14] utilized a gated recurrent unit (GRU) model to classify echocardiogram frames segmented via a semi-Markov process into normal, preserved, and reduced LVEF categories.”
Docherty, K. F., Lam, C. S. P., Rakisheva, A., Coats, A. J. S., Greenhalgh, T., Metra, M., Petrie, M. C., & Rosano, G. M. C. (2023) · European Journal of Heart Failure · doi:10.1002/ejhf.2946 · Cited by 58
Recent advancements and applications of deep learning in heart failure: Α systematic review
Πετμεζάς, Γ., Papageorgiou, V. Ε., Vassilikos, V. P., Pagourelias, E. D., Tsaklidis, G. M., Katsaggelos, A. K., & Maglaveras, N. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108557 · Cited by 58
Self-citationOn the Use of Deep Learning for Video Classification
Rehman, A. U., Belhouari, S. B., Kabir, M. A., & Khan, A. (2023) · Applied Sciences · doi:10.3390/app13032007 · Cited by 55
Papageorgiou, V. Ε. (2021) · Traitement du signal · doi:10.18280/ts.380302 · Cited by 38 · cited 1×
“It is important to note that in contrast to standard segmentation approaches, which have been extensively used in image recognition or disease detection [44], masking retains all image’s information while annotating the most significant part for the classification procedure.”
Self-citationUnterhuber, M., Rommel, K., Kresoja, K., Lurz, J. A., Kornej, J., Hindricks, G., Scholz, M. R., Thiele, H., & Lurz, P. C. (2021) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztab081 · Cited by 37 · cited 1×
“In a related study, Unterhuber et al. [19] developed a CNN model using ECG segments to identify HFpEF patients.”
Krittanawong, C., Omar, A. M. S., Narula, S., Sengupta, P. P., Glicksberg, B. S., Narula, J., & Argulian, E. (2023) · Life · doi:10.3390/life13041029 · Cited by 37
Choi, J., Lee, S. J., Chang, M., Lee, Y., Oh, G. C., & Lee, H. (2022) · Scientific Reports · doi:10.1038/s41598-022-18640-8 · Cited by 35
Classifying heart failure based on ejection fraction: imperfect but enduring
Khan, M. S., Shahid, I., Fonarow, G. C., & Greene, S. J. (2022) · European Journal of Heart Failure · doi:10.1002/ejhf.2470 · Cited by 28
Deep Learning Innovations in Video Classification: A Survey on Techniques and Dataset Evaluations
Mao, M., Lee, A., & Hong, M. (2024) · Electronics · doi:10.3390/electronics13142732 · Cited by 26 · cited 1×
“In addition to the TimeSformer model, several commonly used DL architectures for video classification were evaluated to establish a baseline for comparison [33].”
Savarese, G., Gatti, P., Benson, L., Adamo, M., Chioncel, O. D., Crespo-Leiro, M. G., Anker, S. D., Coats, A. J. S., Filippatos, G. S., Lainščak, M., et al. (2023) · American Heart Journal · doi:10.1016/j.ahj.2023.11.008 · Cited by 21
Πετμεζάς, Γ., Papageorgiou, V. Ε., Vassilikos, V. P., Pagourelias, E. D., Tachmatzidis, D., Tsaklidis, G. M., Katsaggelos, A. K., & Maglaveras, N. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104800 · Cited by 20
Self-citationCarnicelli, A., Clare, R. M., Hofmann, P., Chiswell, K., DeVore, A. D., Vemulapalli, S., Felker, G. M., Sarocco, P., & Mentz, R. J. (2021) · Journal of the American Heart Association · doi:10.1161/jaha.120.021276 · Cited by 16 · cited 1×
“Defined by a left ventricular ejection fraction (LVEF) of 40% or below, HFrEF often results in progressive cardiac remodeling and is associated with urgent hospital admissions due to episodic worsening [4].”
ViViEchoformer: Deep Video Regressor Predicting Ejection Fraction
Akan, T., Alp, S., Bhuiyan, M. S., Helmy, T. A., Orr, A. W., Bhuiyan, M. M. R., Conrad, S. A., Vanchiere, J. A., Kevil, C. G., & Bhuiyan, M. A. N. (2024) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-024-01336-y · Cited by 15
Crockett, D. K., Kelly, C., Brundage, J., Jones, J., & Ockerse, P. M. (2022) · Journal of Ultrasound in Medicine · doi:10.1002/jum.16007 · Cited by 14
Behnami, D., Luong, C. L., Vaseli, H., Girgis, H., Abdi, A. H., Hawley, D., Gin, K., Rohling, R. N., Abolmaesumi, P., & Tsang, T. S. M. (2019) · Computer Methods in Biomechanics and Biomedical Engineering Imaging & Visualization · doi:10.1080/21681163.2019.1650398 · Cited by 11 · cited 1×
“Focusing on echocardiogram-based analysis, Behnami et al. [22] proposed a CNN-RNN model to classify LVEF into high-risk (≤ 40”
Challenges and opportunity in the era of quantitative echocardiography
Yu, C. (2017) · Echo Research and Practice · doi:10.1530/erp-17-0049 · Cited by 9 · cited 1×
“These limitations highlight the need for more consistent and reliable diagnostic tools [6].”
Sakamoto, T., Ito, S., Endo, A., Yoshitomi, H., & Tanabe, K. (2020) · European Heart Journal · doi:10.1093/ehjci/ehaa946.0923 · Cited by 2 · cited 1×
“Moreover, Sakamoto et al. [23] utilized machine learning aiming to classify 162 patients with HFrEF into two clusters.”
Research works printed but not linked: 3
- Bertasius G, Wang H, Torresani L (2021) Is space-time attention all you need for video understanding? Int Conf Mach Learn 813–824. http://proceedings.mlr.press/v139/bertasius21a/bertasius21a.pdf
- Dosovitskiy A, Beyer L, Kolesnikov A et al (2021) An image is worth 16x16 words: Transformers for image recognition at scale. Int Conf Learn Representations. https://openreview.net/pdf?id=YicbFdNTTy
- Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105. http://books.nips.cc/papers/files/nips25/NIPS2012_0534.pdf
Cited by
Cross-dataset video deepfake detection using Transformer and CNN architectures
Petmezas, G. (2026) · Machine Vision and Applications · doi:10.1007/s00138-026-01809-w · Cited by 6
Πετμεζάς, Γ., Papageorgiou, V. Ε., PASSMAN, R. S., Rogers, J. A., Stefanopoulos, L., Katsaggelos, A. K., & Maglaveras, N. (2026) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2026.154199 · Cited by 3
Self-citationA bibliometric analysis of artificial intelligence applications in heart failure
Zhang, Z., Li, Y., Li, T., Wang, H., & Wang, Y. (2026) · Journal of Thoracic Disease · doi:10.21037/jtd-2026-0853
“Cardiac imaging technologies play an indispensable role in HF research and clinical care, enabling direct visualization of cardiac structural features, functional parameters, and hemodynamic alterations—all of which are critical for identifying HF phenotypes and assessing disease severity (37,38).”
Reject Option for Medical Image Classification Using CNN-Derived Latent-Space Neighbourhood Analysis
Arcangeli, A., Santi, L. A. D., Santarelli, M. F., Celi, S., & Positano, V. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02318-y
Metadata
- Authors
- Petmezas, G., Papageorgiou, V. E., Vassilikos, V., Pagourelias, E., Tachmatzidis, D., Tsaklidis, G., Katsaggelos, A. K., & Maglaveras, N.
- Affiliations
- Aristotle University of Thessaloniki (GR)
- Northwestern University (US)
- ORCID
- Venue
- Journal of Imaging Informatics in Medicine
- Published
- 2025-12-01
- Volume
- 39
- Issue
- 4
- Pages
- 3697-3712
- ISSN
- 0897-1889 (ISSN-L)
- 2948-2933
- PMC
- PMC13481916
- OpenAlex
- W4416946943
- Cited by
- 3
- References
- 47
- Citations by year
- 2026: 3
- FWCI
- 2.36
- Type
- journal article
- Language
- en
- Open access
- green
- pmc.ncbi.nlm.nih.gov
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Journal of Imaging Informatics in Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiovascular Function and Risk Factors
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- heart failure with reduced ejection fraction, automated diagnosis, echocardiography, TimeSformer, image segmentation, fine-tuning, data-scarce settings, clinical decision support
- MeSH terms
- Deep Learning, Algorithms, Echocardiography, Heart Failure, Humans, Image Interpretation, Computer-Assisted, Stroke Volume
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.