CoChat
Sign Up
Cited by 19Medical Image Segmentation TechniquesKnee injuries and reconstruction techniquesMedical Imaging and Analysis

Multi-view point-based Registration for Native Knee Kinematics Measurement with Feature Transfer Learning.

Open access
Wang, Cong, Xie, Shuaining, Li, Kang, Wang, Chongyang, Liu, Xudong, Zhao, Liang, Tsai, Tsung-Yuan·2021·Engineering
DOI 10.1016/j.eng.2020.03.016
Open this paper in CoChat — free accountOpen PDF

Abstract

Deep-learning methods provide a promising approach for measuring in-vivo knee joint motion from fast registration of two-dimensional (2D) to three-dimensional (3D) data with a broad range of capture. However, if there are insufficient data for training, the data-driven approach will fail. We propose a feature-based transfer-learning method to extract features from fluoroscopic images. With three subjects and fewer than 100 pairs of real fluoroscopic images, we achieved a mean registration success rate of up to 40%. The proposed method provides a promising solution, using a learning-based registration method when only a limited number of real fluoroscopic images is available.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.eng.2020.03.016

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Each contextualization study that considered multiple objects did not explicitly isolate parameters for each object, but used a single NN with output layer indexed by object.

    “However, we found that each of the contextualization studies which considered multiple objects (Grupp et al., 2020c; Doerr et al., 2020; Wang et al., 2020; Esfandiari et al., 2021) did not explicitly isolate the parameters corresponding to each object, but instead used a single NN with data in the output layer indexed according to the appropriate object.”

    From The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective (2021) · cites this paper as Wang et al., 2020

    Section 3.1.3 describes tracking points with a pseudo-Siamese network to reproduce the pose of each bone, supporting the characterization of this as a multiple-object registration study. Section 3.2 says the network outputs tracked image points, but does not establish that its output layer is indexed by bone or that parameters are not isolated by object. Its branches are described by image source, not by object. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • Only three studies in the review register multiple objects through learning-based approaches, and these independently registered single objects to obtain relative poses.

    “We found that only three studies in our review register multiple objects through learning-based approaches (Varnavas et al., 2013, 2015b; Wang et al., 2020), and moreover, these studies independently registered single objects in order to obtain relative poses.”

    From The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective (2021) · cites this paper as Wang et al., 2020

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 45

  1. Kernkamp WA, Wang C, Li C, Hu H, Van Arkel ERA, Nelissen RGHH, et al. The medial patellofemoral ligament is a dynamic and anisometric structure: an in vivo study on length changes and isometry. Am J Sports Med 2019;47:1645-53.
  2. Li JS, Tsai TY, Clancy MM, Li G, Lewis CL, Felson DT. Weight loss changed gait kinematics in individuals with obesity and knee pain. Gait Posture 2019;68:461-5.
  3. Li G, Li JS, Torriani M, Hosseini A. Short-term contact kinematic changes and longer-term biochemical changes in the cartilage after ACL reconstruction: a pilot study. Ann Biomed Eng 2018;46:1797-805.
  4. Tsai TY, Liow MHL, Li G, Arauz P, Peng Y, Klemt C, et al. Bi-cruciate retaining total knee arthroplasty does not restore native tibiofemoral articular contact kinematics during gait. J Orthop Res 2019;37(9):1929-37.
  5. Perriman A, Leahy E, Semciw AI. The effect of open-versus closed-kinetic- chain exercises on anterior tibial laxity, strength, and function following anterior cruciate ligament reconstruction: a systematic review and meta- analysis. J Orthop Sports Phys Ther 2018;48(7):552-66.
  6. Tsai TY, Lu TW, Kuo MY, Lin CC. Effects of soft tissue artifacts on the calculated kinematics and kinetics of the knee during stair-ascent. J Biomech 2011;44 (6):1182-8.
  7. Lu TW, O'Connor JJ. Bone position estimation from skin marker co-ordinates using global optimisation with joint constraints. J Biomech 1999;32 (2):129-34.
  8. Alexander EJ, Andriacchi TP. Correcting for deformation in skin-based marker systems. J Biomech 2001;34(3):355-61.
  9. Lucchetti L, Cappozzo A, Cappello A, Della CU. Skin movement artefact assessment and compensation in the estimation of knee-joint kinematics. J Biomech 1998;31(11):977-84.
  10. Leardini A, Chiari L, Della Croce U, Cappozzo A. Human movement analysis using stereophotogrammetry: part 3. Soft tissue artifact assessment and compensation. Gait Posture 2005;21(2):212-25.
  11. Borotikar B, Lempereur M, Lelievre M, Burdin V, Ben Salem D, Brochard S. Dynamic MRI to quantify musculoskeletal motion: a systematic review of concurrent validity and reliability, and perspectives for evaluation of musculoskeletal disorders. PLoS ONE 2017;12(12):e0189587.
  12. Tanaka MJ, Elias JJ, Williams AA, Demehri S, Cosgarea AJ. Characterization of patellar maltracking using dynamic kinematic CT imaging in patients with patellar instability. Knee Surg Sports Traumatol Arthrosc 2016;24:3634-41.
  13. Patel VV, Hall K, Ries M, Lotz J, Ozhinsky E, Lindsey C, et al. A three- dimensional MRI analysis of knee kinematics. J Orthop Res 2004;22 (2):283-92.
  14. Muhle C, Brossmann J, Heller M. Kinematic CT and MR imaging of the patellofemoral joint. Eur Radiol 1999;9:508-18.
  15. Li G, Wuerz TH, DeFrate LE. Feasibility of using orthogonal fluoroscopic images to measure in vivo joint kinematics. J Biomech Eng 2004;126(2):314-8.
  16. Acker S, Li R, Murray H, St JP, Banks S, Mu S, et al. Accuracy of single-plane fluoroscopy in determining relative position and orientation of total knee replacement components. J Biomech 2011;44(4):784-7.
  17. Kim J, Li S, Pradhan D, Hammoud R, Chen Q, Yin FF, et al. Comparison of similarity measures for rigid-body CT/dual X-ray image registrations. Technol Cancer Res Treat 2007;6(4):337-46.
  18. Van der Bom IMJ, Klein S, Staring M, Homan R, Bartels LW, Pluim JPW. Evaluation of optimization methods for intensity-based 2D-3D registration in X-ray guided interventions. In: Proceedings of SPIE Medical Imaging; 2011 Feb 12-17; Orlando, FL, USA; 2011.
  19. Alam F, Rahman SU, Khusro S, Ullah S, Khalil A. Evaluation of medical image registration techniques based on nature and domain of the transformation. J Med Imaging Radiat Sci 2016;47(2):178-93.
  20. Ghafurian S, Hacihaliloglu I, Metaxas DN, Tan V, Li K. A computationally efficient 3D/2D registration method based on image gradient direction probability density function. Neurocomputing 2017;229:100-8.
  21. Zheng J, Miao S, Wang ZJ, Liao R. Pairwise domain adaptation module for CNN- based 2-D/3-D registration. J Med Imaging 2018;5(2):21204.
  22. Zheng J, Miao S, Liao R. Learning CNNs with pairwise domain adaption for real- time 6DoF ultrasound transducer detection and tracking from X-ray images. In: Proceedings of the 20th International Conference on Medical Image Computing and Computer-Assisted Intervention; 2017 Sep 11-13; Quebec City, QC, Canada; 2017.
  23. Markelj P, Tomaz ˇevic ˇD, Likar B, Pernuš F. A review of 3D/2D registration methods for image guided interventions. Med Image Anal 2012;16(3):642-61.
  24. Duan Q, Wang G, Wang R, Fu C, Li X, Gong M, et al. SenseCare: a research platform for medical image informatics and interactive 3D visualization 2020. arXiv:2004.07031v1. · arXiv
  25. Toth D, Miao S, Kurzendorfer T, Rinaldi CA, Liao R, Mansi T, et al. 3D/2D model- to-image registration by imitation learning for cardiac procedures. Int J Comput Assisted Radiol Surg 2018;13:1141-9.
  26. Liao H, Lin WA, Zhang J, Zhang J, Luo J, Zhou SK. Multiview 2D/3D rigid registration via a point-of-interest network for tracking and triangulation. In: Proceedings of the 2019 IEEE Conference on Computer Vision Pattern Recognition; 2019 Jun 15-20; Long Beach, CA, USA; 2019. p. 12638-47.
  27. Wallace WA, Johnson F. Detection and correction of geometrical distortion in X-ray fluoroscopic images. J Biomech 1981;14(2):123-5.
  28. Cerciello T, Bifulco P, Cesarelli M, Fratini A. A comparison of denoising methods for X-ray fluoroscopic images. Biomed Signal Process Control 2012;7 (6):550-9.
  29. Moore CS, Liney GP, Beavis AW, Saunderson JR. A method to produce and validate a digitally reconstructed radiograph-based computer simulation for optimisation of chest radiographs acquired with a computed radiography imaging system. Br J Radiol 2011;84(1006):890-902.
  30. Moore CS, Wood TJ, Saunderson JR, Beavis AW. A method to incorporate the effect of beam quality on image noise in a digitally reconstructed radiograph (DRR) based computer simulation for optimisation of digital radiography. Phys Med Biol 2017;62(18):7379-93.
  31. Miao S, Wang ZJ, Zheng Y, Liao R. Real-time 2D/3D registration via CNN regression. In: Proceedings of the 2016 IEEE 13th International Symposium Biomedical Imaging; 2016 Apr 13-16; Prague, Czech Republic; 2016. p. 1430-4.
  32. Pan J, Min Z, Zhang A, Ma H, Meng MQH. Multiview global 2D-3D registration based on branch and bound algorithm. In: Proceedings of the 2019 IEEE International Conference on Robotics and Biomimetics; 2019 Dec 6-8; Dali, China; 2019. p. 3082-7.
  33. Weiss K, Khoshgoftaar TM, Wang D. A survey of transfer learning. J Big Data 2016;3:9.
  34. Csurka G. A comprehensive survey on domain adaptation for visual applications. In: Csurka G, editor. Domain adaptation in computer vision applications. Cham: Springer International Publishing AG; 2017. p. 1-35.
  35. Zhang Y, Miao S, Mansi T, Liao R. Task driven generative modeling for unsupervised domain adaptation: application to X-ray image segmentation. In: Proceedings of the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention; 2018 Sep 16-20; Granada, Spain; 2018. p. 599-607.
  36. Hu J, Sun S, Yang X, Zhou S, Wang X, Fu Y, et al. Towards accurate and robust multi-modal medical image registration using contrastive metric learning. IEEE Access 2019;7:132816-27.
  37. Otake Y, Armand M, Armiger RS, Kutzer MD, Basafa E, Kazanzides P, et al. Intraoperative image-based multiview 2D/3D registration for image-guided orthopaedic surgery: incorporation of fiducial-based C-arm tracking and GPU- acceleration. IEEE Trans Med Imaging 2012;31(4):948-62.
  38. Li T, Zhu H. Research on digitally reconstructed radiograph algorithm improvement based on computed tomography image. In: Proceedings of the 2018 11th International Congress on Image Signal Processing, Biomedical Engineering Informatics; 2018 Oct 13-15; Beijing, China; 2018. p. 1-5.
  39. Fischer B, Modersitzki J. Ill-posed medicine-an introduction to image registration. Inverse Probl 2008;24:034008.
  40. Hartley R, Zisserman A. Multiple view geometry in computer vision. Cambridge: Cambridge University Press; 2003.
  41. Seber GAF. Multivariate observations. New York: John Wiley & Sons; 1983.
  42. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014. arXiv:1409.1556. · arXiv
  43. Baltzopoulos V. A videofluoroscopy method for optical distortion correction and measurement of knee-joint kinematics. Clin Biomech 1995;10(2):85-92.
  44. Tsai TY, Dimitriou D, Li G, Kwon YM. Does total hip arthroplasty restore native hip anatomy? Three-dimensional reconstruction analysis. Int Orthop 2014;38:1577-83.
  45. Van de Kraats EB, Penney GP, Tomazevic D, Van Walsum T, Niessen WJ. Standardized evaluation methodology for 2-D-3-D registration. IEEE Trans Med Imaging 2005;24(9):1177-89.

Cited by

Sorted by citations

Authors
Wang, C., Xie, S., Li, K., Wang, C., Liu, X., Zhao, L., & Tsai, T.-Y.
Affiliations
  • Shanghai Jiao Tong University (CN)
  • Sensetime (China) (CN)
  • Rutgers, The State University of New Jersey (US)
Corresponding author
  • Liu, Xudong
  • Zhao, Liang
  • Tsai, Tsung-Yuan
ORCID
Venue
Engineering
Publisher
Elsevier BV
Published
2021-06
Volume
7
Issue
6
Pages
881-888
ISSN
2095-8099 (ISSN-L)
DOI
10.1016/j.eng.2020.03.016
OpenAlex
W3043844956
Cited by
19
References
3
Citations by year
2026: 1 · 2025: 2 · 2024: 8 · 2023: 2 · 2022: 4 · 2021: 2
FWCI
0.78
Type
journal article
Language
en
Open access
Licence
Locations
  • Engineering (publishedVersion, cc-by-nc-nd)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Natural Science Foundation of China
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Health Sciences › Medicine › Surgery › Knee injuries and reconstruction techniques
  • Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
Keywords
knee kinematics, multi-view registration, limited training data, 2D-3D registration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review