Advertisement
Review Article| Volume 1, P83-94, September 2019

Deep Learning in Musculoskeletal Imaging

  • Fang Liu
    Correspondence
    Corresponding author. Department of Radiology, Wisconsin Institutes for Medical Research, 1111 Highland Avenue, Madison, WI 53705-2275.
    Affiliations
    Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Avenue, Madison, WI 53705-2275, USA
    Search for articles by this author
  • Richard Kijowski
    Affiliations
    Department of Radiology, University of Wisconsin School of Medicine and Public Health, 1111 Highland Avenue, Madison, WI 53705-2275, USA
    Search for articles by this author
      Deep learning methods have been shown to be highly efficient and accurate for segmenting musculoskeletal tissues from medical images, which may eventually allow incorporation of quantitative image analysis into clinical practice.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Advances in Clinical Radiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • LeCun Y.
        • Bengio Y.
        • Hinton G.
        Deep learning.
        Nature. 2015; 521 (Available at:) (Accessed August 18, 2017): 436-444
        • Wernick M.N.
        • Yang Y.
        • Brankov J.G.
        • et al.
        Machine learning in medical imaging.
        IEEE Signal Process Mag. 2010; 27 (Available at:) (Accessed December 11, 2018): 25-38
        • Giger M.L.
        Machine learning in medical imaging.
        J Am Coll Radiol. 2018; 15 (Available at:) (Accessed December 11, 2018): 512-520
        • Erickson B.J.
        • Korfiatis P.
        • Akkus Z.
        • et al.
        Machine learning for medical imaging.
        Radiographics. 2017; 37 (Available at:) (Accessed December 11, 2018): 505-515
        • Russakovsky O.
        • Deng J.
        • Su H.
        • et al.
        ImageNet large scale visual recognition challenge.
        Int J Comput Vis. 2015; 115 (Available at:) (Accessed April 13, 2018): 211-252
        • Krizhevsky A.
        • Sutskever I.
        • Hinton G.E.
        ImageNet classification with deep convolutional neural networks.
        Adv Neural Inf Process Syst. 2012; 25 (Available at:): 1097-1105
        • Litjens G.
        • Kooi T.
        • Bejnordi B.E.
        • et al.
        A survey on deep learning in medical image analysis.
        Med Image Anal. 2017; 42: 60-88
        • Shen D.
        • Wu G.
        • Suk H.-I.
        Deep learning in medical image analysis.
        Annu Rev Biomed Eng. 2017; 19 (Available at:) (Accessed April 18, 2018): 221
      1. Nair V, Hinton GE. Rectified linear units improve restricted Boltzmann machines. Proc 27th Int Conf Mach Learn. Haifa, June 21–24, 2010. p. 807–814.

      2. Bottou L. Large-scale machine learning with stochastic gradient descent. 19th Int Conf Comput Stat Heidelberg. Paris, August 22–27, 2010. p. 177–186.

        • Tajbakhsh N.
        • Shin J.Y.
        • Gurudu S.R.
        • et al.
        Convolutional neural networks for medical image analysis: full training or fine tuning?.
        IEEE Trans Med Imaging. 2016; 35 (Available at:) (Accessed March 7, 2018): 1299-1312
        • Shin H.-C.
        • Roth H.R.
        • Gao M.
        • et al.
        Deep convolutional neural networks for computer-aided detection: cnn architectures, dataset characteristics and transfer learning.
        IEEE Trans Med Imaging. 2016; 35 (Available at:) (Accessed December 14, 2018): 1285-1298
        • McWalter E.J.
        • Wirth W.
        • Siebert M.
        • et al.
        Use of novel interactive input devices for segmentation of articular cartilage from magnetic resonance images.
        Osteoarthr Cartil. 2005; 13 (Available at:) (Accessed March 11, 2017): 48-53
        • Shim H.
        • Chang S.
        • Tao C.
        • et al.
        Knee cartilage: efficient and reproducible segmentation on high-spatial-resolution MR images with the semiautomated graph-cut algorithm method.
        Radiology. 2009; 251 (Available at:) (Accessed March 7, 2018): 548-556
        • Sharma N.
        • Aggarwal L.M.
        Automated medical image segmentation techniques.
        J Med Phys. 2010; 35 (Available at:) (Accessed March 7, 2018): 3-14
        • Prasoon A.
        • Petersen K.
        • Igel C.
        • et al.
        Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network.
        Med Image Comput Comput Assist Interv. 2013; 16 (Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics). Available at:) (Accessed September 17, 2017): 246-253
        • Liu F.
        • Zhou Z.
        • Jang H.
        • et al.
        Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging.
        Magn Reson Med. 2017; (Available at:) (Accessed July 29, 2017)https://doi.org/10.1002/mrm.26841
        • Badrinarayanan V.
        • Kendall A.
        • Cipolla R.
        SegNet: a deep convolutional encoder-decoder architecture for image segmentation. ArXiv e-prints. arXiv preprint arXiv:1511.00561.
        (Available at:) (Accessed February 21, 2017)
        • Ronneberger O.
        • Fischer P.
        • Brox T.
        U-Net: convolutional networks for biomedical image segmentation.
        in: Navab N. Hornegger J. Wells W.M. Med image comput comput interv -- MICCAI 2015 18th Int Conf Munich, ger Oct 5-9, 2015, proceedings, part III. Springer International Publishing, Cham (Switzerland)2015: 234-241https://doi.org/10.1007/978-3-319-24574-4_28
        • Zhou Z.
        • Zhao G.
        • Kijowski R.
        • et al.
        Deep convolutional neural network for segmentation of knee joint anatomy.
        Magn Reson Med. 2018; https://doi.org/10.1002/mrm.27229
        • Deniz C.M.
        • Xiang S.
        • Hallyburton R.S.
        • et al.
        Segmentation of the proximal femur from MR images using deep convolutional neural networks.
        Sci Rep. 2018; 8 (Available at:) (Accessed December 10, 2018): 16485
        • Norman B.
        • Pedoia V.
        • Majumdar S.
        Use of 2D U-net convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine relaxometry and morphometry.
        Radiology. 2018; (Available at:) (Accessed April 12, 2018): 172322
        • Tack A.
        • Mukhopadhyay A.
        • Zachow S.
        Knee menisci segmentation using convolutional neural networks: data from the Osteoarthritis Initiative.
        Osteoarthr Cartilage. 2018; 26 (Available at:) (Accessed December 10, 2018): 680-688
        • Liu F.
        SUSAN: segment unannotated image structure using adversarial network.
        Magn Reson Med. 2018; 81 (Available at:) (Accessed December 10, 2018): 3330-3345
        • Chen H.
        • Zhang Y.
        • Zhang W.
        • et al.
        Low-dose CT via convolutional neural network.
        Biomed Opt Express. 2017; 8 (Available at:) (Accessed December 11, 2018): 679-694
        • Hammernik K.
        • Klatzer T.
        • Kobler E.
        • et al.
        Learning a variational network for reconstruction of accelerated MRI data.
        Magn Reson Med. 2017; 79 (Wiley-Blackwell; Available at:) (Accessed April 29, 2017): 3055-3071
        • Mardani M.
        • Gong E.
        • Cheng J.Y.
        • et al.
        Deep generative adversarial neural networks for compressive sensing (GANCS) MRI.
        IEEE Trans Med Imaging. 2018; 38 (Available at:) (Accessed July 31, 2018): 167-179
        • Chaudhari A.S.
        • Fang Z.
        • Kogan F.
        • et al.
        Super-resolution musculoskeletal MRI using deep learning.
        Magn Reson Med. 2018; 80 (Available at:) (Accessed July 2, 2018): 2139-2154
        • Liu F.
        • Jang H.
        • Kijowski R.
        • et al.
        Deep learning MR imaging–based attenuation correction for PET/MR imaging.
        Radiology. 2017; 286 (Available at:) (Accessed September 26, 2017): 676-684
        • Leynes A.P.
        • Yang J.
        • Wiesinger F.
        • et al.
        Direct PseudoCT generation for Pelvis PET/MRI attenuation correction using deep convolutional neural networks with multi-parametric MRI: zero echo-time and Dixon deep pseudoCT (ZeDD-CT).
        J Nucl Med. 2017; 59 (Available at:) (Accessed November 3, 2017): 852-858
        • Liu F.
        • Jang H.
        • Kijowski R.
        • et al.
        A deep learning approach for 18F-FDG PET attenuation correction.
        EJNMMI Phys. 2018; 5 (Available at:) (Accessed December 11, 2018): 24
        • Pruessmann K.P.
        • Weiger M.
        • Scheidegger M.B.
        • et al.
        SENSE: sensitivity encoding for fast MRI.
        Magn Reson Med. 1999; 42: 952-962
        • Griswold M.A.
        • Jakob P.M.
        • Heidemann R.M.
        • et al.
        Generalized autocalibrating partially parallel acquisitions (GRAPPA).
        Magn Reson Med. 2002; 47: 1202-1210
        • Sodickson D.K.
        • Manning W.J.
        Simultaneous acquisition of spatial harmonics (SMASH): fast imaging with radiofrequency coil arrays.
        Magn Reson Med. 1997; 38 (Available at:) (Accessed July 2, 2018): 591-603
        http://www.ncbi.nlm.nih.gov/pubmed/9324327
        • Lustig M.
        • Donoho D.
        • Pauly J.M.
        Sparse MRI: the application of compressed sensing for rapid MR imaging.
        Magn Reson Med. 2007; 58: 1182-1195
        • Goodfellow I.J.I.
        • Pouget-Abadie J.
        • Mirza M.
        • et al.
        Generative adversarial networks. arXiv Prepr arXiv …. 2014; 1–9.
        (Available at:) (Accessed July 4, 2017)
        • Larson D.B.
        • Chen M.C.
        • Lungren M.P.
        • et al.
        Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs.
        Radiology. 2018; 287 (Available at:) (Accessed December 10, 2018): 313-322
        • Kim D.H.
        • MacKinnon T.
        Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks.
        Clin Radiol. 2018; 73 (Available at:) (Accessed December 10, 2018): 439-445
        • Tiulpin A.
        • Thevenot J.
        • Rahtu E.
        • et al.
        Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach.
        Sci Rep. 2018; 8 (Available at:) (Accessed December 10, 2018): 1727
        • Norman B.
        • Pedoia V.
        • Noworolski A.
        • et al.
        Applying densely connected convolutional neural networks for staging osteoarthritis severity from plain radiographs.
        J Digit Imaging. 2019; 32: 471-477
        • Xue Y.
        • Zhang R.
        • Deng Y.
        • et al.
        A preliminary examination of the diagnostic value of deep learning in hip osteoarthritis.
        PLoS One. 2017; 12 (Available at:) (Accessed December 12, 2017): e0178992
        • Raghavendra U.
        • Bhat N.S.
        • Gudigar A.
        • et al.
        Automated system for the detection of thoracolumbar fractures using a CNN architecture.
        Future Gener Comput Syst. 2018; 85 (Available at:) (Accessed December 10, 2018): 184-189
        • Tomita N.
        • Cheung Y.Y.
        • Hassanpour S.
        Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans.
        Comput Biol Med. 2018; 98 (Available at:) (Accessed December 10, 2018): 8-15
        • Roth H.R.
        • Wang Y.
        • Yao J.
        • et al.
        Deep convolutional networks for automated detection of posterior-element fractures on spine CT.
        in: Tourassi G.D. Armato S.G. 2016: 97850P (Available at:) (Accessed December 10, 2018)
        • Roth H.R.
        • Yao J.
        • Lu L.
        • et al.
        Detection of sclerotic spine metastases via random Aggregation of deep convolutional neural network classifications.
        Springer, Cham (Switzerland)2015: 3-12 (Available at:) (Accessed December 10, 2018)
        • Chmelik J.
        • Jakubicek R.
        • Walek P.
        • et al.
        Deep convolutional neural network-based segmentation and classification of difficult to define metastatic spinal lesions in 3D CT data.
        Med Image Anal. 2018; 49 (Available at:) (Accessed December 10, 2018): 76-88
        • Wang J.
        • Fang Z.
        • Lang N.
        • et al.
        A multi-resolution approach for spinal metastasis detection using deep Siamese neural networks.
        Comput Biol Med. 2017; 84 (Available at:) (Accessed September 15, 2018): 137-146
        • Jamaludin A.
        • Kadir T.
        • Zisserman A.
        SpineNet: automated classification and evidence visualization in spinal MRIs.
        Med Image Anal. 2017; 41 (Available at:) (Accessed December 12, 2017): 63-73
        • Kim K.
        • Kim S.
        • Lee Y.H.
        • et al.
        Performance of the deep convolutional neural network based magnetic resonance image scoring algorithm for differentiating between tuberculous and pyogenic spondylitis.
        Sci Rep. 2018; 8 (Nat Publishing Group; Available at:) (Accessed December 10, 2018): 13124
        • Liu F.
        • Zhou Z.
        • Samsonov A.
        • et al.
        Deep learning approach for evaluating knee MR images: achieving high diagnostic performance for cartilage lesion detection.
        Radiology. 2018; (Available at:) (Accessed July 31, 2018): 172986
        • Pedoia V.
        • Norman B.
        • Mehany S.N.
        • et al.
        3D convolutional neural networks for detection and severity staging of meniscus and PFJ cartilage morphological degenerative changes in osteoarthritis and anterior cruciate ligament subjects.
        J Magn Reson Imaging. 2018; 49 (Available at:) (Accessed December 10, 2018): 400-410
        • Bien N.
        • Rajpurkar P.
        • Ball R.L.
        • et al.
        Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of MRNet. Saria S, editor.
        PLoS Med. 2018; 15 (Available at:) (Accessed December 10, 2018): e1002699