Background:Systematic Power or Color Doppler (CD) ultrasound (US) of joints can be used for early detection of Rheumatoid Arthritis (RA), predicting radiographic progression and early detection of disease flare in established RA [1, 2]. The international standard for performing RA US scanning and evaluation of disease activity is the OMERACT-ELUAR Synovitis Scoring (OESS) system [1, 3].To further mitigate the operator-dependency in scoring disease activity on CD US images in future trials and clinical practice, we proposed the use of convolutional neural networks (CNN) to automatically grade CD US images according to the OESS definitions. This study is a continuation of the findings in our previous work, where we developed a CNN for four-class CD US OESS scoring with a test accuracy of 75.0% [4].Objectives:Since our last contribution, we have further developed the architecture of this neural network and can here present a new idea applying a Cascaded Convolutional Neural Network design. We evaluate the generalizability of this method on unseen data, comparing the CNN with an expert rheumatologist.Methods:The images used for developing the algorithms were graded by a single expert rheumatologist according to the OESS system. The CNNs in the cascade were trained individually, after which they were combined to form the cascade model as shown in figure 1. The algorithms were evaluated on a separate test dataset, which came from the same distribution as the training dataset. The algorithms were compared to the gradings of an expert rheumatologist on a per-joint basis using a Kappa test, and on a per-patient basis using a Wilcoxon Signed Rank test.Figure 1.CNN-1 is the first CNN in the model and distinguishes between RA disease grade (DG) 0 and DG’s 1, 2 and 3. CNN-2 is the second CNN and distinguishes between DG 1 and DG’s 2 and 3. CNN-3 is the final CNN which distinguishes between DG’s 2 and 3.Results:With 1678 images available for training and 322 images for testing the model, the model achieved an overall 4-class accuracy of 83.9%. On a per-patient level, there was no significant difference between the classifications of the model and of a human expert (p=0.85).Conclusion:We have shown that dividing a four-degree classification task into three successive binary classification tasks has resulted in a model capable of making correct classifications in 83.9% of the cases for a test set of ultrasound images with a naturally occurring distribution of RA joint disease activity scores.Furthermore, we have shown that the cascade model can produce classification decisions comparable with a human rheumatologist when applied on a per-patient basis. This emphasizes the potential of using CNNs with this architecture as a strong assistive tool for the objective assessment of disease activity of RA patients.References:[1]D’Agostino M-A, Terslev L, Aegerter P, et al. (2017). Scoring ultrasound synovitis in rheumatoid arthritis: a OMERACT-EULAR ultrasound taskforce-Part 1: definition and development of a standardised, consensus-based scoring system. RMD Open.[2]Paulshus NS, Aga A-B, Olsen I, et al. (2018). Clinical and ultrasound remission after 6 months of treat-to-target therapy in early rheumatoid arthritis: associations to future good radiographic and physical outcomes. Ann Rheum Dis, 77, s. 1425-25.[3]Terslev L, Naredo E, Aegerter P, et al. (3 2017). Scoring ultrasound synovitis in rheumatoid arthritis: a OMERACT-EULAR ultrasound taskforce-Part 2: reliability and application to multiple joints of a standardised consensus-based scoring system. RMD Open.[4]Andersen JKH, Pedersen JS, Laursen MS, et al. Neural networks for automatic scoring of arthritis disease activity on ultrasound images. RMD Open 2019; 5:e000891. doi:10.1136/ rmdopen-2018-000891Disclosure of Interests:None declared
Automatic tongue contour extraction in ultrasound images with convolutional neural networks
