Automated left ventricular dimension assessment using artificial intelligence

Background and purpose Artificial intelligence (AI) has the potential to greatly improve efficiency and reproducibility of quantification in echocardiography, but to gain widespread use it must both meet expert standards of excellence and have a transparent methodology. We developed an online platform to enable multiple collaborators to annotate medical images for training and validating neural networks. Methods Using our online collaborative platform 9 expert echocardiographers labelled 2056 images that comprised the training dataset. They labelled the four points from where the standard parasternal long axis (PLAX) measurements (interventricular septum, posterior wall, left ventricular dimension) would be made. Using these labelled images we trained a 2d convolutional neural network to replicate these labels. Separately, we curated an external validation dataset of the systolic and diastolic frames of 100 PLAX acquisitions. Each of these images were labelled twice by 13 different experts, and the average of the 26 measurements was taken as the consensus standard. We then compared the individual experts and the AI measurements on the external validation dataset to the consensus standard, and calculated the precision standard deviation (SD) of the signed differences from the consensus standard. Results For diastolic septum thickness, the AI had a precision SD of 1.8 mm (ICC 0.81; 95% CI 0.73 to 0.97), compared with 2.0 mm for the individual experts (ICC 0.64; 95% CI 0.57 to 0.72). For diastolic posterior wall thickness, the AI had a precision SD 1.4 mm (ICC 0.54; 95% CI 0.38 to 0.66), and the individual experts 2.2 mm (ICC 0.37; 95% CI 0.29 to 0.46). The AI's precision SD for left ventricular internal dimension was 3.5 mm (ICC 0.93, 95% CI 0.90 to 0.94), and for individual experts was 4.4mm (ICC 0.82, 95% CI 0.78 to 0.95). Both the experts and AI performed better in diastole than systole (precision SD AI 2.5mm vs 4.3mm, p<0.0001; experts 3.3mm vs 5.3mm, p<0.0001). Conclusions AI trained by a group of echocardiography experts was able to perform PLAX measurements which matched the reference standard more closely than any individual expert's own measurements. This open, collaborative approach may be a model for the development of AI that is explainable to, and trusted by clinicians. FUNDunding Acknowledgement Type of funding sources: Public grant(s) – National budget only. Main funding source(s): NIHR Imperil BRC ITMATDr Howard was additionally funded by Wellcome. Online collaborative platform Results of AI and experts