Sharing is Caring – A Systematic Review on how Frequently Dermoscopic Images are Shared in Inter-observer Investigations

Research interest in dermoscopy has accelerated, but the complete dermoscopic image sets used for inter-observer investigations for skin tumors are not often shared to the reader. The aim of this systematic review was to analyze what proportion of images depicting skin tumors are shared in the manuscripts of studies investigating inter-observer variation in the assessment of dermoscopic features and/or patterns. The Embase, MEDLINE, and Scopus databases were screened for eligible studies published from inception to July 2, 2020. For included investigations we extracted the proportion of lesion images presented in the manuscripts and or supplements. Overall, we included 61 studies (52 original investigations and 9 concise reports) in the time period of 1997 to 2020. These investigations combined, included 14,124 skin tumors of which 373 (3%) images were shared. Since data sharing must be promoted, this investigation should be a wake-up call for the dermatology research community and editorial offices.

IIB or not IIB, part 1: retrospective evaluation of Kenney–Doig categorization of equine endometrial biopsies at a veterinary diagnostic laboratory and comparison with published reports

The Kenney–Doig scale is a histopathology categorization (grading) system often used as the standard for assessing endometrial disease and communicating prognostic fertility information for equine breeding prospects. We investigated how Kenney–Doig categories compared within the same institution and across different institutions to determine if observer variability may contribute to category frequencies. We conducted a retrospective analysis of all equine endometrial submission records between 1998 and 2018 at the Western College of Veterinary Medicine (WCVM) and Prairie Diagnostic Services (PDS). Of 726 biopsies, we found the following category distribution: 46 of 726 (6.3%) I, 307 of 726 (42.3%) IIA, 326 of 726 (44.9%) IIB, and 47 of 726 (6.5%) III. We also conducted a review of the literature and included 6 studies reporting Kenney–Doig category distributions. Chi-square analysis showed significant differences between the category distribution found at WCVM and PDS and the category distribution reported in the 6 studies. To account for differences in mare populations, individual category distributions were generated for 5 pathologists at the WCVM and PDS. The Fisher exact test among these 5 Kenney–Doig categories revealed significant differences in category tendencies, suggesting that observer variation affects the use of the scale. Our results suggest that there is a need for prospective inter-rater and intra-rater agreement studies of the repeatability of the Kenney–Doig scale.

Variations in Head and Neck Treatment Plan Quality Assessment Among Radiation Oncologists and Medical Physicists in a Single Radiotherapy Department

BackgroundAgreement between planners and treating radiation oncologists (ROs) on plan quality criteria is essential for consistent planning. Differences between ROs and planning medical physicists (MPs) in perceived quality of head and neck cancer plans were assessed.Materials and MethodsFive ROs and four MPs scored 65 plans for in total 15 patients. For each patient, the clinical (CLIN) plan and two or four alternative plans, generated with automated multi-criteria optimization (MCO), were included. There was always one MCO plan aiming at maximally adhering to clinical plan requirements, while the other MCO plans had a lower aimed quality. Scores were given as follows: 1–7 and 1–2, not acceptable; 3–5, acceptable if further planning would not resolve perceived weaknesses; and 6–7, straightway acceptable. One MP and one RO repeated plan scoring for intra-observer variation assessment.ResultsFor the 36 unique observer pairs, the median percentage of plans for which the two observers agreed on a plan score (100% = 65 plans) was 27.7% [6.2, 40.0]. In the repeat scoring, agreements between first and second scoring were 52.3% and 40.0%, respectively. With a binary division between unacceptable (scores 1 and 2) and acceptable (3–7) plans, the median inter-observer agreement percentage was 78.5% [63.1, 86.2], while intra-observer agreements were 96.9% and 86.2%. There were no differences in observed agreements between RO–RO, MP–MP, and RO–MP pairs. Agreements for the highest-quality, automatically generated MCO plans were higher than for the CLIN plans.ConclusionsInter-observer differences in plan quality scores were substantial and could result in inconsistencies in generated treatment plans. Agreements among ROs were not better than between ROs and MPs, despite large differences in training and clinical role. High-quality automatically generated plans showed the best score agreements.

Impact of cone beam CT on diagnosis of external cervical resorption: the severity of resorption assessed in periapical radiographs and cone beam CT. A prospective clinical study

Objectives: To compare the severity of external cervical resorption (ECR) observed in periapical (PA) images and cone beam CT (CBCT) using the Heithersay classification system and pulp involvement; and to assess inter- and intraobserver reproducibility for three observers. Methods: CBCT examination was performed in 245 teeth (in 190 patients, mean age 40 years, range 12–82) with ECR diagnosed in PA images. Three observers scored the severity of ECR using the Heithersay classification system (severity class 1–4) and pulp involvement (yes/no) in both PA images and CBCT. Percentage concordance and κ-statistics described observer variation in PA images and CBCT for both inter- and intraobserver reproducibility. Results: For all three observers, the ECR score was the same in the two modalities in more than half of cases (average 59%; obs1: 54%, obs2: 63%, obs3: 61%). However, in 38% (obs1: 44%, obs2: 33%, obs3: 36%) of the cases, the observers scored more severe ECR in CBCT than in PA images (p < 0.001). The ECR score changed to a less severe score in CBCT only in 3% (obs1: 1%, obs2: 4%, obs3: 4%). For pulp involvement, 14% (obs1: 7%, obs2: 20%, obs3: 15%) of the cases changed from “no” in PA images to “yes” in CBCT. In general, κ values were higher for CBCT than for PA images for both the Heithersay classification score and pulp involvement. Conclusions: ECR was generally scored as more severe in CBCT than PA images using the Heithersay classification and also more cases had pulp involvement in CBCT.

Evaluation of auto-segmentation accuracy of cloud-based artificial intelligence and atlas-based models

Background Contour delineation, a crucial process in radiation oncology, is time-consuming and inaccurate due to inter-observer variation has been a critical issue in this process. An atlas-based automatic segmentation was developed to improve the delineation efficiency and reduce inter-observer variation. Additionally, automated segmentation using artificial intelligence (AI) has recently become available. In this study, auto-segmentations by atlas- and AI-based models for Organs at Risk (OAR) in patients with prostate and head and neck cancer were performed and delineation accuracies were evaluated. Methods Twenty-one patients with prostate cancer and 30 patients with head and neck cancer were evaluated. MIM Maestro was used to apply the atlas-based segmentation. MIM Contour ProtégéAI was used to apply the AI-based segmentation. Three similarity indices, the Dice similarity coefficient (DSC), Hausdorff distance (HD), and mean distance to agreement (MDA), were evaluated and compared with manual delineations. In addition, radiation oncologists visually evaluated the delineation accuracies. Results Among patients with prostate cancer, the AI-based model demonstrated higher accuracy than the atlas-based on DSC, HD, and MDA for the bladder and rectum. Upon visual evaluation, some errors were observed in the atlas-based delineations when the boundary between the small bowel or the seminal vesicle and the bladder was unclear. For patients with head and neck cancer, no significant differences were observed between the two models for almost all OARs, except small delineations such as the optic chiasm and optic nerve. The DSC tended to be lower when the HD and the MDA were smaller in small volume delineations. Conclusions In terms of efficiency, the processing time for head and neck cancers was much shorter than manual delineation. While quantitative evaluation with AI-based segmentation was significantly more accurate than atlas-based for prostate cancer, there was no significant difference for head and neck cancer. According to the results of visual evaluation, less necessity of manual correction in AI-based segmentation indicates that the segmentation efficiency of AI-based model is higher than that of atlas-based model. The effectiveness of the AI-based model can be expected to improve the segmentation efficiency and to significantly shorten the delineation time.

Locating hyperfunctioning parathyroid glands using 11C-Choline PET/CT: an inter- and intra-observer variation study

Background Use of 11C-Choline PET/CT is gaining ground in detecting hyperfunctioning parathyroid glands in primary hyperparathyroidism. The purpose of this study was to evaluate the robustness of 11C-Choline PET/CT by assessing intra- and inter-observer agreement to determine whether the method was reader sensitive and therefore should only be performed at highly specialised sites with a high number of cases. PET/CT images of 40 patients diagnosed with primary hyperparathyroidism were anonymised and evaluated three times by three readers: an expert reader and two non-experts (non-experts were experienced in PET/CT imaging, but not in 11C-Choline PET/CT in the setting of primary hyperparathyroidism). Number of hyperfunctioning parathyroid glands, location relative to the thyroid gland and confidence of each assessment (low, moderate or high) were noted, and intra- and inter-observer agreement calculated using Fleiss’ kappa method. Sensitivities and specificities of the non-experts were calculated using the expert reader as gold standard. Results Intra-observer agreement was ‘good’ to ‘near perfect’ for all readers. Inter-observer agreement was good between non-experts and the expert, with kappa values ≥ 0.74. Sensitivities between non-experts and the expert were high, > 81%, when assessing which side and 75% when assessing thyroid quadrant. All specificities were > 94%. Reader certainties were ‘high’ in > 80% of cases for the expert and > 70% and > 65%, respectively for the two non-experts. Conclusion 11C-Choline PET/CT is not reader sensitive for the localisation of hyperfunctioning parathyroid glands and may therefore be safely implemented at sites that have a moderate number of cases. Access to a cyclotron laboratory is, however, a necessity for the production of 11C-Choline. The study was conducted in accordance with the Helsinki 2 declaration and The International Council for Harmonisation Guideline for Good Clinical Practice (ICH_GCP) clinical trial, approved by the Research Ethics Committee of the Capital Region of Denmark (Journal-nr.:H-18012490, date of approval: 18 June 2018) and the Danish Medicines Agency (EudraCT no. 2018-000726-63, date of approval: 6 June 2018). The GCP unit in Eastern Denmark has carried out regular monitoring of the trial according to GCP (ID: 2018-1050).

DE-MR simulation imaging for prone radiotherapy after breast-conserving surgery: assessing its application in lumpectomy cavity delineation based on deformable image registration

Background The application of delayed-enhancement magnetic resonance (DE-MR) simulation imaging in lumpectomy cavity (LC) delineation for prone radiotherapy in patients with an invisible seroma or a low seroma clarity score (SCS) after breast-conserving surgery (BCS) based on deformable image registration (DIR) was assessed. Methods Twenty-six patients who were suitable for radiotherapy in prone positions after BCS were enrolled, and both computed tomography (CT) and DE-MR simulation scans were acquired. The LC delineated based on titanium surgical clips on CT images was denoted as LCCT. The LC delineated based on the signal of cavity boundaries on fat-suppressed T2-weighted imaging (T2WI) and multiphase delayed-enhancement T1-weighted imaging (DE-T1WI), which was performed at 2 min, 5 min and 10 min postinjection, were denoted as LCT2, LC2T1, LC5T1 and LC10T1, respectively. Afterwards, DIR was performed to compare the volumes and locations of the LCs with MIM software. The generalized conformity index (CIgen) of inter (intra) observer (Inter-CIgen and Intra-CIgen) was also used to explore the inter(intra) observer variation for LC delineation on each image modality. Results LCCT–LC10T1 provided the best conformal index (CI) and degree of inclusion (DI), increasing by 2.08% and 4.48% compared to LCCT–LCT2, 11.36% and 2.94% for LCCT–LC2T1, and 8.89% and 7.69% for LC5T1–LCCT, respectively. The center of mass (COM) of LCCT–LC10T1 decreased by 17.86%, 6.12% and 13.21% compared with that of LCCT–LCT2, LCCT–LC2T1 and LCCT–LC5T1, respectively. The agreement of LC delineation was strongest for 10th min DE-TIWI (coefficient of variation, COV = 2.30%, Inter-CIgen = 87.06%, Intra-CIgen = 92.64%). Conclusion For patients with a low SCS (SCS ≤ 2) after BCS, it is feasible to contour the LC based on prone DE-MR simulation images. Furthermore, the LC derived from prone DE-T1WI at 10 min was found to be most similar to that derived from prone CT simulation scans using titanium surgical clips regardless of the volume and location of the LC. Inter (intra) variability was minimal for the delineation of the LC based on 10th min DE-TIWI.