Abstract
Study question
Given the subjectivity of blastocyst grading and the challenge of performing routine competency assessments, how consistently do embryologists grade blastocysts when using an easily accessible phone application?
Summary answer
Grading agreement was fair to moderate for inner cell mass (ICM) and trophectoderm (TE), evidence that a mobile application can be used for quality assurance.
What is known already
Embryologists routinely perform external quality assessments (EQA), though the utility of EQA for quality improvement is limited, and more active, user-friendly tools are needed to improve quality assurance in embryology. Blastocyst grading is one of the most important and subjective tasks in clinical embryology, important for both blastocyst ranking and decision to freeze. Inter-user agreement is only fair for ICM (kappa 0.349) and TE grade (kappa 0.397; Storr et al., 2017). Fair agreement has also been reported for decision to freeze for a cohort of blastocysts that exhibited borderline morphology (kappa 0.301; Hammond et al., 2020).
Study design, size, duration
A prospective study of blastocyst grading consistency using ARTCompass, a mobile phone application designed to assess clinical decision making of laboratory staff for andrology and embryology competency. Two assessments, each with 100 images of expanded blastocysts in three planes, were performed by 42 embryologists from 9 clinics in 2 countries between April to July 2020. Survey 1 assessed ICM grading and survey 2 assessed TE grading using the same set of images for consistency.
Participants/materials, setting, methods
Blastocysts were of proportionally mixed grades (ranging from grade A to X) using a modified Gardner system that included “X” for non-viable ICM/TE. Embryologists were advised to complete the tests individually in one sitting. The Fleiss kappa coefficient (k) measured inter-rater agreement among embryologists when assigning blastocyst grade. Kappa value interpretation is as follows <0.20: poor; 0.21–0.40: fair; 0.41–0.60: moderate; 0.61–0.80: good and 0.81–1.00: very good.
Main results and the role of chance
Overall, agreement for ICM and TE grades was moderate among embryologists (kappa 0.47, 0.52, respectively). ICM grade B and C had the lowest agreement (0.37, 0.39), while ICM grade X (no, or degenerate ICM), and TE grade A showed the highest agreement (0.68, 0.62). These results illustrate that embryologists had difficultly classifying ICM grade when it was of moderate to poor quality (grade b or c), likely due to subjectivity in grading size and compaction level, but were good at classifying ICM grade when there was no apparent ICM (grade x). For TE grade, embryologists consistently identified a top-quality TE (grade a), which is reassuring as TE grade is the primary morphological feature used for blastocyst ranking. In general, this QA platform offers ease of use and shows agreement values for ICM and TE that are similar to other studies, suggesting that blastocyst grading with a mobile phone application is a viable option for quality assurance.
Limitations, reasons for caution
Only blastocyst grading was assessed, therefore additional competency assessments using a mobile device should be assessed for accuracy. Further studies are needed to determine if mobile applications can improve competency.
Wider implications of the findings: Ease of use by 42 embryologists indicates mobile applications may provide a user-friendly and accessible platform for QA. Since effective and efficient assessment of competency and KPIs is an ongoing challenge for laboratories, a mobile application is a novel and effective tool to monitor QA parameters in the IVF laboratory
Trial registration number
Not applicable
Quality assurance for mobile applications