Ophthalmic statistics note 13: method agreement studies in ophthalmology—please don’t carry on correlating…

PurposeDue to the paucity of research on web-based job applicant screening (i.e. cybervetting), the purpose of the current study was to examine the psychometric properties of cybervetting, including an examination of the impact of adding structure to the rating process.Design/methodology/approachUsing a mixed-factorial design, 122 supervisors conducted cybervetting evaluations of applicant personality, cognitive ability, written communication skills, professionalism, and overall suitability. Cross-method agreement (i.e. the degree of similarity between cybervetting ratings and other assessment methods), as well as interrater reliability and agreement were examined, and unstructured versus structured cybervetting rating formats were compared.FindingsCybervetting assessments demonstrated high interrater reliability and interrater agreement, but only limited evidence of cross-method agreement was provided. In addition, adding structure to the cybervetting process did not enhance the psychometric properties of this assessment technique.Practical implicationsThis study highlighted that whereas cybervetting raters demonstrated a high degree of consensus in cybervetting-based attributions, there may be concerns regarding assessment accuracy, as cybervetting-based ratings generally differed from applicant test scores and self-assessment ratings. Thus, employers should use caution when utilizing this pre-employment screening technique.Originality/valueWhereas previous research has suggested that cybervetting ratings demonstrate convergence with other traditional assessments (albeit with relatively small effects), these correlational links do not provide information regarding cross-method agreement or method interchangeability. Thus, this study bridges a crucial gap in the literature by examining cross-method agreement for a variety of job-relevant constructs, as well as empirically testing the impact of adding structure to the cybervetting rating process.

Download Full-text

CHROMagar Salmonella Detection Test Kit

Journal of AOAC International ◽

10.1093/jaoac/92.6.1906 ◽

2009 ◽

Vol 92 (6) ◽

pp. 1906-1909 ◽

Cited By ~ 3

Author(s):

Katana Webb ◽

Vicki Ritter ◽

Thomas Hammack

Keyword(s):

Food Environment ◽

Room Temperature ◽

Peanut Butter ◽

Reference Method ◽

Testing Laboratory ◽

Inoculum Level ◽

Test Kit ◽

Uninoculated Control ◽

Positive Results ◽

Method Agreement

Abstract BBL CHROMagar Salmonella was evaluated by an external food testing laboratory for the recovery of Salmonella in peanut butter using the U.S. Food and Drug Administration's Bacteriological Analytical Manual (FDA-BAM) procedure. The peanut butter was found to be negative for the presence of Salmonella and, therefore, was seeded with heat-stressed Salmonella at target concentrations of 0.2 and 2 CFU/25 g. The Salmonella-seeded samples remained at room temperature for 14 days before analysis to stabilize the Salmonella in the food environment. Twenty 25 g test portions from each seeded level and five 25 g samples of uninoculated control samples were processed using enrichment broths as outlined in the FDA-BAM procedure. BBL CHROMagar Salmonella-prepared plates were evaluated with the FDA reference method media (bismuth sulfite, xylose lysine desoxycholate, and Hektoen enteric agars). Fractionally positive results were obtained from the lower inoculum level of peanut butter samples. Five positive cultures were recovered from both the BBL CHROMagar Salmonella and reference methods. The two methods gave identical results for all cultures resulting in a method agreement of 100%. McNemar's 2 test, which assesses the evidence for difference in marginal proportions between two methods, could not be evaluated because it requires one or more discrepant cultures. However, because there were no discrepant cultures, the marginal proportions for the two methods were identical; therefore, there is no evidence of a difference between the methods. This study demonstrates that the results from BBL CHROMagar Salmonella are comparable to the three reference method media for the detection of Salmonella in peanut butter using the FDA-BAM procedures.

Download Full-text

Assessment of DCE Utility for PCa Diagnosis Using PI-RADS v2.1: Effects on Diagnostic Accuracy and Reproducibility

Diagnostics ◽

10.3390/diagnostics10030164 ◽

2020 ◽

Vol 10 (3) ◽

pp. 164 ◽

Cited By ~ 2

Author(s):

Valentina Brancato ◽

Giuseppe Di Costanzo ◽

Luca Basso ◽

Liberatore Tramontano ◽

Marta Puglia ◽

...

Keyword(s):

Diagnostic Accuracy ◽

Multiparametric Mri ◽

Added Value ◽

Operating Characteristics ◽

Dce Mri ◽

Contrast Enhanced ◽

Mri Protocol ◽

Controversial Topic ◽

Clinically Significant ◽

Method Agreement

The role of dynamic contrast-enhanced-MRI (DCE-MRI) for Prostate Imaging-Reporting and Data System (PI-RADS) scoring is a controversial topic. In this retrospective study, we aimed to measure the added value of DCE-MRI in combination with T2-weighted (T2W) and diffusion-weighted imaging (DWI) using PI-RADS v2.1, in terms of reproducibility and diagnostic accuracy, for detection of prostate cancer (PCa) and clinically significant PCa (CS-PCa, for Gleason Score ≥ 7). 117 lesions in 111 patients were identified as suspicion by multiparametric MRI (mpMRI) and addressed for biopsy. Three experienced readers independently assessed PI-RADS score, first using biparametric MRI (bpMRI, including DWI and T2W), and then multiparametric MRI (also including DCE). The inter-rater and inter-method agreement (bpMRI- vs. mpMRI-based scores) were assessed by Cohen’s kappa (κ). Receiver operating characteristics (ROC) analysis was performed to evaluate the diagnostic accuracy for PCa and CS-PCa detection among the two scores. Inter-rater agreement was excellent for the three pairs of readers (κ ≥ 0.83), while the inter-method agreement was good (κ ≥ 0.73). Areas under the ROC curve (AUC) showed similar high-values (0.8 ≤ AUC ≤ 0.85). The reproducibility of PI-RADS v2.1 scoring was comparable and high among readers, without relevant differences, depending on the MRI protocol used. The inclusion of DCE did not influence the diagnostic accuracy.

Download Full-text