A pilot study to assess the construct and face validity of the Northwestern Objective Microanastomosis Assessment Tool

OBJECT Microsurgical skills remain an integral component of neurosurgical education. There is a need for an objective scale to assess microsurgical skills. The objective of this study was to assess the face and construct validity of a benchtraining microanastomosis module and an objective assessment scale, i.e., the Northwestern Objective Microanastomosis Assessment Tool (NOMAT). METHODS Medical students, neurosurgical residents, and postdoctoral research fellows at Northwestern University were enrolled in the study. Trainees were divided into 3 groups based on microsurgical experience: 1) experienced, 2) exposed, and 3) novices. Each trainee completed two end-to-end microanastomoses using a 1-mm and a 3-mm synthetic vessel. Two cameras were installed to capture procedural footage. One neurosurgeon blindly graded the performance of trainees using both objective and subjective methods to assess construct validity. Two neurosurgeons reviewed the contents of the simulation module to assess face validity. RESULTS Twenty-one trainees participated in the study, including 6 experienced, 6 exposed, and 9 novices. The mean NOMAT score for experienced trainees on the 1-mm module was 47.3/70 compared with 26.0/70 and 25.8/70 for exposed and novice trainees, respectively (p = 0.02). Using subjective grading, experienced trainees performed significantly better on the 1-mm module (64.2/100) compared with exposed or novice trainees (23.3/100 and 25.0/100, respectively; p = 0.02). No statistical difference between groups was noted for the 3-mm module with both NOMAT and subjective grading. Experienced trainees took less time to perform both tasks compared with the others. CONCLUSIONS Face and construct validities of the microanastomosis module were established. The scale and the microanastomosis module could help assess the microsurgical skills of neurosurgical trainees and serve as a basis for the creation of a microsurgical curriculum.

Download Full-text

Validation of a Bench-Top Training Model for Retrograde Intrarenal Surgery

Urologia Internationalis ◽

10.1159/000512883 ◽

2021 ◽

pp. 1-6

Author(s):

Sana Hussain ◽

Roshan-e Shahid Rana ◽

M. Hammad Ather

Keyword(s):

Construct Validity ◽

Content Validity ◽

Concurrent Validity ◽

Training Model ◽

Face Validity ◽

Retrograde Intrarenal Surgery ◽

Learning Technique ◽

The Face ◽

The Mean ◽

Self Administered Questionnaire

Objective: To validate the in-house built Styrofoam box bench-top training model for retrograde intrarenal surgery (RIRS). Materials and Methods: This study was performed in the setting of a half-day RIRS course. During the course, participants performed RIRS on a locally built bench-top model. We recruited 26 participants, comprising 20 trainees and 6 experts. Trainees and experts were asked to fill a self-administered questionnaire assessing various components of RIRS to assess the face and content validity. For construct validity, experts using Objective Structured Assessment of Technical Skills (OSATS) rated trainees and the mean OSATS score of junior versus senior residents was compared. Results: As per trainees, the model was 86% (4.3/5) realistic, which was backed by experts who found this model to be 87.6% (4.38/5) realistic. The overall face validity of the model was 86.4% (4.32/5). The overall content validity of this model was 83.4% (4.17/5). Majority of the participants thought that this tool is useful for learning technique (4.38 ± 0.49) and safe-conduct (4.31 ± 0.73) of the procedure. The trainees concurred that the skills acquired are transferable to the operating room (4.23 ± 0.76). However, the construct validity by comparing mean OSATS score of junior versus senior residents was 19.5 ± 3.5 and 23 ± 4.5. A p value of 0.11 could not be established. Conclusion: This study has demonstrated the face, content, and construct validity of the bench-top training model for RIRS. Further evaluation is necessary to compare its effectiveness against other available models to demonstrate concurrent validity.

Download Full-text

Bimanual Fundamentals: Validation of a New Curriculum for Virtual Reality Training of Laparoscopic Skills

Surgical Innovation ◽

10.1177/1553350620953030 ◽

2020 ◽

pp. 155335062095303

Author(s):

Martijn P. H. van Ginkel ◽

Marlies P. Schijven ◽

Wilhelmina M. U. van Grevenstein ◽

Henk W. R. Schreuder

Keyword(s):

Virtual Reality ◽

Construct Validity ◽

Assessment Tool ◽

Objective Assessment ◽

Great Majority ◽

Face Validity ◽

Psychomotor Skills ◽

Medical Centers ◽

Task Time ◽

And Training

Background. To determine face and construct validity for the new Bimanual Fundamentals curriculum for the Simendo® Virtual Reality Laparoscopy Simulator and prove its efficiency as a training and objective assessment tool for surgical resident’s advanced psychomotor skills. Methods. 49 participants were recruited: 17 medical students (novices), 15 residents (intermediates), and 17 medical specialists (experts) in the field of gynecology, general surgery, and urology in 3 tertiary medical centers in the Netherlands. All participants performed the 5 exercises of the curriculum for 3 consecutive times on the simulator. Intermediates and experts filled in a questionnaire afterward, regarding the reality of the simulator and training goals of each exercise. Statistical analysis of performance was performed between novices, intermediates, and experts. Parameters such as task time, collisions/displacements, and path length right and left were compared between groups. Additionally, a total performance score was calculated for each participant. Results. Face validity scores regarding realism and training goals were overall positive (median scores of 4 on a 5-point Likert scale). Participants felt that the curriculum was a useful addition to the previous curricula and the used simulator would fit in their residency programs. Construct validity results showed significant differences on the great majority of measured parameters between groups. The simulator is able to differentiate between performers with different levels of laparoscopic experience. Conclusions. Face and construct validity for the new Bimanual Fundamental curriculum for the Simendo virtual reality simulator could be established. The curriculum is suitable to use in resident’s training programs to improve and maintain advanced psychomotor skills.

Download Full-text

Evaluation of the Modified Naranjo Criteria for Assessing Causal Attribution of Clinical Outcome to Homeopathic Intervention as Presented in Case Reports

Homeopathy ◽

10.1055/s-0040-1701251 ◽

2020 ◽

Vol 109 (04) ◽

pp. 191-197

Author(s):

Chetna Deep Lamba ◽

Vishwa Kumar Gupta ◽

Robbert van Haselen ◽

Lex Rutten ◽

Nidhi Mahajan ◽

...

Keyword(s):

Clinical Outcome ◽

Causal Relationship ◽

Clinical Case ◽

Assessment Tool ◽

Causal Attribution ◽

Case Reports ◽

Face Validity ◽

Rater Agreement ◽

The Face ◽

Selection Of

Abstract Objectives The objective of this study was to establish the reliability and content validity of the “Modified Naranjo Criteria for Homeopathy—Causal Attribution Inventory” as a tool for attributing a causal relationship between the homeopathic intervention and outcome in clinical case reports. Methods Purposive sampling was adopted for the selection of information-rich case reports using pre-defined criteria. Eligible case reports had to fulfil a minimum of nine items of the CARE Clinical Case Reporting Guideline checklist and a minimum of three of the homeopathic HOM-CASE CARE extension items. The Modified Naranjo Criteria for Homeopathy Inventory consists of 10 domains. Inter-rater agreement in the scoring of these domains was determined by calculating the percentage agreement and kappa (κ) values. A κ greater than 0.4, indicating fair agreement between raters, in conjunction with the absence of concerns regarding the face validity, was taken to indicate the validity of a given domain. Each domain was assessed by four raters for the selected case reports. Results Sixty case reports met the inclusion criteria. Inter-rater agreement/concordance per domain was “perfect” for domains 1 (100%, κ = 1.00) and 2 (100%, κ = 1.00); “almost perfect” for domain 8 (97.5%, κ = 0.86); “substantial” for domains 3 (96.7%, κ = 0.80) and 5 (91.1%, κ = 0.70); “moderate” for domains 4 (83.3%, κ = 0.60), 7 (67.8%, κ = 0.46) and 9 (99.2%, κ = 0.50); and “fair” for domain 10 (56.1%, κ = 0.38). For domains 6A (46.7%, κ = 0.03) and 6B (50.3%, κ = 0.18), there was “slight agreement” only. Thus, the validity of the Modified Naranjo Criteria for Homeopathy tool was established for each of its domains, except for the two that pertain to direction of cure (domains 6A and 6B). Conclusion The Modified Naranjo Criteria for Homeopathy—Causal Attribution Inventory was identified as a valid tool for assessing the likelihood of a causal relationship between a homeopathic intervention and clinical outcome. Improved wordings for several criteria have been proposed for the assessment tool, under the new acronym “MONARCH”. Further assessment of two MONARCH domains is required.

Download Full-text

Psychological Stresses and Abuse of Kindergarten Children among Working and Non- Working Mothers

Journal of Educational and Psychological Studies [JEPS] ◽

10.24200/jeps.vol6iss1pp1-20 ◽

2012 ◽

Vol 6 (1) ◽

pp. 1

Author(s):

Aisheh. H. Tawalbeh ◽

Hasan Al- Zawaydeh

Keyword(s):

Child Abuse ◽

Analysis Of Variance ◽

Working Mothers ◽

Statistical Difference ◽

Kindergarten Children ◽

Face Validity ◽

Future Studies ◽

The Mean ◽

The Difference ◽

Mean Differences

The study goal was to investigate the stress among working and non-working mothers and the abuse of their kindergarten age children. Two instruments were developed to measure the stress with two versions for working and non-working mothers, and a child abuse tool. Face validity and retest reliability were conducted. The stress tool contained 25 Items. The tools were administered to a sample of 400, working (200) and nonworking (200) mothers in Amman. A t-test for independent samples was used to test the difference between means of stress and child abuse for working and non-working mothers. Also, two-way analysis of variance was conducted to find the mean differences of the three abuse dimensions: physical, psychological and neglects for working and non-working mothers. Results showed statistical difference (p < 0.05) between means of stress in favor of working mothers, but the difference between means of abuse was in favor of nonworking mothers. The results also showed statistical significant differences (p < 0.05) in the three domains of physical, psychological and neglect abuse, in favor of the non-working mothers. Implications and recommendations for future studies were suggested.

Download Full-text

TP8.2.9 Development & Evaluation of LapPass™: The Laparoscopic Passport

British Journal of Surgery ◽

10.1093/bjs/znab362.082 ◽

2021 ◽

Vol 108 (Supplement_7) ◽

Author(s):

Michelle Fong ◽

Jenny Treglohan ◽

Chelliah Selvasekar ◽

Peter Sedman ◽

Paul Leeder ◽

...

Keyword(s):

Content Validity ◽

Assessment Tool ◽

Objective Assessment ◽

Pass Rate ◽

Time Pass ◽

Intracorporeal Suturing ◽

Ensure Patient Safety ◽

The Mean ◽

Development Evaluation ◽

Strongly Agree

Abstract Aims Laparoscopic surgery is technically challenging and assessment of competency is necessary to ensure patient safety and guide training. Existing tools of assessment are mostly subjective, with a growing need for objective credentialing. LapPass™ was developed by a UK-based laparoscopic society as an accessible simulation assessment tool. The aim of this study was to report on its development and preliminary findings of usability and validity. Methods LapPass™ consists of 4 tasks that test: bimanual dexterity, simulated appendicectomy, dissection and intracorporeal suturing. Participants were prospectively recruited from testing events. Online surveys were sent to assessors and participants to assess the usability, face and content validity of the tool. Options to respond were on a five-point Likert scale with ratings from strongly disagree (1) to strongly agree (5). Results LapPass was launched and offered to trainees as free-of-charge assessment tool. 31 participants and 12 assessors took part. The 1st time pass rate for bimanual dexterity was 19/29 (65.5%), appendicectomy 13/23 (56.5%), dissection 20/27 (74.1%) and intracorporeal suturing 6/19 (31.5%). The mean scores for participants’ usability and validity were 3.8 and 4.12 for bimanual dexterity; 3.96 and 4.37 for appendicectomy; 4.5 and 4.16 for dissection and 3.84 and 4.52 for intracorporeal suturing. Assessors' mean score of usability was 4.5 across all tasks. Assessors scored validity of bimanual dexterity 4.35, appendicectomy 4.42, dissection 3.71 and intracorporeal suturing 4.65. Conclusion LapPass™ is a an accessible objective assessment tool for laparoscopic basic surgical skills with preliminary data to confirm its usability and face and content validity.

Download Full-text

A high-fidelity, virtual reality, transurethral resection of bladder tumor simulator: Validation as a tool for training

Canadian Urological Association Journal ◽

10.5489/cuaj.7285 ◽

2021 ◽

Vol 16 (4) ◽

Author(s):

Jonathan Moore ◽

Stewart Whalen ◽

Neal Rowe ◽

Jason Lee ◽

Michael Ordon ◽

...

Keyword(s):

Construct Validity ◽

Transurethral Resection ◽

Content Validity ◽

Bladder Tumor ◽

Strong Support ◽

Face Validity ◽

Video Recordings ◽

The Face ◽

Hands And Feet ◽

Key Steps

Introduction: Simulation-based training is used to help trainees learn surgical procedures in a safe environment. The objective of our study was to test the face, content, and construct validity of the transurethral resection of bladder tumor (TURBT) module built on the Simbionix TURP Mentor simulator. Methods: Participants performed five standardized cases on the simulator. Domains of the simulator were evaluated on a five-point Likert scale to establish face and content validity. Construct validity was assessed through the simulator's built-in scoring metrics, as well as video recordings of the simulator screen and an anonymized view of participants' hands and feet, which were evaluated using an objective structured assessment of technical skills (OSATS) tool. Results: Ten experienced operators and 15 novices participated. Face validity was somewhat acceptable (mean realism 3.8/5±1.03 standard deviation [SD]; mean appearance 4.1/5±0.57), as was content validity, represented by simulation of key steps (mean 3.9±0.57). The simulator failed to achieve construct validity. There was no difference in mean simulator scores or OSATS scoring between experienced operators and novices. Novices significantly improved their mean simulator scores (305.9 vs. 332.4, p=0.006) and OSATS scoring (15.8 vs. 18.1, p=0.001), while 87% felt their confidence to perform TURBT improved. Overall, 92% of participants agreed that the simulator should be incorporated into residency training. Conclusions: Our study suggests a role for the TURBT module of the Simbionix TURP Mentor simulator as an introduction to TURBT for urology trainees. Strong support was found from both experienced operators and novices for its formal inclusion in resident education.

Download Full-text

"These Questions Have Everything That Happens to me": Analysis of a Femicide Risk Assessment Tool for Abused Women in Brazil

Journal of Family Violence ◽

10.1007/s10896-021-00313-1 ◽

2021 ◽

Author(s):

Dabney P. Evans ◽

Casey D. Xavier Hall ◽

Raiza Wallace Guimarães da Rocha ◽

Sandra Marques Prado ◽

Marcos C. Signorelli

Keyword(s):

Risk Assessment ◽

Assessment Tool ◽

Face Validity ◽

Risk Assessment Tool ◽

Abused Women ◽

Healthcare Settings ◽

The Face ◽

One Stop ◽

Valid Assessment ◽

High Degree

Abstract Purpose The purpose of this mixed-methods triangulation study was to assess the face validity and comprehension of a femicide risk assessment tool, the Danger Assessment-Brazil (DA-Brazil) among women seeking care in a one stop center for abused women in Curitiba, Brazil. Our secondary aim was to assess professionals' perceptions of feasibility for using the DA-Brazil in the same setting. Method Fifty-five women experiencing relationship violence completed the instrument and participated in cognitive interviews about their experience; professionals attending survivors were also interviewed. Results The vast majority of women described the DA-Brazil instrument as being easy to comprehend (n = 41, 73.2%). Nearly half of participants (n = 26, 46.4%) had some kind of question regarding the DA-Brazil calendar, a tool to visualize abuse frequency and severity. Queries aligned with five categories: recollection of dates, scale, relationship status, terminology, and discomfort. Professionals reported that the DA-Brazil instrument would support referral decision-making. Conclusion The overall face validity and comprehension of the DA-Brazil appears to be high. The majority of challenges were around the calendar activity. Professional perceptions of the DA-Brazil suggest a high degree of feasibility for its use in Brazilian healthcare settings. In order for the DA-Brazil to effectively be administered with facilitated support there is a need for training on the best use of the instrument. Accurate assessment of femicide risk is critical in a country like Brazil with high rates of femicide. The DA-Brazil provides a valid assessment of femicide risk and has the potential to trigger early intervention for those at risk.

Download Full-text

Face and construct validity of TU-Delft epidural simulator and the value of real-time visualization

Regional Anesthesia and Pain Medicine ◽

10.1136/rapm-2018-100161 ◽

2019 ◽

Vol 44 (3) ◽

pp. 298-302 ◽

Cited By ~ 2

Author(s):

Nenad Zivkovic ◽

Ganapathy van Samkar ◽

Henning Hermanns ◽

Philipp Lirk ◽

Markus W Hollmann ◽

...

Keyword(s):

Construct Validity ◽

Motor Skills ◽

Face Validity ◽

Epidural Needle ◽

Procedure Time ◽

Total Procedure Time ◽

Prior Use ◽

The Face ◽

Real Time Visualization ◽

Improved Performance

Background and objectivesLearning epidural anesthesia traditionally involves bedside teaching. Visualization aids or a simulator can help in acquiring motor skills, increasing patient safety and steepening the learning curve. We evaluated the face and construct validity of the TU-Delft Epidural Simulator and the effect of needle visualization.MethodsSixty-eight anesthesiologists, anesthesia residents, and final-year medical students tested the epidural simulator. Participants performed six epidural simulations with and six without needle visualization. We tested face validity on a Likert scale questionnaire. We collected data with the simulator software (spinal taps, dura contacts, bone contacts, attempts, and time) and tested for correlation with the performer’s experience (construct validity). A visualization aid was tested in a randomized crossover design.ResultsFace validity as rated by the participants was above average, with a mean of 3.7 (2.0–4.8) on a 5-point scale. Construct validity was indicated by significantly more spinal taps (0.4 [0–4) vs 0.07 [0–2], p=0.04) and more dura contacts (0.58 [0–6] vs 0.37 [0–3], p=0.002) by the inexperienced group compared with the expert group. The visualization aid improved performance by reducing the number of bone contacts and the number of attempts, and by decreasing the procedure time. Prior visualization training reduced the total procedure time from 279 s (69–574) to 180 s (53–605) (p=0.01) for the “blind” procedure.ConclusionsThe TU-Delft Epidural Simulator is a useful tool for teaching motor skills during epidural needle placement. Prior use of a visualization tool improves performance even without visual support during consequent simulations.

Download Full-text

How to enhance and assess reflection in specialist training: a mixed method validation study of a new tool for global assessment of reflection ability

BMC Medical Education ◽

10.1186/s12909-020-02256-5 ◽

2020 ◽

Vol 20 (1) ◽

Author(s):

Gunver Lillevang ◽

Helle Ibsen ◽

Søren Hast Prins ◽

Niels Kristian Kjaer

Keyword(s):

Construct Validity ◽

Method Validation ◽

Validation Study ◽

Mixed Method ◽

Global Assessment ◽

Assessment Tool ◽

Face Validity ◽

Train The Trainer ◽

Specific Complex ◽

Mind Maps

Abstract Background In Danish GP training we had the ambition to enhance and assess global reflection ability, but since we found no appropriate validated method in the literature, we decided to develop a new assessment tool. This tool is based on individual trainee developed mind maps and structured trainer-trainee discussions related to specific complex competencies. We named the tool Global Assessment of Reflection ability (GAR) and conducted a mixed method validation study. Our goal was to investigate whether it is possible to enhance and assess reflection ability using the tool. Methods In order to investigate acceptability, feasibility, face validity, and construct validity of the tool we conducted a mixed method validation study that combined 1) qualitative data obtained from 750 GP trainers participating in train-the-trainer courses, 2) a questionnaire survey sent to 349 GP trainers and 214 GP trainees and 3) a thorough analysis of eight trainer-trainee discussions. Results Our study showed an immediate high acceptance of the GAR tool. Both trainers and trainees found the tool feasible, useful, and relevant with acceptable face validity. Rating of eight audio recordings showed that the tool can demonstrate reflection during assessment of complex competencies. Conclusions We have developed an assessment tool (GAR) to enhance and assess reflection. GAR was found to be acceptable, feasible, relevant and with good face- and construct validity. GAR seems to be able to enhance the trainees’ ability to reflect and provide a good basis for assessment in relation to complex competencies.

Download Full-text

The Development of the Text Evaluation Scale for Child Rights: A Study of Validity and Reliability

World Journal of Education ◽

10.5430/wje.v8n5p139 ◽

2018 ◽

Vol 8 (5) ◽

pp. 139

Author(s):

Yusuf Söylemez

Keyword(s):

Construct Validity ◽

Content Validity ◽

Sample Selection ◽

Factor Loading ◽

Face Validity ◽

Main Study ◽

Validity And Reliability ◽

Child Rights ◽

Cronbach Alpha ◽

The Face

This study was conducted with the aim of developing The Text Evaluation Scale for Child Rights. There are fourdifferent sample chosen for the face validity, content validity and construct validity (for pilot scheme and main study)of the study. For face validity, a sample group of 3 experts chosen with the method of purposeful sampling includingthe researcher was formed. For content validity, snowball method was determined and studied with 12 experts. Forthe study of construct validity, random sampling method was performed for the sample selection in pilot studyinvolving 120 people and in main study involving 510 people.The theoretical framework of the scale was determined by means of the attempts, conventions, studies regarding thesubject basing upon the United Nations Organization Child Rights Convention and additional protocols to thisconvention. One could get minimum 40 and maximum 200 points from the five-point Likert scale which consists of28 positive and 12 negative (total 40) items. The scale has two sub-dimensions which are content and author. Thefact that Cronbach Alpha reliability coefficient is high regarding the sub-dimensions of the scale (authorsub-dimension= 0,822 content sub-dimension= 0,834) shows that the items in the sub-dimensions are consistent withone another. Cronbach Alpha value for the whole of the scale was determined as 0,90 which means that the scale ishighly reliable. Besides, in the scale, there are items of which factor loading value is higher than 0,45. When validityand reliability results are examined, it can be seen that the scale could be utilized to evaluate the text in terms ofbeing suitable for the child rights.

Download Full-text