TP8.2.9 Development & Evaluation of LapPass™: The Laparoscopic Passport

Abstract Aims Laparoscopic surgery is technically challenging and assessment of competency is necessary to ensure patient safety and guide training. Existing tools of assessment are mostly subjective, with a growing need for objective credentialing. LapPass™ was developed by a UK-based laparoscopic society as an accessible simulation assessment tool. The aim of this study was to report on its development and preliminary findings of usability and validity. Methods LapPass™ consists of 4 tasks that test: bimanual dexterity, simulated appendicectomy, dissection and intracorporeal suturing. Participants were prospectively recruited from testing events. Online surveys were sent to assessors and participants to assess the usability, face and content validity of the tool. Options to respond were on a five-point Likert scale with ratings from strongly disagree (1) to strongly agree (5). Results LapPass was launched and offered to trainees as free-of-charge assessment tool. 31 participants and 12 assessors took part. The 1st time pass rate for bimanual dexterity was 19/29 (65.5%), appendicectomy 13/23 (56.5%), dissection 20/27 (74.1%) and intracorporeal suturing 6/19 (31.5%). The mean scores for participants’ usability and validity were 3.8 and 4.12 for bimanual dexterity; 3.96 and 4.37 for appendicectomy; 4.5 and 4.16 for dissection and 3.84 and 4.52 for intracorporeal suturing. Assessors' mean score of usability was 4.5 across all tasks. Assessors scored validity of bimanual dexterity 4.35, appendicectomy 4.42, dissection 3.71 and intracorporeal suturing 4.65. Conclusion LapPass™ is a an accessible objective assessment tool for laparoscopic basic surgical skills with preliminary data to confirm its usability and face and content validity.

Download Full-text

Objective structured practical examination (OSPE) as a tool in formative assessment of II MBBS students, in pathology

International Journal of Research in Medical Sciences ◽

10.18203/2320-6012.ijrms20175723 ◽

2017 ◽

Vol 6 (1) ◽

pp. 221

Author(s):

Ranga Rao D. ◽

Parameswari Babu U. ◽

Kalyan Chakravarthy C. V. ◽

Ramya N.

Keyword(s):

Assessment Tool ◽

Objective Assessment ◽

Cross Sectional Study ◽

Faculty Members ◽

Difficulty Level ◽

Medical Sciences ◽

Cross Sectional ◽

Internal Assessment ◽

Practical Examination ◽

Strongly Agree

Background: Assessment drives the student learning. Regular periodical assessment not only improves learning habits, but also enhances the competence in all levels of medical education. Traditional practical examination is more subjective. It depends on examiners subjectivity, varying difficulty level of various experiments, and also internal marks variation etc. These flaws can be overcome by newer methods like OSPE. The aim of the study was to implement OSPE as a tool of internal assessment for practical skills in the II MBBS. To compare this with traditional practical examination (TPE). To obtain the students and faculty feedback regarding OSPE as a tool of assessment.Methods: A cross sectional study was carried out for 158 students in II internal pathology practical examination for six days in the second week of September 2016 at Department of Pathology, Dr. Pinnamaneni Siddhartha institute of medical sciences & Research Foundation, Chinnaoutpalli. Faculty and students were sensitized; blueprint were used to arrange twenty OSPE stations for the exercises conducted as per TPE and for the same 25 marks as per TPE. Simultaneously, all the students were subjected to both TPE and OSPE at the same time and venue. TPE was assessed by two professors and OSPE by separate eight faculty members independently without interaction with the students. The procedural stations were evaluated by using checklist and the response stations which consisted of short answers and MCQs, facilitated correction. Feedback was given to the student on their performance and feedback was obtained from the students and faculty regarding OSPE by questionnaire with Yes/No answers.Results: Performance score of students in OPSE (13.73 ±2.49) was higher as compared to TPE (9.27±1.86) which was statistically significant. Based on the response to the questionnaire, students perception towards OSPE was analyzed. Majority strongly agree OSPE to be fairer, more transparent and objective in comparison to TPE. In contrast, all the faculty members unanimously opined that OSPE was difficult to arrange, time taken and faculty versus students ratio was high for evaluation. Though, the faculty (91%) overall opined that OSPE should be included as a method of assessment.Conclusions: Present study revealed that OSPE was acceptable, feasible and reliable to the students as well as for faculty for the internal assessment in pathology. Opinions of both students and faculties strongly agreed that OPSE is more effective objective assessment tool.

Download Full-text

Content validity of the Geriatric Health Assessment Instrument

Einstein (São Paulo) ◽

10.1590/s1679-45082016ao3455 ◽

2016 ◽

Vol 14 (2) ◽

pp. 158-177 ◽

Cited By ~ 5

Author(s):

Rhaine Borges Santos Pedreira ◽

Saulo Vasconcelos Rocha ◽

Clarice Alves dos Santos ◽

Lélia Renata Carneiro Vasconcelos ◽

Martha Cerqueira Reis

Keyword(s):

Content Validity ◽

Assessment Tool ◽

Health Assessment ◽

The Elderly ◽

Assessment Instrument ◽

Rater Agreement ◽

Validity Index ◽

Content Validity Index ◽

Percent Agreement ◽

The Mean

ABSTRACT Objective Assess the content validity of the Elderly Health Assessment Tool with low education. Methods The data collection instrument/questionnaire was prepared and submitted to an expert panel comprising four healthcare professionals experienced in research on epidemiology of aging. The experts were allowed to suggest item inclusion/exclusion and were asked to rate the ability of individual items in questionnaire blocks to encompass target dimensions as “not valid”, “somewhat valid” or “valid”, using an interval scale. Percent agreement and the Content Validity Index were used as measurements of inter-rater agreement; the minimum acceptable inter-rater agreement was set at 80%. Results The mean instrument percent agreement rate was 86%, ranging from 63 to 99%, and from 50 to 100% between and within blocks respectively. The Mean Content Validity Index score was 93.47%, ranging from 50 to 100% between individual items. Conclusion The instrument showed acceptable psychometric properties for application in geriatric populations with low levels of education. It enabled identifying diseases and assisted in choice of strategies related to health of the elderly.

Download Full-text

Global Evaluative Assessment of Robotic Skills in Endoscopy (GEARS-E): objective assessment tool for master and slave transluminal endoscopic robot

Endoscopy International Open ◽

10.1055/a-0640-3123 ◽

2018 ◽

Vol 06 (08) ◽

pp. E1065-E1069 ◽

Cited By ~ 3

Author(s):

Nobuyoshi Takeshita ◽

Soo Jay Phee ◽

Philip WaiYan Chiu ◽

Khek Yu Ho

Keyword(s):

Degrees Of Freedom ◽

Assessment Tool ◽

Ex Vivo ◽

Objective Assessment ◽

Global Rating ◽

Maximum Score ◽

Tissue Handling ◽

Robotic Skills ◽

The Mean

Abstract Background and study aims The Master and Slave Transluminal Endoscopic Robot (MASTER) is a novel robotic endosurgical system with two operating arms that offer multiple degrees of freedom. We developed a new assessment tool, the Global Evaluative Assessment of Robotic Skills in Endoscopy (GEARS-E), derived from existing tools in laparoscopic and robotic surgery, and evaluated its utility in the performance of procedures using MASTER. Methods This was a pilot study conducted in vivo and ex vivo on animals. Three operators (Novice-1, Novice-2 and Expert) performed a total of five endoscopic submucosal dissections (ESD) using MASTER. Novice operators had no MASTER experience and the expert had previously performed eight MASTER ESDs. Operator performance was assessed by four independent evaluators using GEARS-E, which has a maximum score of 25 for five domains representing various skill-related variables (depth perception, bimanual dexterity, efficiency, tissue handling and autonomy). Results The mean global rating scores for Novice-1 first attempt, Novice-1 second attempt, Novice-2 first attempt, Novice-2 second attempt and Expertʼs cases were 13.0, 16.0, 13.3, 15.5, and 21.5, respectively. The mean scores of each of the five domains were statistically higher for the second attempts compared to the first attempts for both Novice-1 and Novice-2. The mean scores of each of the five domains for the Expertʼs case were consistently higher than those for the two novice operators in both their first and second attempts. Conclusion Results using GEARS-E showed correlations between surgical experience and MASTER ESD. As an assessment tool for evaluation of surgical skills, GEARS-E has great potential for application in MASTER procedures.

Download Full-text

Development of Articulation Screening Tool to Assess the Articulation Errors in Urdu Speaking School Going Children

UMT Education Review ◽

10.32350/uer.21.02 ◽

2019 ◽

Vol 02 (01) ◽

pp. 23-37

Author(s):

Hamda Muzaffar ◽

Nayab Iftikhar ◽

Muhammad Sikander Ghayas Khan ◽

Sumera Nawaz Malik

Keyword(s):

Pilot Study ◽

Literature Review ◽

Content Validity ◽

Screening Tool ◽

National Language ◽

Test Results ◽

Single Word ◽

The Mean ◽

Intervention Planning ◽

Strongly Agree

Urdu is the national language of Pakistan and more than 100 million people speak Urdu worldwide. Urdu has 43 phonemes, however, there are some distinct phonemes in English and Urdu languages that affect the accuracy of test results. Ultimately, it leads to difficulty in assessment, diagnosis and intervention planning. This study is aimed to develop a criterion based articulation screening tool to identify articulation errors in Urdu speaking school going children. For this purpose, 500 Urdu words were selected after literature review for eliciting spontaneous single word responses at initial, medial and final positions. 10 professionals were requested to review and rate each word on a scale ranging from 1 – 5 (1=strongly disagree, 2= disagree, 3= neutral, 4=agree, 5= strongly agree). Pilot study was conducted on 25 Urdu speaking school going children to check content validity. The mean score of all the responses was calculated and the top three responses were selected for further application. In the second application, 200 school going children including 100 boys and 100 girls were included. Their responses comprising sound utterances at initial, medial and final positions were converted to mean scores. Mean scores of expert opinion, pilot study and final application of UAST were calculated using Excel spreadsheets. The minimum mean score of selected words was 3, whereas the maximum mean score was 5. Final application proves the validity of UAST. The minimum mean score of selected words was 4.5, whereas the maximum mean score was 5. The study proved that UAST is a valid screening tool to screen articulation errors in Urdu speaking school going children.

Download Full-text

A pilot study to assess the construct and face validity of the Northwestern Objective Microanastomosis Assessment Tool

Journal of Neurosurgery ◽

10.3171/2014.12.jns131814 ◽

2015 ◽

Vol 123 (1) ◽

pp. 103-109 ◽

Cited By ~ 14

Author(s):

Salah G. Aoun ◽

Tarek Y. El Ahmadieh ◽

Najib E. El Tecle ◽

Marc R. Daou ◽

Joseph G. Adel ◽

...

Keyword(s):

Construct Validity ◽

Statistical Difference ◽

Assessment Tool ◽

Objective Assessment ◽

Face Validity ◽

Northwestern University ◽

The Face ◽

Postdoctoral Research ◽

The Mean ◽

Neurosurgical Education

OBJECT Microsurgical skills remain an integral component of neurosurgical education. There is a need for an objective scale to assess microsurgical skills. The objective of this study was to assess the face and construct validity of a benchtraining microanastomosis module and an objective assessment scale, i.e., the Northwestern Objective Microanastomosis Assessment Tool (NOMAT). METHODS Medical students, neurosurgical residents, and postdoctoral research fellows at Northwestern University were enrolled in the study. Trainees were divided into 3 groups based on microsurgical experience: 1) experienced, 2) exposed, and 3) novices. Each trainee completed two end-to-end microanastomoses using a 1-mm and a 3-mm synthetic vessel. Two cameras were installed to capture procedural footage. One neurosurgeon blindly graded the performance of trainees using both objective and subjective methods to assess construct validity. Two neurosurgeons reviewed the contents of the simulation module to assess face validity. RESULTS Twenty-one trainees participated in the study, including 6 experienced, 6 exposed, and 9 novices. The mean NOMAT score for experienced trainees on the 1-mm module was 47.3/70 compared with 26.0/70 and 25.8/70 for exposed and novice trainees, respectively (p = 0.02). Using subjective grading, experienced trainees performed significantly better on the 1-mm module (64.2/100) compared with exposed or novice trainees (23.3/100 and 25.0/100, respectively; p = 0.02). No statistical difference between groups was noted for the 3-mm module with both NOMAT and subjective grading. Experienced trainees took less time to perform both tasks compared with the others. CONCLUSIONS Face and construct validities of the microanastomosis module were established. The scale and the microanastomosis module could help assess the microsurgical skills of neurosurgical trainees and serve as a basis for the creation of a microsurgical curriculum.

Download Full-text

Chinesisation, adaptation and validation of the Chelsea Critical Care Physical Assessment Tool in critically ill patients: a cross-sectional observational study

BMJ Open ◽

10.1136/bmjopen-2020-045550 ◽

2021 ◽

Vol 11 (4) ◽

pp. e045550

Author(s):

Zhigang Zhang ◽

Guoqiang Wang ◽

Yuchen Wu ◽

Jin Guo ◽

Nannan Ding ◽

...

Keyword(s):

Critical Care ◽

Observational Study ◽

Critically Ill ◽

Content Validity ◽

Assessment Tool ◽

Western China ◽

Validity Index ◽

Cross Sectional ◽

Physical Assessment ◽

Content Validity Index

PurposeTo translate and adapt the Chelsea Critical Care Physical Assessment Tool (CPAx) into Chinese version (‘CPAx-Chi’), test the reliability and validity of CPAx-Chi, and verify the cut-off point for the diagnosis of intensive care unit-acquired weakness (ICU-AW).Study designCross-sectional observational study.MethodsForward and back translation, cross-cultural adaptation and pretesting of CPAx into CPAx-Chi were based on the Brislin model. Participants were recruited from the general ICU of five third-grade class-A hospitals in western China. Two hundred critically ill adult patients (median age: 53 years; 64% men) with duration of ICU stay ≥48 hours and Glasgow Coma Scale ≥11 were included in this study. Two researchers simultaneously and independently assessed eligible patients using the Medical Research Council Muscle Score (MRC-Score) and CPAx-Chi.ResultsThe content validity index of items was 0.889. The content validity index of scale was 0.955. Taking the MRC-Score scale as standard, the criterion validity of CPAx-Chi was r=0.758 (p<0.001) for researcher A, and r=0.65 (p<0.001) for researcher B. Cronbach’s α was 0.939. The inter-rater reliability was 0.902 (p<0.001). The area under the receiver operating characteristic curves of CPAx-Chi for diagnosing ICU-AW based on MRC-Score ≤48 were 0.899 (95% CI 0.862 to 1.025) and 0.874 (95% CI 0.824 to 0.925) for researcher B. The best cut-off point for CPAx-Chi for the diagnosis of ICU-AW was 31.5. The sensitivity was 87% and specificity was 77% for researcher A, whereas it was 0.621, 31.5, 75% and 87% for researcher B, respectively. The consistency was high when taking CPAx-Chi ≤31 and MRC-Score ≤48 as the cut-off points for the diagnosis of ICU-AW. Cohen’s kappa=0.845 (p=0.02) in researcher A and 0.839 (p=0.04) for researcher B.ConclusionsCPAx-Chi demonstrated content validity, criterion-related validity and reliability. CPAx-Chi showed the best accuracy in assessment of patients at risk of ICU-AW with good sensitivity and specificity at a recommended cut-off of 31.

Download Full-text

Do we need to improve the reporting of evidence in tendinopathy management? A critical appraisal of systematic reviews with recommendations on strength of evidence assessment

BMJ Open Sport & Exercise Medicine ◽

10.1136/bmjsem-2020-000920 ◽

2021 ◽

Vol 7 (1) ◽

pp. e000920

Author(s):

Dimitris Challoumas ◽

Neal L Millar

Keyword(s):

Systematic Reviews ◽

Assessment Tool ◽

Data Extraction ◽

Cochrane Collaboration ◽

Mean Difference ◽

Strength Of Evidence ◽

The Mean ◽

Randomised Controlled ◽

Meta Analyses ◽

The Cochrane Collaboration

ObjectiveTo critically appraise the quality of published systematic reviews (SRs) of randomised controlled trials (RCTs) in tendinopathy with regard to handling and reporting of results with special emphasis on strength of evidence assessment.Data sourcesMedline from inception to June 2020.Study eligibilityAll SRs of RCTs assessing the effectiveness of any intervention(s) on any location of tendinopathy.Data extraction and synthesisIncluded SRs were appraised with the use of a 12-item tool devised by the authors arising from the Preferred Reporting Items in Systematic Reviews and Meta-Analyses statement and other relevant guidance. Subgroup analyses were performed based on impact factor (IF) of publishing journals and date of publication.ResultsA total of 57 SRs were included published in 38 journals between 2006 and 2020. The most commonly used risk-of-bias (RoB) assessment tool and strength of evidence assessment tool were the Cochrane Collaboration RoB tool and the Cochrane Collaboration Back Review Group tool, respectively. The mean score on the appraisal tool was 46.5% (range 0%–100%). SRs published in higher IF journals (>4.7) were associated with a higher mean score than those in lower IF journals (mean difference 26.4%±8.8%, p=0.004). The mean score of the 10 most recently published SRs was similar to that of the first 10 published SRs (mean difference 8.3%±13.7%, p=0.54). Only 23 SRs (40%) used the results of their RoB assessment in data synthesis and more than half (n=30; 50%) did not assess the strength of evidence of their results. Only 12 SRs (21%) assessed their strength of evidence appropriately.ConclusionsIn light of the poor presentation of evidence identified by our review, we provide recommendations to increase transparency and reproducibility in future SRs.

Download Full-text

Validity and Internal Consistency of the Preschool-FLAT, a New Tool for the Assessment of Food Literacy in Young Children from the Training-To-Health Project

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph17082759 ◽

2020 ◽

Vol 17 (8) ◽

pp. 2759

Author(s):

Garden Tabacchi ◽

Giuseppe Battaglia ◽

Giuseppe Messina ◽

Antonio Paoli ◽

Antonio Palma ◽

...

Keyword(s):

Internal Consistency ◽

Content Validity ◽

Structural Equation ◽

Discriminant Validity ◽

Assessment Tool ◽

Intervention Group ◽

Equation Modeling ◽

Control Group ◽

Food Literacy ◽

Health Project

Background: The importance of assessing “food literacy” since youth has been highlighted and, to this purpose, valid and consistent instruments are needed. This study aimed to assess the validity and internal consistency of the preschool-FLAT (Food Literacy Assessment Tool). Methods. 505 children from 21 kindergartens, recruited within the Training-to-Health Project in Palermo (Italy), underwent oral sessions and activities on food-related aspects. Their knowledge/skills were recorded in the preschool-FLAT. The following scale measures were assessed: Content validity; internal consistency (Chronbach’s alpha coefficients); construct validity (Structural Equation Modeling—SEM); discriminant validity (intervention subgroup of 100 children vs. control group of 27 children). Results. Acceptable content validity of a 16-items scale and overall adequate internal consistency were revealed: Content validity index (CVI) 0.94, content validity ratio (CVR) 0.88, Chronbach’s alpha 0.76. The SEM revealed a 4-factor model fitting the data well (comparative fit index 0.939, root mean square error of approximation 0.033). Discriminant validity was good (intervention group scoring higher than control, p < 0.001, unpaired Student’s t-test). Conclusion. The preschool-FLAT revealed good psychometric properties, adequate validity and internal consistency. This is the only instrument in the literature specifically targeted to 3–6 years old children that could be effectively used to assess food literacy.

Download Full-text

Development and psychometric properties of the Caring Behaviors Assessment Tool Nursing Version-Short Form

International Journal for Quality in Health Care ◽

10.1093/intqhc/mzaa134 ◽

2020 ◽

Vol 32 (10) ◽

pp. 701-707

Author(s):

Mehtap Akgün ◽

Selma Turan Kavradim ◽

İlkay Boz ◽

Zeynep Özer

Keyword(s):

Psychometric Properties ◽

Root Mean Square ◽

Internal Consistency ◽

Content Validity ◽

Assessment Tool ◽

Short Form ◽

Mean Square ◽

Structural Validity ◽

Caring Behaviors ◽

Fit Index

Abstract Objectives To develop and examine the psychometric properties of the Caring Behaviors Assessment Tool Nursing Version-Short Form (CBAN-SF) based on the Theory of Human Caring to assess the nurses’ perceptions about caring behaviors. Design This study is based on the Consensus-based Standards for the Selection of Health Measurement Instruments checklist. Setting The study was conducted at the medical-surgical services of Akdeniz University Hospital between October 2019 and January 2020. Participants This study was conducted with 216 nurses working in the surgery and internal clinics. Main Outcome Measures Psychometric properties of the Turkish version of the CBAN-SF with 27 items. Results It was found that the Content Validity Index (CVI) for the items of the draft scale was between 0.972 and 1.00 and the instrument’s CVI had an average score of 0.994. The CBAN-SF had good fit indexes (chi-square goodness of fit / degrees of freedom = 2.914, root mean square error of approximation = 0.075, comparative fit index = 0.984, non-normed fit index = 0.983, normed fit index = 0.972 and standardized root mean square residuals = 0.054) in structural validity. For internal consistency, the Cronbach’s alpha, Spearman–Brown and the Guttman split-half coefficients were all 0.974. The Cronbach’s alpha coefficient for the seven subfactors of the scale ranged between 0.793 and 0.904 and had acceptable internal consistency. The item-total score correlation of the scale was 0.648–0.829, and the factor loadings were 0.455–0.769. Conclusion The structural validity, internal consistency and content validity of the CBAN-SF supported to be a reliable and valid tool for assessment of caring behaviors by nurses.

Download Full-text