scholarly journals Ensuring the Validity of Technical Skill Assessment Outcomes in Technology Education in Nigeria

2016 ◽  
Vol 6 (5) ◽  
Author(s):  
Nkechi Patricia-Mary Esomonu ◽  
Martins Ndibem Esomonu ◽  
Gbenga Kayode Oluwatoyo
Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2666
Author(s):  
Renáta Nagyné Elek ◽  
Tamás Haidegger

BACKGROUND: Sensor technologies and data collection practices are changing and improving quality metrics across various domains. Surgical skill assessment in Robot-Assisted Minimally Invasive Surgery (RAMIS) is essential for training and quality assurance. The mental workload on the surgeon (such as time criticality, task complexity, distractions) and non-technical surgical skills (including situational awareness, decision making, stress resilience, communication, leadership) may directly influence the clinical outcome of the surgery. METHODS: A literature search in PubMed, Scopus and PsycNet databases was conducted for relevant scientific publications. The standard PRISMA method was followed to filter the search results, including non-technical skill assessment and mental/cognitive load and workload estimation in RAMIS. Publications related to traditional manual Minimally Invasive Surgery were excluded, and also the usability studies on the surgical tools were not assessed. RESULTS: 50 relevant publications were identified for non-technical skill assessment and mental load and workload estimation in the domain of RAMIS. The identified assessment techniques ranged from self-rating questionnaires and expert ratings to autonomous techniques, citing their most important benefits and disadvantages. CONCLUSIONS: Despite the systematic research, only a limited number of articles was found, indicating that non-technical skill and mental load assessment in RAMIS is not a well-studied area. Workload assessment and soft skill measurement do not constitute part of the regular clinical training and practice yet. Meanwhile, the importance of the research domain is clear based on the publicly available surgical error statistics. Questionnaires and expert-rating techniques are widely employed in traditional surgical skill assessment; nevertheless, recent technological development in sensors and Internet of Things-type devices show that skill assessment approaches in RAMIS can be much more profound employing automated solutions. Measurements and especially big data type analysis may introduce more objectivity and transparency to this critical domain as well. SIGNIFICANCE: Non-technical skill assessment and mental load evaluation in Robot-Assisted Minimally Invasive Surgery is not a well-studied area yet; while the importance of this domain from the clinical outcome’s point of view is clearly indicated by the available surgical error statistics.


Author(s):  
Afipah Muyassarah ◽  
Tursina Ratu ◽  
Muhammad Erfan

<p class="AbstractEnglish"><strong>Abstract: </strong>STEM (Science, Technology, Education, and Mathematics) is learning that not only involves scientific and mathematical concepts (cognitive aspects) but also technically about how a concept is obtained (motoric). The aspect of motor ability is one of the aspects obtained by students in the learning process apart from the cognitive and affective aspects, but generally most studies put forward cognitive aspects (learning outcomes) so that motor skills are rarely exposed. Therefore this study aims to find out how much influence the STEM-based Physics Learning has on students' motor skills. This research is a quasi-experimental study with the design of One Groups Pretest-Posttest. Data was collected by student worksheet and Student Motor Skill Assessment Instrument Sheets and analysed by paired sample t-test. The results showed that the results of paired sample t test (t value = -64,761 far below t table) and the obtained N-Gain value was 0.55 (moderate gain) so it can be concluded that STEM-based physics learning is significantly capable of increasing students' motor ability.</p><p class="AbstrakIndonesia"><strong>Abstrak: </strong>Pembelajaran STEM <em>(Science, Technology, Education, and Mathematics) </em>merupakan pembelajaran yang tidak hanya melibatkan kosep-konsep sains dan matematika saja (aspek kognitif) tetapi juga secara teknis mengenai bagaimana suatu konsep itu diperoleh (motorik). Aspek kemampuan motorik merupakan salah satu aspek yang diperoleh siswa dalam proses pembelajaran selain daripada aspek kognitif dan afektif, namun umumnya kebanyakan penelitian lebih mengedepankan aspek kognitif (hasil belajar) sehingga kemampuan motorik jarang terekspose. Oleh karena itu penelitian ini bertuuan untuk mengetahui seberapa besar pengaruh Pembelajaran Fisika berbasis STEM terhadap kemampuan motorik siswa. Penelitian ini termasuk penelitian kuasi eksperimen dengan desain <em>One Groups Pretest-Posttest</em>. Data dikumpulkan dengan LKPD dan Lembar Instrumen Penilaian Kemampuan Motorik Siswa serta dianalisis dengan uji-t sampel berpasangan. Hasil penelitian menunjukkan bahwa hasil uji <em>paired sample</em> <em>t test</em> (nilai t = -64,761 jauh dibawah t tabel (-1.69092), dan nilai N-Gain yang diperoleh sebesar 0,55 (gain sedang) sehingga dapat ditarik kesimpulan bahwa pembelajaran fisika berbasis STEM secara signifikan mampu meningkatnya capaian kemampuan motorik siswa.</p>


2021 ◽  
Vol 108 (Supplement_7) ◽  
Author(s):  
Oliver Salazar ◽  
Simon Erridge ◽  
Jasmine Winter Beatty ◽  
Ara Darzi ◽  
Sanjay Purkayastha ◽  
...  

Abstract Aims Technical skill is associated with improved postoperative outcomes. Adoption of a formalised high-stakes assessment of surgical skill is technically challenging and limited by the financial and human resources available. We aimed to assess the ability to adopt gaze behaviour analysis as an assessment of surgical skill within live open inguinal herniorrhaphy. Methods Surgeons’ gaze was measured with Tobii Pro eye-tracking Glasses 2 (Tobii AB). All grades of surgeons were included. Primary outcomes were dwell time (%) and fixation frequency (count/s), as markers of cognition, on areas of interest correlated to mean Objective Skill Assessment of Technical Skill score. Secondary outcomes assessed effort and concentration levels through maximum pupil diameter (mm) and rate of pupil change (mm/s) correlated to perceived workload (SURG-TLX). Three operative segments underwent analysis: mesh preparation, fixation and muscle closure. Spearman’s and Pearson’s correlation were performed with significance set at p &lt; 0.05. Results 5 cases were analysed, totalling 270 minutes of video footage. All participants were senior surgical trainees and right-hand-dominant. The median number of hernia operations performed was 160 (range:100-500). The median ASA score of each patient participant was 2 (range:1-2). The median operation length was 45 mins (range:40-90 mins). There were no statistically significant primary outcomes from this pilot data (p &gt; 0.05). Conclusions This pilot study demonstrated the feasibility of recording gaze behaviours for comparison against formal skills assessment to determine the role of eye tracking in live high stakes technical skills assessment. A full study will now commence based on formal power calculation.


2018 ◽  
Vol 199 (4S) ◽  
Author(s):  
Yetong Zhang ◽  
Hei Law ◽  
Tae-Kyung Kim ◽  
David Miller ◽  
James Montie ◽  
...  

2018 ◽  
Vol 128 (10) ◽  
pp. 2296-2300 ◽  
Author(s):  
Mathilde Labbé ◽  
Meredith Young ◽  
Lily H.P. Nguyen

Author(s):  
Ishwar Kalbandi ◽  
Mohammad Suhaib Ahmed ◽  
Rohan Atale ◽  
Ajinkya Bhujbal ◽  
Nikhunj Saraf

2008 ◽  
Vol 3 (3) ◽  
pp. 91-101 ◽  
Author(s):  
Niko G. Lagumen ◽  
Dale J. Butterwick ◽  
David M. Paskevich ◽  
Tak S. Fung ◽  
Tyrone L. Donnon

Objective: To establish the intra-rater reliability of nine content-validated Technical Skill Assessment Instruments (TSAI) for the skills of athletic taping. Setting: University of Calgary. Subjects: Canadian Certified Athletic Therapists, CAT(C), with a mean ± SD of 9.6 ± 10.8 years as a CAT(C), 7.8 ± 10.9 years as a Supervisory Athletic Therapist, 8.5 ± 12.0 years teaching athletic taping skills, and 9.2 ± 11.5 years evaluating athletic taping skills. Design: Six Certified Athletic Therapists from Canada completed the repetitive evaluations of nine different athletic taping scenarios. Each rater evaluated the performance of a student therapist taping a standardized patient while using the appropriate TSAI designed for each athletic taping scenario. Evaluations occurred once per month for five successive months. Raters viewed the performances on a portable DVD player at a central testing site. Measurements: The percent scores of 270 completed TSAIs were used for analysis. ICC (3, k) was used to quantify the intra-rater reliability. We used a One-way ANOVA with repeated measures to determine if mean differences across testing months existed within raters. Significance was achieved with α = 0.05. Results: ICC values for the nine TSAIs ranged from 0.65 to 0.95 with Ankle 3 and Thumb 2 achieving the lowest and highest ICC values respectively. One-way ANOVA with repeated measures did not provide significant mean differences between testing months within each rater. Conclusion: The nine TSAIs possess substantial to almost perfect reliability with seven TSAIs appropriate for summative evaluations and two appropriate for formative evaluations.


2014 ◽  
Vol 49 (3) ◽  
pp. 368-372 ◽  
Author(s):  
Mark R. Lafave ◽  
Dale J. Butterwick

Context: Athletic taping skills are highly valued clinical competencies in the athletic therapy and training profession. The Technical Skill Assessment Instrument (TSAI) has been content validated and tested for intrarater reliability. Objective: To test the reliability of the TSAI using a more robust measure of reliability, generalizability theory, and to hypothetically and mathematically project the optimal number of raters and scenarios to reliably measure athletic taping skills in the future. Setting: Mount Royal University. Design: Observational study. Patients or Other Participants: A total of 29 university students (8 men, 21 women; age = 20.79 ± 1.59 years) from the Athletic Therapy Program at Mount Royal University. Intervention(s): Participants were allowed 10 minutes per scenario to complete prophylactic taping for a standardized patient presenting with (1) a 4-week-old second-degree ankle sprain and (2) a thumb that had been hyperextended. Two raters judged student performance using the TSAI. Main Outcome Measure(s): Generalizability coefficients were calculated using variance scores for raters, participants, and scenarios. A decision study was calculated to project the optimal number of raters and scenarios to achieve acceptable levels of reliability. Generalizability coefficients were interpreted the same as other reliability coefficients, with 0 indicating no reliability and 1.0 indicating perfect reliability. Results: The result of our study design (2 raters, 1 standardized patient, 2 scenarios) was a generalizability coefficient of 0.67. Decision study projects indicated that 4 scenarios were necessary to reliably measure athletic taping skills. Conclusions: We found moderate reliability coefficients. Researchers should include more scenarios to reliably measure athletic taping skills. They should also focus on the development of evidence-based practice guidelines and standards of athletic taping and should test those standards using a psychometrically sound instrument, such as the TSAI.


Sign in / Sign up

Export Citation Format

Share Document