scholarly journals RAS-NOTECHS: validity and reliability of a tool for measuring non-technical skills in robotic-assisted surgery settings

2021 ◽  
Author(s):  
Julia Schreyer ◽  
Amelie Koch ◽  
Annika Herlemann ◽  
Armin Becker ◽  
Boris Schlenker ◽  
...  
Keyword(s):  
Assessment Tool ◽  
Qualitative Interviews ◽  
Surgical Care ◽  
Technical Skills ◽  
Rater Agreement ◽  
Validity And Reliability ◽  
Delphi Consensus ◽  
Robotic Assisted ◽  
Robotic Assisted Surgery ◽  
Strong Agreement

Abstract Background Non-technical skills (NTS) are essential for safe surgical practice as they impact workflow and patient outcomes. Observational tools to measure operating room (OR) teams’ NTS have been introduced. However, there are none that account for the specific teamwork challenges introduced by robotic-assisted surgery (RAS). We set out to develop and content-validate a tool to assess multidisciplinary NTS in RAS. Methodology Stepwise, multi-method procedure. Observations in different surgical departments and a scoping literature review were first used to compile a set of RAS-specific teamwork behaviours. This list was refined and expert validated using a Delphi consensus approach consisting of qualitative interviews and a quantitative survey. Then, RAS-specific behaviours were merged with a well-established assessment tool on OR teamwork (NOTECHS II). Finally, the new tool—RAS-NOTECHS—was applied in standardized observations of real-world procedures to test its reliability (inter-rater agreement via intra-class correlations). Results Our scoping review revealed 5242 articles, of which 21 were included based on pre-established inclusion criteria. We elicited 16 RAS-specific behaviours from the literature base. These were synthesized with further 18 behavioural markers (obtained from 12 OR-observations) into a list of 26 behavioural markers. This list was reviewed by seven RAS experts and condensed to 15 expert-validated RAS-specific behavioural markers which were then merged into NOTECHS II. For five observations of urologic RAS procedures (duration: 13 h and 41 min), inter-rater agreement for identification of behavioural markers was strong. Agreement of RAS-NOTECHS scores indicated moderate to strong agreement. Conclusions RAS-NOTECHS is the first observational tool for multidisciplinary NTS in RAS. In preliminary application, it has been shown to be reliable. Since RAS is rapidly increasing and challenges for effective and safe teamwork remain at the forefront of quality and safety of surgical care, RAS-NOTECHS may contribute to training and improvement efforts in technology-facilitated surgeries.

Objective structured assessment of technical skill in temporal bone dissection: validation of a novel tool

2021 ◽  
pp. 1-11
Author(s):  
M Stavrakas ◽  
G Menexes ◽  
S Triaridis ◽  
P Bamidis ◽  
J Constantinidis ◽  
...  
Keyword(s):  
Temporal Bone ◽  
Rating Scale ◽  
Assessment Tool ◽  
Three Dimensional ◽  
Technical Skills ◽  
Technical Skill ◽  
Global Rating Scale ◽  
Global Rating ◽  
Validity And Reliability ◽  
Cortical Mastoidectomy

Abstract Objective This study developed an assessment tool that was based on the objective structured assessment for technical skills principles, to be used for evaluation of surgical skills in cortical mastoidectomy. The objective structured assessment of technical skill is a well-established tool for evaluation of surgical ability. This study also aimed to identify the best material and printing method to make a three-dimensional printed temporal bone model. Methods Twenty-four otolaryngologists in training were asked to perform a cortical mastoidectomy on a three-dimensional printed temporal bone (selective laser sintering resin). They were scored according to the objective structured assessment of technical skill in temporal bone dissection tool developed in this study and an already validated global rating scale. Results Two external assessors scored the candidates, and it was concluded that the objective structured assessment of technical skill in temporal bone dissection tool demonstrated some main aspects of validity and reliability that can be used in training and performance evaluation of technical skills in mastoid surgery. Conclusion Apart from validating the new tool for temporal bone dissection training, the study showed that evolving three-dimensional printing technologies is of high value in simulation training with several advantages over traditional teaching methods.

Robotic-Assisted Surgery Training (RAST) Program: An Educational Research Protocol

2021 ◽  
Author(s):  
Maria Castaldi ◽  
Mathias Palmer ◽  
Jorge Con ◽  
Ziad Abouezzi ◽  
Rifat Latifi ◽  
...  
Keyword(s):  
Educational Research ◽  
Operating Room ◽  
Operating Time ◽  
Technical Skills ◽  
Educational Environment ◽  
Research Protocol ◽  
Robotic Assisted ◽  
Surgery Training ◽  
Robotic Assisted Surgery ◽  
Over Time

Technology has had a dramatic impact on how diseases are diagnosed and treated. Although cut, sew, and tie remain the staples of surgical craft, new technical skills are required. While there is no replacement for live operative experience, training outside the operating room offers structured educational opportunities and stress modulation. A stepwise program for acquiring new technical skills required in robotic surgery involves three modules: ergonomic, psychomotor, and procedural. This is a prospective, educational research protocol aiming at evaluating the responsiveness of general surgery residents in Robotic-Assisted Surgery Training (RAST). Responsiveness is defined as change in performance over time. Performance is measured by the following content-valid metrics for each module. Module 1 proficiency in ergonomics includes: cart deploy, boom control, cart driving, camera port docking, targeting anatomy, flex joint, clearance joint and port nozzle adjusting, and routine and emergent undocking. Module 2 proficiency in psychomotor skills includes tissue handling, accuracy error, knot quality, and operating time. Module 3 proficiency in procedural skills prevents deviations from standardized sequential procedural steps in order to test length of specimen resection, angle for transection, vessel stump length post ligation, distance of anastomosis from critical landmarks, and proximal and distal resection margins. Resident responsiveness over time will be assessed comparing the results of baseline testing with final testing. Educational interventions will include viewing one instructional video prior to module commencement, response to module-specific multiple-choice questions, and individual weekly training sessions with a robotic instructor in the operating room. Residents will progress through modules upon successful final testing and will evaluate the educational environment with the Dundee Ready Educational Environment Measure (DREEM) inventory. The RAST program protocol outlined herein is an educational challenge with the primary endpoint to provide evidence that formal instruction has an impact on proficiency and safety in executing robotic skills.

Inter-Agency Communication and Operations Capabilities during a Hospital Functional Exercise: Reliability and Validity of a Measurement Tool

2010 ◽  
Vol 25 (1) ◽  
pp. 52-58 ◽  
Author(s):  
Elena Savoia ◽  
Paul D. Biddinger ◽  
Jon Burstein ◽  
Michael A. Stoto
Keyword(s):  
Assessment Tool ◽  
Reliability And Validity ◽  
Preliminary Evidence ◽  
Measurement Tool ◽  
Rater Agreement ◽  
Validity And Reliability ◽  
The Public ◽  
External Evaluators ◽  
Functional Exercise ◽  
Qualitative Performance

AbstractIntroduction:As proxies for actual emergencies, drills and exercises can raise awareness, stimulate improvements in planning and training, and provide an opportunity to examine how different components of the public health system would combine to respond to a challenge. Despite these benefits, there remains a substantial need for widely accepted and prospectively validated tools to evaluate agencies' and hospitals' performance during such events. Unfortunately, to date, few studies have focused on addressing this need. The purpose of this study was to assess the validity and reliability of a qualitative performance assessment tool designed to measure hospitals' communication and operational capabilities during a functional exercise.Methods:The study population included 154 hospital personnel representing nine hospitals that participated in a functional exercise in Massachusetts in June 2008. A 25-item questionnaire was developed to assess the following three hospital functional capabilities: (1) inter-agency communication; (2) communication with the public; and (3) disaster operations. Analyses were conducted to examine internal consistency, associations among scales, the empirical structure of the items, and inter-rater agreement.Results:Twenty-two questions were retained in the final instrument, which demonstrated reliability with alpha coefficients of 0.83 or higher for all scales. A three-factor solution from the principal components analysis accounted for 57% of the total variance, and the factor structure was consistent with the original hypothesized domains. Inter-rater agreement between participants' self-reported scores and external evaluators' scores ranged from moderate to good.Conclusions:The resulting 22-item performance measurement tool reliably measured hospital capabilities in a functional exercise setting, with preliminary evidence of concurrent and criterion-related validity.

scholarly journals A scale for measuring perceived bureaucratic readiness for smart cities in Indonesia

2019 ◽  
Vol 22 (1) ◽  
pp. 25-39 ◽  
Author(s):  
Arif Budy Pratama ◽  
Satria Aji Imawan
Keyword(s):  
Smart City ◽  
Online Survey ◽  
Assessment Tool ◽  
Smart Cities ◽  
Qualitative Interviews ◽  
City Government ◽  
Future Research ◽  
Validity And Reliability ◽  
Content Type ◽  
Government Bureaucracy

Purpose The purpose of this paper is to develop and validate a scale for measuring perceived bureaucratic readiness for smart city initiatives. Design/methodology/approach The present study employs a mixed method approach to achieve its research objectives. An exploratory study, consisting of literature review and qualitative interviews with key informants, was conducted to develop an initial instrument for measuring bureaucratic readiness. An online survey of 40 civil servants involved in smart city programmes in the Yogyakarta City government was then administered to test the instrument’s validity and reliability. Findings Perceived bureaucratic readiness can be measured through four dimensions: commitment of the upper echelons, legal support, information technology resources and governance. Research limitations/implications The proposed scale provides an alternative instrument for measuring perceived bureaucratic readiness for smart city initiatives. However, as data were only derived from one city government, they are relatively small in scope. Future research can be conducted for generalisation by replicating this study in other cities, thereby measuring its effectiveness in other contexts and settings. Practical implications This study not only provides a better understanding of bureaucratic readiness for smart city initiatives, but also proposes an assessment tool as a practical means of assessing bureaucratic readiness. The quantification of readiness is beneficial to putting smart city programmes into practice, as it allows smart city managers to assess the internal bureaucracy’s level of readiness. It also allows managers to mitigate and further policy agendas and thereby improve the bureaucracy’s support for smart city programmes. Originality/value Literature sometimes underestimates the role of bureaucracy in smart city implementation while overly stressing stakeholders, vendors and technology. This paper attempts to contribute to smart city research by reaching beyond the technological perspective and focusing on local government bureaucracy. None of the extant literature provides a scale for measuring bureaucratic readiness. The study thus proposes a systematic way to develop a means of measuring perceived bureaucratic readiness for smart city programmes.

scholarly journals How to Bring Surgery to the Next Level: Interpretable Skills Assessment in Robotic-Assisted Surgery

2020 ◽  
Vol 36 (6) ◽  
pp. 463-470
Author(s):  
Kristen C. Brown ◽  
Kiran D. Bhattacharyya ◽  
Sue Kulason ◽  
Aneeq Zia ◽  
Anthony Jarc
Keyword(s):  
Logistic Regression ◽  
Technical Skills ◽  
Technical Skill ◽  
Small Subset ◽  
Robotic Assisted ◽  
Robotic Assisted Surgery ◽  
Surgical Trainees ◽  
Objective Metrics ◽  
Data Driven Approach ◽  
Similar Accuracy

<b><i>Introduction:</i></b> A surgeon’s technical skills are an important factor in delivering optimal patient care. Most existing methods to estimate technical skills remain subjective and resource intensive. Robotic-assisted surgery (RAS) provides a unique opportunity to develop objective metrics using key elements of intraoperative surgeon behavior which can be captured unobtrusively, such as instrument positions and button presses. Recent studies have shown that objective metrics based on these data (referred to as objective performance indicators [OPIs]) correlate to select clinical outcomes during robotic-assisted radical prostatectomy. However, the current OPIs remain difficult to interpret directly and, therefore, to use within structured feedback to improve surgical efficiencies. <b><i>Methods:</i></b> We analyzed kinematic and event data from da Vinci surgical systems (Intuitive Surgical, Inc., Sunnyvale, CA, USA) to calculate values that can summarize the use of robotic instruments, referred to as OPIs. These indicators were mapped to broader technical skill categories of established training protocols. A data-driven approach was then applied to further sub-select OPIs that distinguish skill for each technical skill category within each training task. This subset of OPIs was used to build a set of logistic regression classifiers that predict the probability of expertise in that skill to identify targeted improvement and practice. The final, proposed feedback using OPIs was based on the coefficients of the logistic regression model to highlight specific actions that can be taken to improve. <b><i>Results:</i></b> We determine that for the majority of skills, only a small subset of OPIs (2–10) are required to achieve the highest model accuracies (80–95%) for estimating technical skills within clinical-like tasks on a porcine model. The majority of the skill models have similar accuracy as models predicting overall expertise for a task (80–98%). Skill models can divide a prediction into interpretable categories for simpler, targeted feedback. <b><i>Conclusion:</i></b> We define and validate a methodology to create interpretable metrics for key technical skills during clinical-like tasks when performing RAS. Using this framework for evaluating technical skills, we believe that surgical trainees can better understand both what can be improved and how to improve.

Evaluation of the Modified Naranjo Criteria for Assessing Causal Attribution of Clinical Outcome to Homeopathic Intervention as Presented in Case Reports

Homeopathy ◽  
2020 ◽  
Vol 109 (04) ◽  
pp. 191-197
Author(s):  
Chetna Deep Lamba ◽  
Vishwa Kumar Gupta ◽  
Robbert van Haselen ◽  
Lex Rutten ◽  
Nidhi Mahajan ◽  
...  
Keyword(s):  
Clinical Outcome ◽  
Causal Relationship ◽  
Clinical Case ◽  
Assessment Tool ◽  
Causal Attribution ◽  
Case Reports ◽  
Face Validity ◽  
Rater Agreement ◽  
The Face ◽  
Selection Of

Abstract Objectives The objective of this study was to establish the reliability and content validity of the “Modified Naranjo Criteria for Homeopathy—Causal Attribution Inventory” as a tool for attributing a causal relationship between the homeopathic intervention and outcome in clinical case reports. Methods Purposive sampling was adopted for the selection of information-rich case reports using pre-defined criteria. Eligible case reports had to fulfil a minimum of nine items of the CARE Clinical Case Reporting Guideline checklist and a minimum of three of the homeopathic HOM-CASE CARE extension items. The Modified Naranjo Criteria for Homeopathy Inventory consists of 10 domains. Inter-rater agreement in the scoring of these domains was determined by calculating the percentage agreement and kappa (κ) values. A κ greater than 0.4, indicating fair agreement between raters, in conjunction with the absence of concerns regarding the face validity, was taken to indicate the validity of a given domain. Each domain was assessed by four raters for the selected case reports. Results Sixty case reports met the inclusion criteria. Inter-rater agreement/concordance per domain was “perfect” for domains 1 (100%, κ = 1.00) and 2 (100%, κ = 1.00); “almost perfect” for domain 8 (97.5%, κ = 0.86); “substantial” for domains 3 (96.7%, κ = 0.80) and 5 (91.1%, κ = 0.70); “moderate” for domains 4 (83.3%, κ = 0.60), 7 (67.8%, κ = 0.46) and 9 (99.2%, κ = 0.50); and “fair” for domain 10 (56.1%, κ = 0.38). For domains 6A (46.7%, κ = 0.03) and 6B (50.3%, κ = 0.18), there was “slight agreement” only. Thus, the validity of the Modified Naranjo Criteria for Homeopathy tool was established for each of its domains, except for the two that pertain to direction of cure (domains 6A and 6B). Conclusion The Modified Naranjo Criteria for Homeopathy—Causal Attribution Inventory was identified as a valid tool for assessing the likelihood of a causal relationship between a homeopathic intervention and clinical outcome. Improved wordings for several criteria have been proposed for the assessment tool, under the new acronym “MONARCH”. Further assessment of two MONARCH domains is required.

scholarly journals Detailed cost of robotic-assisted surgery in the Australian public health sector: from implementation to a multi-specialty caseload

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Kate McBride ◽  
Daniel Steffens ◽  
Christina Stanislaus ◽  
Michael Solomon ◽  
Teresa Anderson ◽  
...  
Keyword(s):  
Public Sector ◽  
Health Sector ◽  
Public Hospitals ◽  
Fixed Costs ◽  
Tertiary Referral Hospital ◽  
Robotic Assisted ◽  
The Public ◽  
Robotic Assisted Surgery ◽  
Specific Implementation ◽  
Theatre Utilisation

Abstract Background A barrier to the uptake of robotic-assisted surgery (RAS) continues to be the perceived high costs. A lack of detailed costing information has made it difficult for public hospitals in particular to determine whether use of the technology is justified. This study aims to provide a detailed description of the patient episode costs and the contribution of RAS specific costs for multiple specialties in the public sector. Methods A retrospective descriptive costing review of all RAS cases undertaken at a large public tertiary referral hospital in Sydney, Australia from August 2016 to December 2018 was completed. This included RAS cases within benign gynaecology, cardiothoracic, colorectal and urology, with the total costs described utilizing various inpatient costing data, and RAS specific implementation, maintenance and consumable costs. Results Of 211 RAS patients, substantial variation was found between specialties with the overall median cost per patient being $19,269 (Interquartile range (IQR): $15,445 to $32,199). The RAS specific costs were $8828 (46%) made up of fixed costs including $4691 (24%) implementation and $2290 (12%) maintenance, both of which are volume dependent; and $1848 (10%) RAS consumable costs. This was in the context of 37% robotic theatre utilisation. Conclusions There is considerable variation across surgical specialties for the cost of RAS. It is important to highlight the different cost components and drivers associated with a RAS program including its dependence on volume and how it fits within funding systems in the public sector.

Sign in / Sign up

Export Citation Format

Share Document