scholarly journals Estimating the number of usability problems affecting medical devices: modelling the discovery matrix

2020 ◽  
Vol 20 (1) ◽  
Author(s):  
Vincent Vandewalle ◽  
Alexandre Caron ◽  
Coralie Delettrez ◽  
Renaud Périchon ◽  
Sylvia Pelayo ◽  
...  
Keyword(s):  
Medical Devices ◽  
Market Access ◽  
Real Life ◽  
Usability Testing ◽  
Probability Of Detection ◽  
Usability Problems ◽  
Usability Problem ◽  
Wide Range ◽  
The Matrix ◽  
Problem Detection

Abstract Background Usability testing of medical devices are mandatory for market access. The testings’ goal is to identify usability problems that could cause harm to the user or limit the device’s effectiveness. In practice, human factor engineers study participants under actual conditions of use and list the problems encountered. This results in a binary discovery matrix in which each row corresponds to a participant, and each column corresponds to a usability problem. One of the main challenges in usability testing is estimating the total number of problems, in order to assess the completeness of the discovery process. Today’s margin-based methods fit the column sums to a binomial model of problem detection. However, the discovery matrix actually observed is truncated because of undiscovered problems, which corresponds to fitting the marginal sums without the zeros. Margin-based methods fail to overcome the bias related to truncation of the matrix. The objective of the present study was to develop and test a matrix-based method for estimating the total number of usability problems. Methods The matrix-based model was based on the full discovery matrix (including unobserved columns) and not solely on a summary of the data (e.g. the margins). This model also circumvents a drawback of margin-based methods by simultaneously estimating the model’s parameters and the total number of problems. Furthermore, the matrix-based method takes account of a heterogeneous probability of detection, which reflects a real-life setting. As suggested in the usability literature, we assumed that the probability of detection had a logit-normal distribution. Results We assessed the matrix-based method’s performance in a range of settings reflecting real-life usability testing and with heterogeneous probabilities of problem detection. In our simulations, the matrix-based method improved the estimation of the number of problems (in terms of bias, consistency, and coverage probability) in a wide range of settings. We also applied our method to five real datasets from usability testing. Conclusions Estimation models (and particularly matrix-based models) are of value in estimating and monitoring the detection process during usability testing. Matrix-based models have a solid mathematical grounding and, with a view to facilitating the decision-making process for both regulators and device manufacturers, should be incorporated into current standards.

scholarly journals Estimating the number of usability problems affecting medical devices: modelling the discovery matrix

2020 ◽  
Author(s):  
Vincent Vandewalle ◽  
Alexandre Caron ◽  
Coralie Delettrez ◽  
Renaud Périchon ◽  
Sylvia Pelayo ◽  
...  
Keyword(s):  
Medical Devices ◽  
Market Access ◽  
Real Life ◽  
Usability Testing ◽  
Probability Of Detection ◽  
Usability Problems ◽  
Usability Problem ◽  
Wide Range ◽  
The Matrix ◽  
Problem Detection

Abstract Background Usability testing of medical devices are mandatory for market access. The testings’ goal is to identify usability problems that could cause harm to the user or limit the device’s effectiveness. In practice, human factor engineers study participants under actual conditions of use and list the problems encountered. This results in a binary discovery matrix in which each row corresponds to a participant, and each column corresponds to a usability problem. One of the main challenges in usability testing is estimating the total number of problems, in order to assess the completeness of the discovery process. Today’s margin-based methods fit the column sums to a binomial model of problem detection. However, the discovery matrix actually observed is truncated because of undiscovered problems, which corresponds to fitting the marginal sums without the zeros. Margin-based methods fail to overcome the bias related to truncation of the matrix. The objective of the present study was to develop and test a matrix-based method for estimating the total number of usability problems. Methods The matrix-based model was based on the full discovery matrix (including unobserved columns) and not solely on a summary of the data (e.g. the margins). This model also circumvents a drawback of margin-based methods by simultaneously estimating the model’s parameters and the total number of problems. Furthermore, the matrix-based method takes account of a heterogeneous probability of detection, which reflects a real-life setting. Results We assessed the matrix-based method’s performance in a range of settings reflecting real-life usability testing and with heterogeneous probabilities of problem detection. In our simulations, the matrix-based method improved the estimation of the number of problems (in terms of bias, consistency, and coverage probability) in a wide range of settings. We also applied our method to five real datasets from usability testing. Conclusions Estimation models (and particularly matrix-based models) are of value in estimating and monitoring the detection process during usability testing. Matrix-based models have a solid mathematical grounding and, with a view to facilitating the decision-making process for both regulators and device manufacturers, should be incorporated into current standards.

scholarly journals Estimating the number of usability problems affecting medical devices: modelling the discovery matrix

2020 ◽  
Author(s):  
Vincent Vandewalle ◽  
Alexandre Caron ◽  
Coralie Delettrez ◽  
Renaud Périchon ◽  
Sylvia Pelayo ◽  
...  
Keyword(s):  
Medical Devices ◽  
Market Access ◽  
Real Life ◽  
Usability Testing ◽  
Probability Of Detection ◽  
Usability Problems ◽  
Usability Problem ◽  
Wide Range ◽  
The Matrix ◽  
Problem Detection

Abstract Background . Usability testing of medical devices are mandatory for market access. The testings’ goal is to identify usability problems that could cause harm to the user or limit the device’s effectiveness. In practice, human factor engineers study participants under actual conditions of use and list the problems encountered. This results in a binary discovery matrix in which each row corresponds to a participant, and each column corresponds to a usability problem. One of the main challenges in usability testing is estimating the total number of problems, in order to assess the completeness of the discovery process. Today’s margin-based methods fit the column sums to a binomial model of problem detection. However, the discovery matrix actually observed is truncated because of undiscovered problems, which corresponds to fitting the marginal sums without the zeros. Margin-based methods fail to overcome the bias related to truncation of the matrix. The objective of the present study was to develop and test a matrix-based method for estimating the total number of usability problems. Methods . The matrix-based model was based on the full discovery matrix (including unobserved columns) and not solely on a summary of the data (e.g. the margins). This model also circumvents a drawback of margin-based methods by simultaneously estimating the model’s parameters and the total number of problems. Furthermore, the matrix-based method takes account of a heterogeneous probability of detection, which reflects a real-life setting. Results . We assessed the matrix-based method’s performance in a range of settings reflecting real-life usability testing and with heterogeneous probabilities of problem detection. In our simulations, the matrix-based method improved the estimation of the number of problems (in terms of bias, consistency, and coverage probability) in a wide range of settings. We also applied our method to five real datasets from usability testing. Conclusions . Estimation models (and particularly matrix-based models) are of value in estimating and monitoring the detection process during usability testing. Matrix-based models have a solid mathematical grounding and, with a view to facilitating the decision-making process for both regulators and device manufacturers, should be incorporated into current standards.

scholarly journals Estimating the number of usability problems affecting medical devices: modelling the discovery matrix

2020 ◽  
Author(s):  
Vincent Vandewalle ◽  
Alexandre Caron ◽  
Coralie Delettrez ◽  
Renaud Périchon ◽  
Sylvia Pelayo ◽  
...  
Keyword(s):  
Medical Devices ◽  
Market Access ◽  
Real Life ◽  
Usability Testing ◽  
Probability Of Detection ◽  
Usability Problems ◽  
Usability Problem ◽  
Wide Range ◽  
The Matrix ◽  
Problem Detection

Abstract Background. Usability testing of medical devices are mandatory for market access. The testings’ goal is to identify usability problems that could cause harm to the user or limit the device’s effectiveness. In practice, human factor engineers study participants under actual conditions of use and list the problems encountered. This results in a binary discovery matrix in which each row corresponds to a participant, and each column corresponds to a usability problem. One of the main challenges in usability testing is estimating the total number of problems, in order to assess the completeness of the discovery process. Today’s margin-based methods fit the column sums to a binomial model of problem detection. However, the discovery matrix actually observed is truncated because of undiscovered problems, which corresponds to fitting the marginal sums without the zeros. Margin-based methods fail to overcome the bias related to truncation of the matrix. The objective of the present study was to develop and test a matrix-based method for estimating the total number of usability problems.Methods. The matrix-based model was based on the full discovery matrix (including unobserved columns) and not solely on a summary of the data (e.g. the margins). This model also circumvents a drawback of margin-based methods by simultaneously estimating the model’s parameters and the total number of problems. Furthermore, the matrix-based method takes account of a heterogeneous probability of detection, which reflects a real-life setting. As suggested in the usability literature, we assumed that the probability of detection had a logit-normal distribution.Results. We assessed the matrix-based method’s performance in a range of settings reflecting real-life usability testing and with heterogeneous probabilities of problem detection. In our simulations, the matrix-based method improved the estimation of the number of problems (in terms of bias, consistency, and coverage probability) in a wide range of settings. We also applied our method to five real datasets from usability testing.Conclusions. Estimation models (and particularly matrix-based models) are of value in estimating and monitoring the detection process during usability testing. Matrix-based models have a solid mathematical grounding and, with a view to facilitating the decision-making process for both regulators and device manufacturers, should be incorporated into current standards.

scholarly journals Estimating the number of usability problems affecting medical devices: modelling the discovery matrix

2019 ◽  
Author(s):  
Vincent Vandewalle ◽  
Alexandre Caron ◽  
Coralie Delettrez ◽  
Sylvia Pelayo ◽  
Alain Duhamel ◽  
...  
Keyword(s):  
Medical Devices ◽  
Market Access ◽  
Real Life ◽  
Real Data ◽  
Probability Of Detection ◽  
Unknown Parameters ◽  
Usability Problems ◽  
Wide Range ◽  
The Matrix ◽  
Problem Detection

Abstract Background Usability studies of medical devices are mandatory for market access. The studies’ goal is to identify and eliminate usability problems that could cause harm the user or limit the device’s effectiveness. In practice, human factor engineers study participants under actual conditions of use and list the problems encountered. This results in a binary discovery matrix in which each row corresponds to a participant, and each column corresponds to a usability problem. One of the main challenges in usability studies is estimating the total number of problems, in order to assess the completeness of the discovery process. Today’s margin-based methods fit the column sums to a binomial model of problem detection. However, the discovery matrix actually observed is truncated because of undiscovered problems, which corresponds to fitting the marginal sums without the zeros. Margin-based methods fail to overcome the bias related to truncation of the matrix. The objective of the present study was to develop and test a matrix-based method for estimating the total number of usability problems. Methods The matrix-based method models the likelihood of the discovery matrix, and allows one to account for all the available information. It also circumvents a drawback of margin-based methods by simultaneously estimating two unknown parameters: the probability of problem detection and the total number of problems. Furthermore, the matrix-based method takes account of a heterogeneous probability of detection, which better reflects a real-life setting. As suggested in the usability literature, a logit-normal prior for the probability of detection is selected. Results We assessed the matrix-based method’s performance in a range of settings reflecting real-life usability studies and with both homogeneous and heterogeneous probabilities of problem detection. In our simulations, the matrix-based method improved the estimation of the number of problems (in terms of bias, consistency, and coverage probability of the confidence interval) in a wide range of settings. We also applied our method to real data from a usability study of infusion pumps. Conclusions Our method should be applied by regulators and device manufacturers to estimate the number of usability problems using the set of statistical routines provided.

Usability Methods for Ensuring Health Information Technology Safety: Evidence-Based Approaches Contribution of the IMIA Working Group Health Informatics for Patient Safety

2013 ◽  
Vol 22 (01) ◽  
pp. 20-27 ◽  
Author(s):  
A. Kushniruk ◽  
C. Nohr ◽  
H. Takeda ◽  
S. Kuwata ◽  
C. Carvalho ◽  
...  
Keyword(s):  
Information Technology ◽  
Health Information Technology ◽  
Health Information ◽  
Usability Testing ◽  
Evidence Based ◽  
Usability Engineering ◽  
Usability Problems ◽  
Wide Range ◽  
System Usability ◽  
Clinical Simulations

Summary Objectives: Issues related to lack of system usability and potential safety hazards continue to be reported in the health information technology (HIT) literature. Usability engineering methods are increasingly used to ensure improved system usability and they are also beginning to be applied more widely for ensuring the safety of HIT applications. These methods are being used in the design and implementation of many HIT systems. In this paper we describe evidence- based approaches to applying usability engineering methods. Methods: A multi-phased approach to ensuring system usability and safety in healthcare is described. Usability inspection methods are first described including the development of evidence-based safety heuristics for HIT. Laboratory-based usability testing is then conducted under artificial conditions to test if a system has any base level usability problems that need to be corrected. Usability problems that are detected are corrected and then a new phase is initiated where the system is tested under more realistic conditions using clinical simulations. This phase may involve testing the system with simulated patients. Finally, an additional phase may be conducted, involving a naturalistic study of system use under real-world clinical conditions. Results: The methods described have been employed in the analysis of the usability and safety of a wide range of HIT applications, including electronic health record systems, decision support systems and consumer health applications. It has been found that at least usability inspection and usability testing should be applied prior to the widespread release of HIT. However, wherever possible, additional layers of testing involving clinical simulations and a naturalistic evaluation will likely detect usability and safety issues that may not otherwise be detected prior to widespread system release. Conclusion: The framework presented in the paper can be applied in order to develop more usable and safer HIT, based on multiple layers of evidence.

An Empirical Study of Usability Testing: Heuristic Evaluation Vs. User Testing

2002 ◽  
Vol 46 (8) ◽  
pp. 774-778 ◽  
Author(s):  
Enlie Wang ◽  
Barrett Caldwell
Keyword(s):  
Cost Benefit Analysis ◽  
Low Cost ◽  
Cost Benefit ◽  
Usability Testing ◽  
Heuristic Evaluation ◽  
Benefit Analysis ◽  
User Testing ◽  
Testing Methods ◽  
Usability Problems ◽  
Wide Range

In this study, two different usability-testing methods (Heuristic Evaluation and User Testing) were selected to test the usability of a pre-release version of software searching for Science, Mathematics and Engineering education materials. Our major goal is to compare Heuristic Evaluation and User Testing in terms of efficiency, effectiveness and cost/benefit analysis. We found that Heuristic Evaluation was more efficient than User Testing in finding usability problems (41 vs. 10), while User Testing was more effective than Heuristic Evaluation in finding major problems (70% vs.12%). in general, Heuristic Evaluation appears to be more economic in finding a wide range of usability problems by incurring a low cost in comparison to User Testing. However, User Testing can provide more insightful data from real users such as user's performance and satisfaction.

Validation of the Applied Biosystems 7500 Fast Instrument for Detection of Listeria Species with the SureTect Listeria Species PCR Assay

2016 ◽  
Vol 99 (2) ◽  
pp. 407-416
Author(s):  
Jonathan Cloke ◽  
Julia Arizanova ◽  
David Crabtree ◽  
Helen Simpson ◽  
Katharine Evans ◽  
...  
Keyword(s):  
Reference Method ◽  
Probability Of Detection ◽  
Stainless Steel Surface ◽  
Pcr Assay ◽  
Listeria Species ◽  
Surface Samples ◽  
Wide Range ◽  
Significant Difference ◽  
The Matrix ◽  
Level 2

Abstract The Thermo Scientific™ SureTect™ Listeria species Real-Time PCR Assay was certified during 2013 by the AOAC Research Institute (RI) Performance Tested MethodsSM program as a rapid method for the detection of Listeria species from a wide range of food matrixes and surface samples. A method modification study was conducted in 2015 to extend the matrix claims of the product to a wider range of food matrixes. This report details the method modification study undertaken to extend the use of this PCR kit to the Applied Biosystems™ 7500 Fast PCR Instrument and Applied Biosystems RapidFinder™ Express 2.0 software allowing use of the assay on a 96-well format PCR cycler in addition to the current workflow, using the 24-well Thermo Scientific PikoReal™ PCR Instrument and Thermo Scientific SureTect software. The method modification study presented in this report was assessed by the AOAC-RI as being a level 2 method modification study, necessitating a method developer study on a representative range of food matrixes covering raw ground turkey, 2% fat pasteurized milk, and bagged lettuce as well as stainless steel surface samples. All testing was conducted in comparison to the reference method detailed in International Organization for Standardization (ISO) 6579:2002. No significant difference by probability of detection statistical analysis was found between the SureTect Listeria species PCR Assay or the ISO reference method methods for any of the three food matrixes and the surface samples analyzed during the study.

Home Learning in Times of COVID: Experiences of Parents

2020 ◽  
Vol 7 (1) ◽  
pp. 9 ◽  
Author(s):  
Shelina Bhamani ◽  
Areeba Zainab Makhdoom ◽  
Vardah Bharuchi ◽  
Nasreen Ali ◽  
Sidra Kaleem ◽  
...  
Keyword(s):  
Real Life ◽  
Sampling Technique ◽  
Online Classes ◽  
Google Docs ◽  
Learning Gap ◽  
Wide Range ◽  
Home Learning ◽  
Collection Data ◽  
Learning At Home ◽  
At Home

<p align="center"><em>The widespread prevalence of COVID-19 pandemic has affected academia and parents alike. Due to the sudden closure of schools, students are missing social interaction which is vital for better learning and grooming while most schools have started online classes. This has become a tough routine for the parents working online at home since they have to ensure their children’s education. The study presented was designed to explore the experiences of home learning in times of COVID-19. A descriptive qualitative study was planned to explore the experiences of parents about home learning and management during COVID-19 to get an insight into real-life experiences.  Purposive sampling technique was used for data collection.  Data were collected from 19 parents falling in the inclusion criteria. Considering the lockdown problem, the data were collected via Google docs form with open-ended questions related to COVID-19 and home learning. Three major themes emerged after the data analysis: impact of COVID on children learning; support given by schools; and strategies used by caregivers at home to support learning. It was analyzed that the entire nation and academicians around the world have come forward to support learning at home offering a wide range of free online avenues to support parents to facilitate home-learning. Furthermore, parents too have adapted quickly to address the learning gap that have emerged in their children’s learning in these challenging times. Measures should be adopted to provide essential learning skills to children at home. Centralized data dashboards and educational technology may be used to keep the students, parents and schools updated.</em></p>

scholarly journals Tunable characteristics of low-frequency bandgaps in two-dimensional multivibrator phononic crystal plates under prestrain

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Hai-Fei Zhu ◽  
Xiao-Wei Sun ◽  
Ting Song ◽  
Xiao-Dong Wen ◽  
Xi-Xuan Liu ◽  
...  
Keyword(s):  
Phononic Crystal ◽  
Real Life ◽  
Low Frequency ◽  
Acoustic Vibration ◽  
Crystal Plate ◽  
Frequency Noise ◽  
Two Dimensional ◽  
The Matrix ◽  
Noise Frequency ◽  
Realtime Control

AbstractIn view of the influence of variability of low-frequency noise frequency on noise prevention in real life, we present a novel two-dimensional tunable phononic crystal plate which is consisted of lead columns deposited in a silicone rubber plate with periodic holes and calculate its bandgap characteristics by finite element method. The low-frequency bandgap mechanism of the designed model is discussed simultaneously. Accordingly, the influence of geometric parameters of the phononic crystal plate on the bandgap characteristics is analyzed and the bandgap adjustability under prestretch strain is further studied. Results show that the new designed phononic crystal plate has lower bandgap starting frequency and wider bandwidth than the traditional single-sided structure, which is due to the coupling between the resonance mode of the scatterer and the long traveling wave in the matrix with the introduction of periodic holes. Applying prestretch strain to the matrix can realize active realtime control of low-frequency bandgap under slight deformation and broaden the low-frequency bandgap, which can be explained as the multiple bands tend to be flattened due to the localization degree of unit cell vibration increases with the rise of prestrain. The presented structure improves the realtime adjustability of sound isolation and vibration reduction frequency for phononic crystal in complex acoustic vibration environments.

scholarly journals A Systematic Review and Qualitative Synthesis Resulting in a Typology of Elementary Classroom Movement Integration Interventions

2020 ◽  
Vol 6 (1) ◽  
Author(s):  
Spyridoula Vazou ◽  
Collin A. Webster ◽  
Gregory Stewart ◽  
Priscila Candal ◽  
Cate A. Egan ◽  
...  
Keyword(s):  
Physical Activity ◽  
Teacher Collaboration ◽  
A Priori ◽  
Real Life ◽  
Dose Intensity ◽  
Routine Practice ◽  
Qualitative Synthesis ◽  
Wide Range ◽  
Meta Analyses ◽  
Movement Integration

Abstract Background/Objective Movement integration (MI) involves infusing physical activity into normal classroom time. A wide range of MI interventions have succeeded in increasing children’s participation in physical activity. However, no previous research has attempted to unpack the various MI intervention approaches. Therefore, this study aimed to systematically review, qualitatively analyze, and develop a typology of MI interventions conducted in primary/elementary school settings. Subjects/Methods Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed to identify published MI interventions. Irrelevant records were removed first by title, then by abstract, and finally by full texts of articles, resulting in 72 studies being retained for qualitative analysis. A deductive approach, using previous MI research as an a priori analytic framework, alongside inductive techniques were used to analyze the data. Results Four types of MI interventions were identified and labeled based on their design: student-driven, teacher-driven, researcher-teacher collaboration, and researcher-driven. Each type was further refined based on the MI strategies (movement breaks, active lessons, other: opening activity, transitions, reward, awareness), the level of intrapersonal and institutional support (training, resources), and the delivery (dose, intensity, type, fidelity). Nearly half of the interventions were researcher-driven, which may undermine the sustainability of MI as a routine practice by teachers in schools. An imbalance is evident on the MI strategies, with transitions, opening and awareness activities, and rewards being limitedly studied. Delivery should be further examined with a strong focus on reporting fidelity. Conclusions There are distinct approaches that are most often employed to promote the use of MI and these approaches may often lack a minimum standard for reporting MI intervention details. This typology may be useful to effectively translate the evidence into practice in real-life settings to better understand and study MI interventions.

Sign in / Sign up

Export Citation Format

Share Document