scholarly journals An Immersive and Interactive Platform for Cognitive Assessment and Rehabilitation (bWell): Design and Iterative Development Process

10.2196/26629 ◽  
2021 ◽  
Vol 8 (4) ◽  
pp. e26629
Author(s):  
Vincent Gagnon Shaigetz ◽  
Catherine Proulx ◽  
Anne Cabral ◽  
Nusrat Choudhury ◽  
Mark Hewko ◽  
...  
Keyword(s):  
Cognitive Assessment ◽  
Development Process ◽  
Clinical Care ◽  
Self Regulation ◽  
National Research Council ◽  
Cognitive Domain ◽  
Developmental Approach ◽  
Domain Experts ◽  
Health Technologies ◽  
Immersive Technologies

Background Immersive technologies like virtual reality can enable clinical care that meaningfully aligns with real-world deficits in cognitive functioning. However, options in immersive 3D environments are limited, partly because of the unique challenges presented by the development of a clinical care platform. These challenges include selecting clinically relevant features, enabling tasks that capture the full breadth of deficits, ensuring longevity in a rapidly changing technology landscape, and performing the extensive technical and clinical validation required for digital interventions. Complicating development, is the need to integrate recommendations from domain experts at all stages. Objective The Cognitive Health Technologies team at the National Research Council Canada aims to overcome these challenges with an iterative process for the development of bWell, a cognitive care platform providing multisensory cognitive tasks for adoption by treatment providers. Methods The team harnessed the affordances of immersive technologies while taking an interdisciplinary research and developmental approach, obtaining active input from domain experts with iterative deliveries of the platform. The process made use of technology readiness levels, agile software development, and human-centered design to advance four main activities: identification of basic requirements and key differentiators, prototype design and foundational research to implement components, testing and validation in lab settings, and recruitment of external clinical partners. Results bWell was implemented according to the findings from the design process. The main features of bWell include multimodal (fully, semi, or nonimmersive) and multiplatform (extended reality, mobile, and PC) implementation, configurable exercises that pair standardized assessment with adaptive and gamified variants for therapy, a therapist-facing user interface for task administration and dosing, and automated activity data logging. bWell has been designed to serve as a broadly applicable toolkit, targeting general aspects of cognition that are commonly impacted across many disorders, rather than focusing on 1 disorder or a specific cognitive domain. It comprises 8 exercises targeting different domains: states of attention (Egg), visual working memory (Theater), relaxation (Tent), inhibition and cognitive control (Mole), multitasking (Lab), self-regulation (Butterfly), sustained attention (Stroll), and visual search (Cloud). The prototype was tested and validated with healthy adults in a laboratory environment. In addition, a cognitive care network (5 sites across Canada and 1 in Japan) was established, enabling access to domain expertise and providing iterative input throughout the development process. Conclusions Implementing an interdisciplinary and iterative approach considering technology maturity brought important considerations for the development of bWell. Altogether, this harnesses the affordances of immersive technology and design for a broad range of applications, and for use in both cognitive assessment and rehabilitation. The technology has attained a maturity level of prototype implementation with preliminary validation carried out in laboratory settings, with next steps to perform the validation required for its eventual adoption as a clinical tool.

scholarly journals An Immersive and Interactive Platform for Cognitive Assessment and Rehabilitation (bWell): Design and Iterative Development Process (Preprint)

2020 ◽  
Author(s):  
Vincent Gagnon Shaigetz ◽  
Catherine Proulx ◽  
Anne Cabral ◽  
Nusrat Choudhury ◽  
Mark Hewko ◽  
...  
Keyword(s):  
Cognitive Assessment ◽  
Development Process ◽  
Clinical Care ◽  
Self Regulation ◽  
National Research Council ◽  
Cognitive Domain ◽  
Developmental Approach ◽  
Domain Experts ◽  
Health Technologies ◽  
Immersive Technologies

BACKGROUND Immersive technologies like virtual reality can enable clinical care that meaningfully aligns with real-world deficits in cognitive functioning. However, options in immersive 3D environments are limited, partly because of the unique challenges presented by the development of a clinical care platform. These challenges include selecting clinically relevant features, enabling tasks that capture the full breadth of deficits, ensuring longevity in a rapidly changing technology landscape, and performing the extensive technical and clinical validation required for digital interventions. Complicating development, is the need to integrate recommendations from domain experts at all stages. OBJECTIVE The Cognitive Health Technologies team at the National Research Council Canada aims to overcome these challenges with an iterative process for the development of bWell, a cognitive care platform providing multisensory cognitive tasks for adoption by treatment providers. METHODS The team harnessed the affordances of immersive technologies while taking an interdisciplinary research and developmental approach, obtaining active input from domain experts with iterative deliveries of the platform. The process made use of technology readiness levels, agile software development, and human-centered design to advance four main activities: identification of basic requirements and key differentiators, prototype design and foundational research to implement components, testing and validation in lab settings, and recruitment of external clinical partners. RESULTS bWell was implemented according to the findings from the design process. The main features of bWell include multimodal (fully, semi, or nonimmersive) and multiplatform (extended reality, mobile, and PC) implementation, configurable exercises that pair standardized assessment with adaptive and gamified variants for therapy, a therapist-facing user interface for task administration and dosing, and automated activity data logging. bWell has been designed to serve as a broadly applicable toolkit, targeting general aspects of cognition that are commonly impacted across many disorders, rather than focusing on 1 disorder or a specific cognitive domain. It comprises 8 exercises targeting different domains: states of attention (Egg), visual working memory (Theater), relaxation (Tent), inhibition and cognitive control (Mole), multitasking (Lab), self-regulation (Butterfly), sustained attention (Stroll), and visual search (Cloud). The prototype was tested and validated with healthy adults in a laboratory environment. In addition, a cognitive care network (5 sites across Canada and 1 in Japan) was established, enabling access to domain expertise and providing iterative input throughout the development process. CONCLUSIONS Implementing an interdisciplinary and iterative approach considering technology maturity brought important considerations for the development of bWell. Altogether, this harnesses the affordances of immersive technology and design for a broad range of applications, and for use in both cognitive assessment and rehabilitation. The technology has attained a maturity level of prototype implementation with preliminary validation carried out in laboratory settings, with next steps to perform the validation required for its eventual adoption as a clinical tool.

Characterizing cognitive deficits and potential predictors in multiple sclerosis: A large nationwide study applying Brief International Cognitive Assessment for Multiple Sclerosis in standard clinical care

2020 ◽  
Vol 14 (3) ◽  
pp. 347-369 ◽  
Author(s):  
Alina Renner ◽  
Sharon J. Baetge ◽  
Melanie Filser ◽  
Sebastian Ullrich ◽  
Christoph Lassek ◽  
...  
Keyword(s):  
Multiple Sclerosis ◽  
Cognitive Deficits ◽  
Cognitive Assessment ◽  
Clinical Care ◽  
Nationwide Study

Validation of the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) in the Russian Population

2021 ◽  
pp. 1-8
Author(s):  
Evgeniy Evdoshenko ◽  
Kristina Laskova ◽  
Maria Shumilina ◽  
Ekaterina Nekrashevich ◽  
Maria Andreeva ◽  
...  
Keyword(s):  
Multiple Sclerosis ◽  
Cognitive Assessment ◽  
Verbal Learning ◽  
Cognitive Status ◽  
Russian Population ◽  
Cognitive Domain ◽  
Control Group ◽  
Retest Reliability ◽  
Valid Instrument ◽  
Test Retest Reliability

Abstract Objective: Cognitive dysfunction is common in multiple sclerosis (MS). The Brief International Cognitive Assessment for MS (BICAMS) battery of tests has been suggested as a measure for the evaluation of the cognitive status of MS patients. This study aims to validate the BICAMS battery in the Russian population of MS patients. Methods: Age- and sex-matched MS patients (n = 98) and healthy individuals (n = 86) were included in the study. Symbol Digit Modalities Test (SDMT), California Verbal Learning Test, 2nd edition (CVLT-II) and the Brief Visuospatial Memory Test – Revised (BVMT-R) were administered to all participants. The battery was readministered 1 month later to 44 MS patients to investigate the test–retest reliability. Results: MS patients exhibited a significantly lower performance in testing with BICAMS than the control group in all three neuropsychological tests. Test–retest reliability was good for SDMT and CVLT-II (r = .82 and r = .85, respectively) and adequate for BVMT-R (r = .70). Based on the proposed criterion for impairment as z score below 1.5 SD the mean of the control group, we found that 34/98 (35%) of MS patients were found impaired at least in one cognitive domain. Patients with Expanded Disability Status Scale score ≥3.5 performed significantly worse than controls (SDMT, p < .0001; CVLT–II, p = .03; BVMT-R, p = .0004), while those with ≤3.0 scores did not. Conclusion: This study demonstrates that the BICAMS battery is a valid instrument to identify cognitive impairment in MS patients and it can be recommended for routine use in the Russian Federation.

scholarly journals A Survey of Domain Knowledge Elicitation in Applied Machine Learning

2021 ◽  
Vol 5 (12) ◽  
pp. 73
Author(s):  
Daniel Kerrigan ◽  
Jessica Hullman ◽  
Enrico Bertini
Keyword(s):  
Machine Learning ◽  
Domain Knowledge ◽  
Development Process ◽  
Model Development ◽  
Knowledge Elicitation ◽  
Future Directions ◽  
Domain Experts ◽  
Applied Machine Learning ◽  
Elicitation Process ◽  
Model Development Process

Eliciting knowledge from domain experts can play an important role throughout the machine learning process, from correctly specifying the task to evaluating model results. However, knowledge elicitation is also fraught with challenges. In this work, we consider why and how machine learning researchers elicit knowledge from experts in the model development process. We develop a taxonomy to characterize elicitation approaches according to the elicitation goal, elicitation target, elicitation process, and use of elicited knowledge. We analyze the elicitation trends observed in 28 papers with this taxonomy and identify opportunities for adding rigor to these elicitation approaches. We suggest future directions for research in elicitation for machine learning by highlighting avenues for further exploration and drawing on what we can learn from elicitation research in other fields.

Initial experiences with virtual reality as a tool for observation in needs-driven health technology innovation

2020 ◽  
Vol 6 (1) ◽  
pp. 10-12
Author(s):  
Kenneth H Perrone ◽  
Katherine Sherman Blevins ◽  
Lyn Denend ◽  
Richard Fan ◽  
Justin Huelman ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Undergraduate Students ◽  
New Technology ◽  
Clinical Care ◽  
Patient Privacy ◽  
Privacy Concerns ◽  
Health Technologies ◽  
Clinical Observations ◽  
Clinical Scenarios ◽  
Hospital Policies

The Stanford University Biodesign Innovation Fellowship teaches a needs-based methodology for the innovation of health technologies. This involves the direct observation of patient care in a variety of settings, ranging from the hospital to the home, to identify unmet needs that can be addressed via innovative new technology-based solutions. Expanding this model to educate a larger population of undergraduate and graduate students is limited by access to real clinical observations, partly due to hospital policies and patient privacy concerns. We hypothesise that the use of virtual reality (VR) can be an effective tool to provide students access to a variety of clinical scenarios for identifying needs for innovation. In this preliminary study, two undergraduate students observed clinical care live in the operating room (OR) and using VR headsets. The students identified needs in both settings and compared the two experiences with a short survey. While VR did not offer a complete replication of the OR experience, it served as a viable tool for learning how to make observations. VR merits further investigation as an educational tool for needs finding and as a proxy for live clinical observations.

scholarly journals Test Accuracy of the Montreal Cognitive Assessment in Screening for Early Poststroke Neurocognitive Disorder

Stroke ◽  
2021 ◽  
Vol 52 (1) ◽  
pp. 317-320
Author(s):  
Ragnhild Munthe-Kaas ◽  
Stina Aam ◽  
Ingvild Saltvedt ◽  
Torgeir Bruun Wyller ◽  
Sarah T. Pendlebury ◽  
...  
Keyword(s):  
Mental Disorders ◽  
Cognitive Assessment ◽  
Test Battery ◽  
Montreal Cognitive Assessment ◽  
Cognitive Domain ◽  
Cognitive Test ◽  
Test Accuracy ◽  
Neurocognitive Disorder ◽  
Statistical Manual ◽  
Diagnostic And Statistical Manual

Background and Purpose: We determined the diagnostic accuracy of the Montreal Cognitive Assessment (MoCA) for poststroke neurocognitive disorder defined according to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, criteria in a prospective observational study. Methods: Consecutive participants able to complete a cognitive test battery and MoCA 3 months poststroke were included. The reference standard of neurocognitive disorder was defined as a score of ≥1.5 SD below the normative mean in ≥1 cognitive domain on the cognitive test battery. Results: Among 521 participants (43.6% women; mean age/SD, 71.5/12.0 years; mean education/SD, 12.4/3.8 years), the area under the receiver operating characteristic curve of MoCA for neurocognitive disorder was 0.80 (95% CI, 0.76–0.84). Using the standard MoCA cutoff <26, sensitivity was 0.71 (0.69–0.79) with specificity of 0.73 (0.66–0.76). MoCA cutoff of <27 gave higher sensitivity (0.82 [0.77–0.85]) at the expense of specificity (0.60 [0.53–0.66]). Discussion: MoCA has reasonable accuracy for poststroke neurocognitive disorder diagnosed using the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, criteria. Registration: URL: https://www.clinicaltrials.gov . Unique identifier: NCT02650531.

A Fuzzy Logic-Based Model for Classifying Software Modules in Order to Achieve Dependable Software

2020 ◽  
Vol 11 (4) ◽  
pp. 45-57
Author(s):  
Subhashis Chatterjee ◽  
Bappa Maji
Keyword(s):  
Fuzzy Logic ◽  
Software Metrics ◽  
Development Process ◽  
Software Developers ◽  
Data Set ◽  
Domain Experts ◽  
Software Dependability ◽  
Proposed Model ◽  
Software Modules ◽  
Software Lifecycle

In this study, a fuzzy logic-based framework has been proposed to predict the situation of the software modules in the earlier phases of software lifecycle. The proposed model has taken into account domain experts' opinions and the available state of different software metrics as inputs. On the basis of dependability measures of software, different modules have been ranked earlier in the development process. Effect of the modules on the reliability, security, and availability of software has been judged by the proposed technique based on Mahalanobis distance metric. The study of software dependability in early phase assists the software developers to take corrective actions, which leads to minimize the testing efforts as well as development time. The proposed technique has been implemented on the promise software engineering repository data set. Performance of the proposed methodology is promising in identifying the fault-prone software modules. The result has also been compared with some known methodologies.

scholarly journals Protocol for the development of versions of the Montreal Cognitive Assessment (MoCA) for people with hearing or vision impairment

BMJ Open ◽  
2019 ◽  
Vol 9 (3) ◽  
pp. e026246 ◽  
Author(s):  
Piers Dawes ◽  
Annie Pye ◽  
David Reeves ◽  
Wai Kent Yeung ◽  
Saima Sheikh ◽  
...  
Keyword(s):  
Cognitive Assessment ◽  
Clinical Care ◽  
Vision Impairment ◽  
Assessment Tools ◽  
Montreal Cognitive Assessment ◽  
Link Type ◽  
Conference Presentations ◽  
Ethical Approval ◽  
Twitter Account ◽  
The Uk

IntroductionHearing and vision impairments are highly prevalent among older adults and impact commonly used cognitive assessment tools for the identification of dementia. Adaptations of such tests for people with hearing or vision impairment have not been adequately validated among populations with such sensory impairment.Methods and analysisWe will develop two versions of the Montreal Cognitive Assessment (MoCA) for people with acquired hearing impairment (MoCA-H) or vision impairment (MoCA-V). The MoCA-H and MoCA-V will exclude the existing MoCA items that are presented in spoken or visual format, respectively, and include new suitably adapted items. Participants (n=792) with combinations of hearing, vision and cognitive impairment will complete standard or adapted versions of the MoCA across three language sites (English, French and Greek). Development of the MoCA-H and the MoCA-V will be based on analysis of adapted and standard MoCA items following model-based development to select the combination of items for the MoCA-H and MoCA-V that provide optimal sensitivity and specificity for detection of dementia.Ethics and disseminationThe study has received ethical approval from respective centres in the UK, France, Greece and Cyprus. The results of the study will be disseminated through peer-reviewed publication, conference presentations, the study website (https://www.sense-cog.eu/), the SENSE-Cog Twitter account (@sense_cog) and the MoCA test website (https://www.mocatest.org/). The main outputs of the study will be versions of the MoCA that are appropriate for use with adults with acquired hearing or vision impairment and will contribute significantly to the clinical care of older people.

scholarly journals The Radar Graph: the Development of an Educational Tool to Demonstrate Resident Competency

2012 ◽  
Vol 4 (2) ◽  
pp. 220-226 ◽  
Author(s):  
Drew M. Keister ◽  
Daniel Larson ◽  
Julie Dostal ◽  
Jay Baglia
Keyword(s):  
Construct Validity ◽  
Direct Observation ◽  
Development Process ◽  
Clinical Care ◽  
Assessment System ◽  
Competency Assessment ◽  
Promising Tool ◽  
Competency Based ◽  
The Individual ◽  
Medical Competence

Abstract Background Despite the movement toward competency-based assessment by accrediting bodies in recent years, there is no consensus on how to best assess medical competence. Direct observation is a useful tool. At the same time, a comprehensive assessment system based on direct observation has been difficult to develop. Intervention We developed a system that translates data obtained from checklists of observed behaviors completed during educational activities, including direct observation of clinical care, into a graphic tool (the “radar graph”) usable for both formative and summative assessment. Using unique, observable behaviors to evaluate levels of competency on the Dreyfus scale, we assessed resident performance in 6 learning sites within our residency. Data are represented on a radar graph, which residents and faculty used to recognize both strengths and areas for growth to guide educational planning for the individual learner. Results Initial data show that the radar graphs have construct validity because the development process accurately reflects the desired construct, assessors were adequately trained, and the radar graphs demonstrated resident growth over time. A form completion rate of 90% for &gt;1500 disseminated assessments suggests the feasibility of our process. Conclusions The radar graph is a promising tool for use in resident feedback and competency assessment. Further research is needed to determine the full utility of the radar graphs, including a better understanding of the tool's reliability and construct validity.

Sign in / Sign up

Export Citation Format

Share Document