A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High Quality Data (Preprint)

2021 ◽  
Author(s):  
Victoria Leong ◽  
Kausar Raheel ◽  
Jia Yi Sim ◽  
Kriti Kacker ◽  
Vasilis M Karlaftis ◽  
...  
Keyword(s):  
Data Quality ◽  
Response Times ◽  
Psychological Research ◽  
Cognitive Testing ◽  
Quality Data ◽  
High Quality ◽  
Online Data ◽  
Test Environment ◽  
Web Based ◽  
Cognitive Data

BACKGROUND The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted both safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the fore as a promising solution for rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. OBJECTIVE Here, we examine the opportunities and challenges afforded by the societal shift toward web-based testing, highlight an urgent need to establish a standard data quality assurance framework for online studies, and develop and validate a new supervised online testing methodology, remote guided testing (RGT). METHODS A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (N=41) or online using remote guided testing (N=44), delivered using identical web-based platforms (CANTAB, Inquisit and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, response times), as well as overall task performance measures. RESULTS The results indicated that, across all measures of data quality and performance, RGT data was statistically-equivalent to data collected in person in the lab. Moreover, RGT participants out-performed the lab group on measured verbal intelligence, which could reflect test environment differences, including possible effects of mask-wearing on communication. CONCLUSIONS These data suggest that the RGT methodology could help to ameliorate concerns regarding online data quality and - particularly for studies involving high-risk or rare cohorts - offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance. CLINICALTRIAL N.A.

scholarly journals A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High Quality Data

2021 ◽  
Author(s):  
Victoria Leong ◽  
Kausar Raheel ◽  
Sim Jia Yi ◽  
Kriti Kacker ◽  
Vasilis M. Karlaftis ◽  
...  
Keyword(s):  
Data Quality ◽  
Response Times ◽  
Psychological Research ◽  
Cognitive Testing ◽  
Quality Data ◽  
High Quality ◽  
Online Data ◽  
Test Environment ◽  
Web Based ◽  
Cognitive Data

Background. The global COVID-19 pandemic has triggered a fundamental reexamination of how human psychological research can be conducted both safely and robustly in a new era of digital working and physical distancing. Online web-based testing has risen to the fore as a promising solution for rapid mass collection of cognitive data without requiring human contact. However, a long-standing debate exists over the data quality and validity of web-based studies. Here, we examine the opportunities and challenges afforded by the societal shift toward web-based testing, highlight an urgent need to establish a standard data quality assurance framework for online studies, and develop and validate a new supervised online testing methodology, remote guided testing (RGT). Methods. A total of 85 healthy young adults were tested on 10 cognitive tasks assessing executive functioning (flexibility, memory and inhibition) and learning. Tasks were administered either face-to-face in the laboratory (N=41) or online using remote guided testing (N=44), delivered using identical web-based platforms (CANTAB, Inquisit and i-ABC). Data quality was assessed using detailed trial-level measures (missed trials, outlying and excluded responses, response times), as well as overall task performance measures. Results. The results indicated that, across all measures of data quality and performance, RGT data was statistically-equivalent to data collected in person in the lab. Moreover, RGT participants out-performed the lab group on measured verbal intelligence, which could reflect test environment differences, including possible effects of mask-wearing on communication. Conclusions. These data suggest that the RGT methodology could help to ameliorate concerns regarding online data quality and - particularly for studies involving high-risk or rare cohorts - offer an alternative for collecting high-quality human cognitive data without requiring in-person physical attendance.

scholarly journals Methods to Detect Low Quality Data and Its Implication for Psychological Research

2017 ◽  
Author(s):  
Erin Michelle Buchanan ◽  
John E. Scofield
Keyword(s):  
Data Collection ◽  
Response Times ◽  
Psychological Research ◽  
Sensitivity Analyses ◽  
Detection Methods ◽  
Quality Data ◽  
Screening Methods ◽  
Data Detection ◽  
Online Data ◽  
Survey Responses

Web-based data collection methods such as Amazon's Mechanical Turk (AMT) are an appealing option to recruit participants quickly and cheaply for psychological research. While concerns regarding data quality have emerged with AMT, several studies have exhibited that data collected via AMT are as reliable as traditional college samples and are often more diverse and representative of noncollege populations. The development of methods to screen for low quality data, however, has been less explored. Omitting participants based on simple screening methods in isolation, such as response time or attention checks may not be adequate identification methods, with an inability to delineate between high or low effort participants. Additionally, problematic survey responses may arise from survey automation techniques such as survey bots or automated form fillers. The current project developed low quality data detection methods while overcoming previous screening limitations. Multiple checks were employed, such as page response times, distribution of survey responses, the number of utilized choices from a given range of scale options, click counts, and manipulation checks. This method was tested on a survey taken with an easily available plug-in survey bot, as well as compared to data collected by human participants providing both high effort and randomized, or low effort, answers. Identified cases can then be used as part of sensitivity analyses to warrant exclusion from further analyses. This algorithm can be a promising tool to identify low quality or automated data via AMT or other online data collection platforms.

scholarly journals A New Remote Guided Method for Supervised Web-Based Cognitive Testing to Ensure High Quality Data (Preprint)

10.2196/28368 ◽  
2021 ◽  
Author(s):  
Victoria Leong ◽  
Kausar Raheel ◽  
Jia Yi Sim ◽  
Kriti Kacker ◽  
Vasilis M Karlaftis ◽  
...  
Keyword(s):  
Cognitive Testing ◽  
Quality Data ◽  
High Quality ◽  
High Quality Data ◽  
Web Based

Abstract 205: Etiologic Ischemic Stroke Phenotypes in the NINDS Stroke Genetics Network

Stroke ◽  
2015 ◽  
Vol 46 (suppl_1) ◽  
Author(s):  
Hakan Ay ◽  
Ethem M Arsava ◽  
Robert D. Brown ◽  
Steven J Kittner ◽  
Jin-Moo Lee ◽  
...  
Keyword(s):  
Ischemic Stroke ◽  
Geographic Location ◽  
Quality Data ◽  
High Quality ◽  
Good Reliability ◽  
Web Based ◽  
Stroke Subtypes ◽  
Phenotype Data ◽  
Epidemiological Characteristics

Background and Purpose: NINDS Stroke Genetics Network (SiGN) is an international consortium of ischemic stroke studies that aims to generate high quality phenotype data to identify the genetic basis of ischemic stroke subtypes. The goal of this analysis is to characterize the etiopathogenetic basis of ischemic stroke in the consortium. Methods: This analysis included 16,954 subjects with imaging-confirmed ischemic stroke from 12 US studies and 11 studies from 8 European countries. 52 trained and certified adjudicators used the web-based Causative Classification of Stroke System for etiologic stroke classification through chart reviews to determine both phenotypic (abnormal test findings categorized in major etiologic groups without weighting towards the most likely cause in the presence of multiple etiologies) and causative subtypes in each subject. Classification reliability was assessed with blinded re-adjudication of 1509 randomly selected cases. Findings: The figure shows the distribution of etiologic categories. Overall, only 40% to 54% of cases with a given major ischemic stroke etiology (phenotypic subtype) were classified into the same final causative category with high confidence. There was good agreement for both causative (kappa 0·72, 95%CI:0·69-0·75) and phenotypic classifications (kappa 0·73, 95%CI:0·70-0·75). Conclusions: This study provides high quality data on etiologic stroke subtypes and demonstrates that etiologic subtypes can be determined with good reliability in studies that include investigators with different expertise and background, institutions with different stroke evaluation protocols and geographic location, and patient populations with different epidemiological characteristics. The discordance between phenotypic and causative stroke subtypes suggests that the presence of an abnormality in a stroke patient does not necessarily mean that it is the cause of stroke.

scholarly journals Exceptional Data Quality using Intelligent Matching and Retrieval

AI Magazine ◽  
2010 ◽  
Vol 31 (1) ◽  
pp. 65 ◽  
Author(s):  
Clint R. Bidlack ◽  
Michael P Wellman
Keyword(s):  
Data Quality ◽  
Real Time ◽  
Data Migration ◽  
Quality Data ◽  
Sales Managers ◽  
Web Based ◽  
Sales And Marketing ◽  
Retrieval Algorithms ◽  
Marketing Operations ◽  
User Friendly

Recent advances in enterprise web-based software have created a need for sophisticated yet user-friendly data quality solutions. A new category of data quality solutions are discussed that fill this need using intelligent matching and retrieval algorithms. Solutions are focused on customer and sales data and include real-time inexact search, batch processing, and data migration. Users are empowered to maintain higher quality data resulting in more efficient sales and marketing operations. Sales managers spend more time with customers and less time managing data.

The New Data Quality Task Group (DQTG): ensuring high quality data today and in the future

Metabolomics ◽  
2014 ◽  
Vol 10 (4) ◽  
pp. 539-540 ◽  
Author(s):  
Daniel W. Bearden ◽  
Richard D. Beger ◽  
David Broadhurst ◽  
Warwick Dunn ◽  
Arthur Edison ◽  
...  
Keyword(s):  
Data Quality ◽  
Task Group ◽  
Quality Data ◽  
High Quality ◽  
High Quality Data ◽  
The Future

scholarly journals A Systematic Framework for Analyzing Observation Data in Patient-Centered Registries: Case Study for Patients With Depression (Preprint)

2020 ◽  
Author(s):  
Maryam Zolnoori ◽  
Mark D Williams ◽  
William B Leasure ◽  
Kurt B Angstman ◽  
Che Ngufor
Keyword(s):  
Data Quality ◽  
Quality Data ◽  
Patient Centered ◽  
High Quality ◽  
Data Set ◽  
High Quality Data ◽  
Research Questions ◽  
Quality Issues ◽  
The Impact ◽  
Systematic Framework

BACKGROUND Patient-centered registries are essential in population-based clinical care for patient identification and monitoring of outcomes. Although registry data may be used in real time for patient care, the same data may further be used for secondary analysis to assess disease burden, evaluation of disease management and health care services, and research. The design of a registry has major implications for the ability to effectively use these clinical data in research. OBJECTIVE This study aims to develop a systematic framework to address the data and methodological issues involved in analyzing data in clinically designed patient-centered registries. METHODS The systematic framework was composed of 3 major components: visualizing the multifaceted and heterogeneous patient-centered registries using a data flow diagram, assessing and managing data quality issues, and identifying patient cohorts for addressing specific research questions. RESULTS Using a clinical registry designed as a part of a collaborative care program for adults with depression at Mayo Clinic, we were able to demonstrate the impact of the proposed framework on data integrity. By following the data cleaning and refining procedures of the framework, we were able to generate high-quality data that were available for research questions about the coordination and management of depression in a primary care setting. We describe the steps involved in converting clinically collected data into a viable research data set using registry cohorts of depressed adults to assess the impact on high-cost service use. CONCLUSIONS The systematic framework discussed in this study sheds light on the existing inconsistency and data quality issues in patient-centered registries. This study provided a step-by-step procedure for addressing these challenges and for generating high-quality data for both quality improvement and research that may enhance care and outcomes for patients. INTERNATIONAL REGISTERED REPORT DERR1-10.2196/18366

scholarly journals A Systematic Framework for Analyzing Observation Data in Patient-Centered Registries: Case Study for Patients With Depression

10.2196/18366 ◽  
2020 ◽  
Vol 9 (10) ◽  
pp. e18366
Author(s):  
Maryam Zolnoori ◽  
Mark D Williams ◽  
William B Leasure ◽  
Kurt B Angstman ◽  
Che Ngufor
Keyword(s):  
Data Quality ◽  
Quality Data ◽  
Patient Centered ◽  
High Quality ◽  
Data Set ◽  
High Quality Data ◽  
Research Questions ◽  
Quality Issues ◽  
The Impact ◽  
Systematic Framework

Background Patient-centered registries are essential in population-based clinical care for patient identification and monitoring of outcomes. Although registry data may be used in real time for patient care, the same data may further be used for secondary analysis to assess disease burden, evaluation of disease management and health care services, and research. The design of a registry has major implications for the ability to effectively use these clinical data in research. Objective This study aims to develop a systematic framework to address the data and methodological issues involved in analyzing data in clinically designed patient-centered registries. Methods The systematic framework was composed of 3 major components: visualizing the multifaceted and heterogeneous patient-centered registries using a data flow diagram, assessing and managing data quality issues, and identifying patient cohorts for addressing specific research questions. Results Using a clinical registry designed as a part of a collaborative care program for adults with depression at Mayo Clinic, we were able to demonstrate the impact of the proposed framework on data integrity. By following the data cleaning and refining procedures of the framework, we were able to generate high-quality data that were available for research questions about the coordination and management of depression in a primary care setting. We describe the steps involved in converting clinically collected data into a viable research data set using registry cohorts of depressed adults to assess the impact on high-cost service use. Conclusions The systematic framework discussed in this study sheds light on the existing inconsistency and data quality issues in patient-centered registries. This study provided a step-by-step procedure for addressing these challenges and for generating high-quality data for both quality improvement and research that may enhance care and outcomes for patients. International Registered Report Identifier (IRRID) DERR1-10.2196/18366

eMaintenance ontologies for data quality support

2015 ◽  
Vol 21 (3) ◽  
pp. 358-374 ◽  
Author(s):  
Mustafa Aljumaili ◽  
Karina Wandt ◽  
Ramin Karim ◽  
Phillip Tretten
Keyword(s):  
Data Quality ◽  
Production Process ◽  
Quality Data ◽  
Literature Study ◽  
High Quality ◽  
Control Data ◽  
Management Tools ◽  
High Quality Data ◽  
Content Type ◽  
Data Production

Purpose – The purpose of this paper is to explore the main ontologies related to eMaintenance solutions and to study their application area. The advantages of using these ontologies to improve and control data quality will be investigated. Design/methodology/approach – A literature study has been done to explore the eMaintenance ontologies in the different areas. These ontologies are mainly related to content structure and communication interface. Then, ontologies will be linked to each step of the data production process in maintenance. Findings – The findings suggest that eMaintenance ontologies can help to produce a high-quality data in maintenance. The suggested maintenance data production process may help to control data quality. Using these ontologies in every step of the process may help to provide management tools to provide high-quality data. Research limitations/implications – Based on this study, it can be concluded that further research could broaden the investigation to identify more eMaintenance ontologies. Moreover, studying these ontologies in more technical details may help to increase the understandability and the use of these standards. Practical implications – It has been concluded in this study that applying eMaintenance ontologies by companies needs additional cost and time. Also the lack or the ineffective use of eMaintenance tools in many enterprises is one of the limitations for using these ontologies. Originality/value – Investigating eMaintenance ontologies and connecting them to maintenance data production is important to control and manage the data quality in maintenance.

Data Quality Assessment

2009 ◽  
pp. 378-384 ◽  
Author(s):  
Juliusz L. Kulikowski
Keyword(s):  
Information Processing ◽  
Data Quality ◽  
Quality Assessment ◽  
Quality Data ◽  
Quality Of Data ◽  
High Quality ◽  
Data Quality Assessment ◽  
Systems Effectiveness ◽  
And Control ◽  
Education Science

For many years the fact that for a high information processing systems’ effectiveness high quality of data is not less important than high systems’ technological performance was not widely understood and accepted. The way to understanding the complexity of data quality notion was also long, as it will be shown below. However, a progress in modern information processing systems development is not possible without improvement of data quality assessment and control methods. Data quality is closely connected both with data form and value of information carried by the data. High-quality data can be understood as data having an appropriate form and containing valuable information. Therefore, at least two aspects of data are reflected in this notion: 1st - technical facility of data processing, and 2nd - usefulness of information supplied by the data in education, science, decision making, etc.

Sign in / Sign up

Export Citation Format

Share Document