scholarly journals Creating efficiencies in the extraction of data from randomized trials: a prospective evaluation of a machine learning and text mining tool

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Allison Gates ◽  
Michelle Gates ◽  
Shannon Sim ◽  
Sarah A. Elliott ◽  
Jennifer Pillay ◽  
...  
Keyword(s):  
Machine Learning ◽  
Text Mining ◽  
Randomized Trials ◽  
Data Extraction ◽  
Learning Tools ◽  
Time Savings ◽  
Mining Tool ◽  
Data Elements ◽  
Text Mining Tool ◽  
Manual Extraction

Abstract Background Machine learning tools that semi-automate data extraction may create efficiencies in systematic review production. We evaluated a machine learning and text mining tool’s ability to (a) automatically extract data elements from randomized trials, and (b) save time compared with manual extraction and verification. Methods For 75 randomized trials, we manually extracted and verified data for 21 data elements. We uploaded the randomized trials to an online machine learning and text mining tool, and quantified performance by evaluating its ability to identify the reporting of data elements (reported or not reported), and the relevance of the extracted sentences, fragments, and overall solutions. For each randomized trial, we measured the time to complete manual extraction and verification, and to review and amend the data extracted by the tool. We calculated the median (interquartile range [IQR]) time for manual and semi-automated data extraction, and overall time savings. Results The tool identified the reporting (reported or not reported) of data elements with median (IQR) 91% (75% to 99%) accuracy. Among the top five sentences for each data element at least one sentence was relevant in a median (IQR) 88% (83% to 99%) of cases. Among a median (IQR) 90% (86% to 97%) of relevant sentences, pertinent fragments had been highlighted by the tool; exact matches were unreliable (median (IQR) 52% [33% to 73%]). A median 48% of solutions were fully correct, but performance varied greatly across data elements (IQR 21% to 71%). Using ExaCT to assist the first reviewer resulted in a modest time savings compared with manual extraction by a single reviewer (17.9 vs. 21.6 h total extraction time across 75 randomized trials). Conclusions Using ExaCT to assist with data extraction resulted in modest gains in efficiency compared with manual extraction. The tool was reliable for identifying the reporting of most data elements. The tool’s ability to identify at least one relevant sentence and highlight pertinent fragments was generally good, but changes to sentence selection and/or highlighting were often required. Protocol https://doi.org/10.7939/DVN/RQPJKS

scholarly journals Creating Efficiencies in the Extraction of Data From Randomized Trials: A Prospective Evaluation of a Machine Learning and Text Mining Tool

2021 ◽  
Author(s):  
Allison Gates ◽  
Michelle Gates ◽  
Shannon Sim ◽  
Sarah A. Elliott ◽  
Jennifer Pillay ◽  
...  
Keyword(s):  
Machine Learning ◽  
Text Mining ◽  
Randomized Trials ◽  
Data Extraction ◽  
Data Element ◽  
Time Savings ◽  
Mining Tool ◽  
Data Elements ◽  
Text Mining Tool ◽  
Manual Extraction

Background. Machine learning tools that semi-automate data extraction may create efficiencies in systematic review production. We prospectively evaluated an online machine learning and text mining tool’s ability to (a) automatically extract data elements from randomized trials, and (b) save time compared with manual extraction and verification. Methods. For 75 randomized trials published in 2017, we manually extracted and verified data for 21 unique data elements. We uploaded the randomized trials to ExaCT, an online machine learning and text mining tool, and quantified performance by evaluating the tool’s ability to identify the reporting of data elements (reported or not reported), and the relevance of the extracted sentences, fragments, and overall solutions. For each randomized trial, we measured the time to complete manual extraction and verification, and to review and amend the data extracted by ExaCT (simulating semi-automated data extraction). We summarized the relevance of the extractions for each data element using counts and proportions, and calculated the median and interquartile range (IQR) across data elements. We calculated the median (IQR) time for manual and semiautomated data extraction, and overall time savings. Results. The tool identified the reporting (reported or not reported) of data elements with median (IQR) 91 percent (75% to 99%) accuracy. Performance was perfect for four data elements: eligibility criteria, enrolment end date, control arm, and primary outcome(s). Among the top five sentences for each data element at least one sentence was relevant in a median (IQR) 88 percent (83% to 99%) of cases. Performance was perfect for four data elements: funding number, registration number, enrolment start date, and route of administration. Among a median (IQR) 90 percent (86% to 96%) of relevant sentences, pertinent fragments had been highlighted by the system; exact matches were unreliable (median (IQR) 52 percent [32% to 73%]). A median 48 percent of solutions were fully correct, but performance varied greatly across data elements (IQR 21% to 71%). Using ExaCT to assist the first reviewer resulted in a modest time savings compared with manual extraction by a single reviewer (17.9 vs. 21.6 hours total extraction time across 75 randomized trials). Conclusions. Using ExaCT to assist with data extraction resulted in modest gains in efficiency compared with manual extraction. The tool was reliable for identifying the reporting of most data elements. The tool’s ability to identify at least one relevant sentence and highlight pertinent fragments was generally good, but changes to sentence selection and/or highlighting were often required.

scholarly journals MACE2K: A Text-Mining Tool to Extract Literature-based Evidence for Variant Interpretation using Machine Learning

2020 ◽  
Author(s):  
Samir Gupta ◽  
Shruti Rao ◽  
Trisha Miglani ◽  
Yasaswini Iyer ◽  
Junxia Lin ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Text Mining ◽  
Genomic Medicine ◽  
Relevant Information ◽  
Biomedical Literature ◽  
Variant Interpretation ◽  
Learning Models ◽  
Mining Tool ◽  
Text Mining Tool

AbstractInterpretation of a given variant’s pathogenicity is one of the most profound challenges to realizing the promise of genomic medicine. A large amount of information about associations between variants and diseases used by curators and researchers for interpreting variant pathogenicity is buried in biomedical literature. The development of text-mining tools that can extract relevant information from the literature will speed up and assist the variant interpretation curation process. In this work, we present a text-mining tool, MACE2k that extracts evidence sentences containing associations between variants and diseases from full-length PMC Open Access articles. We use different machine learning models (classical and deep learning) to identify evidence sentences with variant-disease associations. Evaluation shows promising results with the best F1-score of 82.9% and AUC-ROC of 73.9%. Classical ML models had a better recall (96.6% for Random Forest) compared to deep learning models. The deep learning model, Convolutional Neural Network had the best precision (75.6%), which is essential for any curation task.

scholarly journals MedMiner: An Internet Text-Mining Tool for Biomedical Information, with Application to Gene Expression Profiling

BioTechniques ◽  
1999 ◽  
Vol 27 (6) ◽  
pp. 1210-1217 ◽  
Author(s):  
L. Tanabe ◽  
U. Scherf ◽  
L.H. Smith ◽  
J.K. Lee ◽  
L. Hunter ◽  
...  
Keyword(s):  
Gene Expression ◽  
Text Mining ◽  
Gene Expression Profiling ◽  
Expression Profiling ◽  
Mining Tool ◽  
Text Mining Tool

scholarly journals SicknessMiner: a deep-learning-driven text-mining tool to abridge disease-disease associations

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Nícia Rosário-Ferreira ◽  
Victor Guimarães ◽  
Vítor S. Costa ◽  
Irina S. Moreira
Keyword(s):  
Text Mining ◽  
Named Entity Recognition ◽  
Entity Recognition ◽  
Named Entity ◽  
Disease Similarity ◽  
Disease Associations ◽  
Named Entity Normalization ◽  
Mining Tool ◽  
Or Gene ◽  
Text Mining Tool

Abstract Background Blood cancers (BCs) are responsible for over 720 K yearly deaths worldwide. Their prevalence and mortality-rate uphold the relevance of research related to BCs. Despite the availability of different resources establishing Disease-Disease Associations (DDAs), the knowledge is scattered and not accessible in a straightforward way to the scientific community. Here, we propose SicknessMiner, a biomedical Text-Mining (TM) approach towards the centralization of DDAs. Our methodology encompasses Named Entity Recognition (NER) and Named Entity Normalization (NEN) steps, and the DDAs retrieved were compared to the DisGeNET resource for qualitative and quantitative comparison. Results We obtained the DDAs via co-mention using our SicknessMiner or gene- or variant-disease similarity on DisGeNET. SicknessMiner was able to retrieve around 92% of the DisGeNET results and nearly 15% of the SicknessMiner results were specific to our pipeline. Conclusions SicknessMiner is a valuable tool to extract disease-disease relationship from RAW input corpus.

Text mining tool for translating terms of contract into technical specifications: Development and application in the railway sector

2021 ◽  
Vol 124 ◽  
pp. 103357
Author(s):  
G. Fantoni ◽  
E. Coli ◽  
F. Chiarello ◽  
R. Apreda ◽  
F. Dell’Orletta ◽  
...  
Keyword(s):  
Text Mining ◽  
Technical Specifications ◽  
Mining Tool ◽  
Text Mining Tool

scholarly journals Testing a text mining tool for emerging risk identification

2016 ◽  
Vol 13 (12) ◽  
Author(s):  
Niels B. Lucas Luijckx ◽  
Fred J. van de Brug ◽  
Winfried R. Leeman ◽  
Jos M.B.M. van der Vossen ◽  
Hilde J. Cnossen
Keyword(s):  
Text Mining ◽  
Risk Identification ◽  
Emerging Risk ◽  
Mining Tool ◽  
Text Mining Tool

scholarly journals Extracting kinetic information from literature with KineticRE

2015 ◽  
Vol 12 (4) ◽  
pp. 56-68
Author(s):  
Ana Alão Freitas ◽  
Hugo Costa ◽  
Isabel Rocha
Keyword(s):  
Text Mining ◽  
Metabolic Networks ◽  
Scientific Literature ◽  
Kluyveromyces Lactis ◽  
Relevant Information ◽  
Text Documents ◽  
Kinetic Information ◽  
Mining Tool ◽  
Text Mining Tool

Summary To better understand the dynamic behavior of metabolic networks in a wide variety of conditions, the field of Systems Biology has increased its interest in the use of kinetic models. The different databases, available these days, do not contain enough data regarding this topic. Given that a significant part of the relevant information for the development of such models is still wide spread in the literature, it becomes essential to develop specific and powerful text mining tools to collect these data. In this context, this work has as main objective the development of a text mining tool to extract, from scientific literature, kinetic parameters, their respective values and their relations with enzymes and metabolites. The approach proposed integrates the development of a novel plug-in over the text mining framework @Note2. In the end, the pipeline developed was validated with a case study on Kluyveromyces lactis, spanning the analysis and results of 20 full text documents.

Sign in / Sign up

Export Citation Format

Share Document