Iterative guided machine learning-assisted systematic literature reviews: a diabetes case study

Abstract Background Systematic Reviews (SR), studies of studies, use a formal process to evaluate the quality of scientific literature and determine ensuing effectiveness from qualifying articles to establish consensus findings around a hypothesis. Their value is increasing as the conduct and publication of research and evaluation has expanded and the process of identifying key insights becomes more time consuming. Text analytics and machine learning (ML) techniques may help overcome this problem of scale while still maintaining the level of rigor expected of SRs. Methods In this article, we discuss an approach that uses existing examples of SRs to build and test a method for assisting the SR title and abstract pre-screening by reducing the initial pool of potential articles down to articles that meet inclusion criteria. Our approach differs from previous approaches to using ML as a SR tool in that it incorporates ML configurations guided by previously conducted SRs, and human confirmation on ML predictions of relevant articles during multiple iterative reviews on smaller tranches of citations. We applied the tailored method to a new SR review effort to validate performance. Results The case study test of the approach proved a sensitivity (recall) in finding relevant articles during down selection that may rival many traditional processes and show ability to overcome most type II errors. The study achieved a sensitivity of 99.5% (213 out of 214) of total relevant articles while only conducting a human review of 31% of total articles available for review. Conclusions We believe this iterative method can help overcome bias in initial ML model training by having humans reinforce ML models with new and relevant information, and is an applied step towards transfer learning for ML in SR.

Download Full-text

Iterative Guided Machine Learning-Assisted Systematic Literature Reviews: A Diabetes Case Study

10.21203/rs.3.rs-48078/v2 ◽

2021 ◽

Author(s):

John Zimmerman ◽

Robin Soler ◽

Lavinder James ◽

Murphy Sarah ◽

Atkins Charisma ◽

...

Keyword(s):

Machine Learning ◽

Scientific Literature ◽

Relevant Information ◽

Text Analytics ◽

Inclusion Criteria ◽

Literature Reviews ◽

Study Test ◽

Model Training

Abstract Background: Systematic Reviews (SR), studies of studies, use a formal process to evaluate the quality of scientific literature and determine ensuing effectiveness from qualifying articles to establish consensus findings around a hypothesis. Their value is increasing as the conduct and publication of research and evaluation has expanded and the process of identifying key insights becomes more time consuming. Text analytics and Machine Learning (ML) techniques may help overcome this problem of scale while still maintaining the level of rigor expected of SRs.Methods: In this article, we discuss an approach that uses existing examples of SRs to build and test a method for assisting the SR title and abstract pre-screening by reducing the initial pool of potential articles down to articles that meet inclusion criteria. Our approach differs from previous approaches to using ML as a SR tool in that it incorporates ML configurations guided by previously conducted SRs, and human confirmation on ML predictions of relevant articles during multiple iterative reviews on smaller tranches of citations. We applied the tailored method to a new SR review effort to validate performance. Results: the case study test of the approach proved a sensitivity (recall) in finding relevant articles during down selection that may rival many traditional processes. Conclusions: We believe this iterative method can help overcome bias in initial ML model training by having humans reinforce ML models with new and relevant information, and is an applied step towards transfer learning for ML in SR.

Download Full-text

Iterative Guided Machine Learning-Assisted Systematic Literature Reviews: A Diabetes Case Study

10.21203/rs.3.rs-48078/v1 ◽

2020 ◽

Author(s):

John Zimmerman ◽

Robin Soler ◽

Lavender James ◽

Murphy Sarah ◽

Atkins Charisma ◽

...

Keyword(s):

Machine Learning ◽

Scientific Literature ◽

Relevant Information ◽

Text Analytics ◽

Inclusion Criteria ◽

Literature Reviews ◽

Study Test ◽

Model Training

Abstract Background: Systematic Reviews (SR), studies of studies, use a formal process to evaluate the quality of scientific literature and determine ensuing effectiveness from qualifying articles to establish consensus findings around a hypothesis. Their value is increasing as the conduct and publication of research and evaluation has expanded and the process of identifying key insights becomes more time consuming. Text analytics and Machine Learning (ML) techniques may help overcome this problem of scale while still maintaining the level of rigor expected of SRs.Methods: In this article, we discuss an approach that uses existing examples of SRs to build and test a method for assisting the SR title and abstract pre-screening by reducing the initial pool of potential articles down to articles that meet inclusion criteria. Our approach differs from previous approaches to using ML as a SR tool in that it incorporates ML configurations guided by previously conducted SRs, and human confirmation on ML predictions of relevant articles during multiple iterative reviews on smaller tranches of citations. We applied the tailored method to a new SR review effort to validate performance.Results: The case study test of the approach proved a sensitivity (recall) in finding relevant articles during down selection that may rival many traditional processes.Conclusions: We believe this iterative method can help overcome bias in initial ML model training by having humans reinforce ML models with new and relevant information, and is an applied step towards transfer learning for ML in SR.

Download Full-text

A Personalized Machine-Learning-Enabled Method for Efficient Research in Ethnopharmacology. The Case of the Southern Balkans and the Coastal Zone of Asia Minor

Applied Sciences ◽

10.3390/app11135826 ◽

2021 ◽

Vol 11 (13) ◽

pp. 5826

Author(s):

Evangelos Axiotis ◽

Andreas Kontogiannis ◽

Eleftherios Kalpoutzakis ◽

George Giannakopoulos

Keyword(s):

Machine Learning ◽

Coastal Zone ◽

Extraction Process ◽

Asia Minor ◽

Efficiency And Effectiveness ◽

Effectiveness And Efficiency ◽

Intelligent Tools ◽

Southern Balkans

Ethnopharmacology experts face several challenges when identifying and retrieving documents and resources related to their scientific focus. The volume of sources that need to be monitored, the variety of formats utilized, and the different quality of language use across sources present some of what we call “big data” challenges in the analysis of this data. This study aims to understand if and how experts can be supported effectively through intelligent tools in the task of ethnopharmacological literature research. To this end, we utilize a real case study of ethnopharmacology research aimed at the southern Balkans and the coastal zone of Asia Minor. Thus, we propose a methodology for more efficient research in ethnopharmacology. Our work follows an “expert–apprentice” paradigm in an automatic URL extraction process, through crawling, where the apprentice is a machine learning (ML) algorithm, utilizing a combination of active learning (AL) and reinforcement learning (RL), and the expert is the human researcher. ML-powered research improved the effectiveness and efficiency of the domain expert by 3.1 and 5.14 times, respectively, fetching a total number of 420 relevant ethnopharmacological documents in only 7 h versus an estimated 36 h of human-expert effort. Therefore, utilizing artificial intelligence (AI) tools to support the researcher can boost the efficiency and effectiveness of the identification and retrieval of appropriate documents.

Download Full-text

Machine-Learning Points at Endoscopic, Quality of Life, and Olfactory Parameters as Outcome Criteria for Endoscopic Paranasal Sinus Surgery in Chronic Rhinosinusitis

Journal of Clinical Medicine ◽

10.3390/jcm10184245 ◽

2021 ◽

Vol 10 (18) ◽

pp. 4245

Author(s):

Jörn Lötsch ◽

Constantin A. Hintschich ◽

Petros Petridis ◽

Jürgen Pade ◽

Thomas Hummel

Keyword(s):

Quality Of Life ◽

Machine Learning ◽

Chronic Rhinosinusitis ◽

Paranasal Sinus ◽

Endoscopic Sinus Surgery ◽

Relevant Information ◽

Olfactory Function ◽

Sinus Surgery ◽

Paranasal Sinus Surgery

Chronic rhinosinusitis (CRS) is often treated by functional endoscopic paranasal sinus surgery, which improves endoscopic parameters and quality of life, while olfactory function was suggested as a further criterion of treatment success. In a prospective cohort study, 37 parameters from four categories were recorded from 60 men and 98 women before and four months after endoscopic sinus surgery, including endoscopic measures of nasal anatomy/pathology, assessments of olfactory function, quality of life, and socio-demographic or concomitant conditions. Parameters containing relevant information about changes associated with surgery were examined using unsupervised and supervised methods, including machine-learning techniques for feature selection. The analyzed cohort included 52 men and 38 women. Changes in the endoscopic Lildholdt score allowed separation of baseline from postoperative data with a cross-validated accuracy of 85%. Further relevant information included primary nasal symptoms from SNOT-20 assessments, and self-assessments of olfactory function. Overall improvement in these relevant parameters was observed in 95% of patients. A ranked list of criteria was developed as a proposal to assess the outcome of functional endoscopic sinus surgery in CRS patients with nasal polyposis. Three different facets were captured, including the Lildholdt score as an endoscopic measure and, in addition, disease-specific quality of life and subjectively perceived olfactory function.

Download Full-text

Mold and Casting Table Maintenance Management System at Aluar: A Key to Quality Billets and Saving Costs

Materials Science Forum ◽

10.4028/www.scientific.net/msf.693.208 ◽

2011 ◽

Vol 693 ◽

pp. 208-216

Author(s):

Leonel S. Batalla ◽

Oscar A. Godoy ◽

María Victoria Canullo

Keyword(s):

Management System ◽

High Standard ◽

Relevant Information ◽

Quality Standard ◽

Maintenance Management ◽

Internal Quality ◽

Casting Technology ◽

High Quality Standard

It is well known that mold and casting table maintenance has a direct impact on the internal quality and surface quality of billets. In this paper, the evolution of the main key performance indicators (KPIs) associated with the main consumables of the Wagstaff Air slip billet casting technology is shown, such as molds, casting rings, transition plates and thimbles. The strategies taken to preserve a high standard of quality whilst monitoring the associated costs, are discussed. An automated Mold and Casting Table Maintenance Management system has been developed in house to make available the relevant information from the casting table servicing at the casting pit and at the mold room. A case study is described where this tool allowed us to reduce costs, keeping a high quality standard of the casting table maintenance, ensuring the internal quality of the final product.

Download Full-text

How Sure Can We Be about ML Methods-Based Evaluation of Compound Activity: Incorporation of Information about Prediction Uncertainty Using Deep Learning Techniques

Molecules ◽

10.3390/molecules25061452 ◽

2020 ◽

Vol 25 (6) ◽

pp. 1452

Author(s):

Igor Sieradzki ◽

Damian Leśniak ◽

Sabina Podlewska

Keyword(s):

Machine Learning ◽

Training Set ◽

Activity Prediction ◽

Prediction Uncertainty ◽

Screening Experiments ◽

Learning Techniques ◽

Model Training ◽

Short Time ◽

Selection Of

A great variety of computational approaches support drug design processes, helping in selection of new potentially active compounds, and optimization of their physicochemical and ADMET properties. Machine learning is a group of methods that are able to evaluate in relatively short time enormous amounts of data. However, the quality of machine-learning-based prediction depends on the data supplied for model training. In this study, we used deep neural networks for the task of compound activity prediction and developed dropout-based approaches for estimating prediction uncertainty. Several types of analyses were performed: the relationships between the prediction error, similarity to the training set, prediction uncertainty, number and standard deviation of activity values were examined. It was tested whether incorporation of information about prediction uncertainty influences compounds ranking based on predicted activity and prediction uncertainty was used to search for the potential errors in the ChEMBL database. The obtained outcome indicates that incorporation of information about uncertainty of compound activity prediction can be of great help during virtual screening experiments.

Download Full-text

PISIoT: A Machine Learning and IoT-Based Smart Health Platform for Overweight and Obesity Control

Applied Sciences ◽

10.3390/app9153037 ◽

2019 ◽

Vol 9 (15) ◽

pp. 3037 ◽

Cited By ~ 1

Author(s):

Isaac Machorro-Cano ◽

Giner Alor-Hernández ◽

Mario Andrés Paredes-Valverde ◽

Uriel Ramos-Deonati ◽

José Luis Sánchez-Cervantes ◽

...

Keyword(s):

Machine Learning ◽

Learning Algorithm ◽

Overweight And Obesity ◽

Process Data ◽

Associated Conditions ◽

Smart Health ◽

Medical Recommendations ◽

And Control

Overweight and obesity are affecting productivity and quality of life worldwide. The Internet of Things (IoT) makes it possible to interconnect, detect, identify, and process data between objects or services to fulfill a common objective. The main advantages of IoT in healthcare are the monitoring, analysis, diagnosis, and control of conditions such as overweight and obesity and the generation of recommendations to prevent them. However, the objects used in the IoT have limited resources, so it has become necessary to consider other alternatives to analyze the data generated from monitoring, analysis, diagnosis, control, and the generation of recommendations, such as machine learning. This work presents PISIoT: a machine learning and IoT-based smart health platform for the prevention, detection, treatment, and control of overweight and obesity, and other associated conditions or health problems. Weka API and the J48 machine learning algorithm were used to identify critical variables and classify patients, while Apache Mahout and RuleML were used to generate medical recommendations. Finally, to validate the PISIoT platform, we present a case study on the prevention of myocardial infarction in elderly patients with obesity by monitoring biomedical variables.

Download Full-text

P.013 Machine Learning on Drawing Tests of Cognition: A Systematic Review

Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques ◽

10.1017/cjn.2021.295 ◽

2021 ◽

Vol 48 (s3) ◽

pp. S23-S23

Author(s):

R Kamhawy ◽

R Mcginn ◽

H He ◽

J Ho ◽

M Sharma ◽

...

Keyword(s):

Machine Learning ◽

Systematic Review ◽

Inclusion Criteria ◽

Quality Of Evidence ◽

Drawing Test ◽

Published Evidence ◽

Full Text Review ◽

Grade Methodology ◽

Future Work

Background: Machine learning (ML) methods hold promise in allowing early detection of dementia. We performed a systematic review to assess the quality of published evidence for using ML methods applied to drawing tests of cognition, and to describe the accuracy of the methods. Methods: Embase, Medline, and Cochrane Central Library databases were searched for potential studies up to December 8, 2018 by four independent reviewers. Included articles satisfied the following criteria: 1) use of ML on 2) a drawing test in order to 3) assess cognition. The quality of evidence was then assessed using GRADE methodology. Results: The initial search yielded 4620 citations. Of these, 64 were eligible for full text review. 18 articles then met inclusion criteria. Median AUC across all models was 0.765, with certain ML algorithms performing better in terms of AUC or diagnostic accuracy. However, based on GRADE, the quality of evidence was deemed very low. Conclusions: ML has been applied by several groups to drawing tests of cognition. The quality of evidence is currently too low to make recommendations on their use. Future work must focus on improving reporting, and using standard algorithms and larger, more diverse datasets to improve comparability and generalizability.

Download Full-text

Preventing Human Error on Online Transaction (A Case Study of B.com)

E3S Web of Conferences ◽

10.1051/e3sconf/201912522003 ◽

2019 ◽

Vol 125 ◽

pp. 22003

Author(s):

Novie Susanto ◽

Heru Prastawa ◽

Zodiake Loventa ◽

Muhammad Lufti ◽

Felix Pandan

Keyword(s):

Human Error ◽

Relevant Information ◽

Website Design ◽

Error Identification ◽

Study Methods ◽

Internet Media ◽

Online Buying ◽

Product Specifications

Buying and selling transactions using internet media has advantages related to time and costs. However, buyers often feel difficult when accessing online websites. There are several types of errors that are experienced by buyers when using online buying and selling site services. It is including mistakes in selecting display menus, difficulties in finding items needed because there are too many choices available, errors in interpreting menus used, and sometimes difficulties in knowing product specifications because no relevant information is available on the site. In this study, we discuss the application of the HTA and SHERPA method to assess one of the online buying and selling sites currently used by Indonesian people, namely B.com. There are 100 respondents participated in this study. The study methods are including error identification, analyzing the error using SHERPA, and evaluating the website design. The result of the study provides some recommendation to the online buyer such as ensuring the quality of internet network, filling the data carefully, and confirming the purpose and nominal value of the transaction before it sent to the seller.

Download Full-text

Learning Algorithms of Sentiment Analysis

10.4018/978-1-7998-8473-6.ch012 ◽

2022 ◽

pp. 176-194

Author(s):

Suania Acampa ◽

Ciro Clemente De Falco ◽

Domenico Trezza

Keyword(s):

Machine Learning ◽

Sentiment Analysis ◽

Scientific Community ◽

Learning Algorithms ◽

Learning Approaches ◽

Italian Language ◽

Analysis Techniques ◽

Stratified Sample

The uncritical application of automatic analysis techniques can be insidious. For this reason, the scientific community is very interested in the supervised approach. Can this be enough? This chapter aims to these issues by comparing three machine learning approaches to measuring the sentiment. The case study is the analysis of the sentiment expressed by the Italians on Twitter during the first post-lockdown day. To start the supervised model, it has been necessary to build a stratified sample of tweets by daily and classifying them manually. The model to be test provides for further analysis at the end of the process useful for comparing the three models: index will be built on the tweets processed with the aim of detecting the goodness of the results produced. The comparison of the three algorithms helps the authors to understand not only which is the best approach for the Italian language but tries to understand which strategy is to verify the quality of the data obtained.

Download Full-text