Learning Models of Predicate Logical Theories with Neural Networks Based on Topos Theory

SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as attentional gain – an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.

Download Full-text

Deep Malaria Parasite Detection in Thin Blood Smear Microscopic Images

Applied Sciences ◽

10.3390/app11052284 ◽

2021 ◽

Vol 11 (5) ◽

pp. 2284

Author(s):

Asma Maqsood ◽

Muhammad Shahid Farid ◽

Muhammad Hassan Khan ◽

Marcin Grzegorzek

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Red Blood Cells ◽

Convolutional Neural Networks ◽

Blood Cells ◽

Superior Performance ◽

Learning Models ◽

Infected Female ◽

The World ◽

Augmentation Techniques

Malaria is a disease activated by a type of microscopic parasite transmitted from infected female mosquito bites to humans. Malaria is a fatal disease that is endemic in many regions of the world. Quick diagnosis of this disease will be very valuable for patients, as traditional methods require tedious work for its detection. Recently, some automated methods have been proposed that exploit hand-crafted feature extraction techniques however, their accuracies are not reliable. Deep learning approaches modernize the world with their superior performance. Convolutional Neural Networks (CNN) are vastly scalable for image classification tasks that extract features through hidden layers of the model without any handcrafting. The detection of malaria-infected red blood cells from segmented microscopic blood images using convolutional neural networks can assist in quick diagnosis, and this will be useful for regions with fewer healthcare experts. The contributions of this paper are two-fold. First, we evaluate the performance of different existing deep learning models for efficient malaria detection. Second, we propose a customized CNN model that outperforms all observed deep learning models. It exploits the bilateral filtering and image augmentation techniques for highlighting features of red blood cells before training the model. Due to image augmentation techniques, the customized CNN model is generalized and avoids over-fitting. All experimental evaluations are performed on the benchmark NIH Malaria Dataset, and the results reveal that the proposed algorithm is 96.82% accurate in detecting malaria from the microscopic blood smears.

Download Full-text

414 Deep Neural Networks: A Survey Tool for Obstructive Sleep Apnea Prediction

SLEEP ◽

10.1093/sleep/zsab072.413 ◽

2021 ◽

Vol 44 (Supplement_2) ◽

pp. A164-A164

Author(s):

Pahnwat Taweesedt ◽

JungYoon Kim ◽

Jaehyun Park ◽

Jangwoon Park ◽

Munish Sharma ◽

...

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Obstructive Sleep Apnea ◽

Sleep Apnea ◽

Deep Neural Networks ◽

Support Vector ◽

Learning Models ◽

Obstructive Sleep ◽

Screening Questionnaires ◽

Machine Learning Models

Abstract Introduction Obstructive sleep apnea (OSA) is a common sleep-related breathing disorder with an estimation of one billion people. Full-night polysomnography is considered the gold standard for OSA diagnosis. However, it is time-consuming, expensive and is not readily available in many parts of the world. Many screening questionnaires and scores have been proposed for OSA prediction with high sensitivity and low specificity. The present study is intended to develop models with various machine learning techniques to predict the severity of OSA by incorporating features from multiple questionnaires. Methods Subjects who underwent full-night polysomnography in Torr sleep center, Texas and completed 5 OSA screening questionnaires/scores were included. OSA was diagnosed by using Apnea-Hypopnea Index ≥ 5. We trained five different machine learning models including Deep Neural Networks with the scaled principal component analysis (DNN-PCA), Random Forest (RF), Adaptive Boosting classifier (ABC), and K-Nearest Neighbors classifier (KNC) and Support Vector Machine Classifier (SVMC). Training:Testing subject ratio of 65:35 was used. All features including demographic data, body measurement, snoring and sleepiness history were obtained from 5 OSA screening questionnaires/scores (STOP-BANG questionnaires, Berlin questionnaires, NoSAS score, NAMES score and No-Apnea score). Performance parametrics were used to compare between machine learning models. Results Of 180 subjects, 51.5 % of subjects were male with mean (SD) age of 53.6 (15.1). One hundred and nineteen subjects were diagnosed with OSA. Area Under the Receiver Operating Characteristic Curve (AUROC) of DNN-PCA, RF, ABC, KNC, SVMC, STOP-BANG questionnaire, Berlin questionnaire, NoSAS score, NAMES score, and No-Apnea score were 0.85, 0.68, 0.52, 0.74, 0.75, 0.61, 0.63, 0,61, 0.58 and 0,58 respectively. DNN-PCA showed the highest AUROC with sensitivity of 0.79, specificity of 0.67, positive-predictivity of 0.93, F1 score of 0.86, and accuracy of 0.77. Conclusion Our result showed that DNN-PCA outperforms OSA screening questionnaires, scores and other machine learning models. Support (if any):

Download Full-text

Explainable Graph Neural Networks for Organic Cages

10.26434/chemrxiv-2021-zhcb1 ◽

2021 ◽

Author(s):

Qi Yuan ◽

Filip Szczypiński ◽

Kim Jelfs

Keyword(s):

Neural Networks ◽

Material Science ◽

Predictive Power ◽

Internal Cavity ◽

Learning Models ◽

Molecular Fragments ◽

Molecular Separations ◽

Graph Neural Networks ◽

Structural Insights ◽

Machine Learning Models

The development of accurate and explicable machine learning models to predict the properties of topologically complex systems is a challenge in material science. Porous organic cages, a class of polycyclic molecular materials, have potential application in molecular separations, catalysis and encapsulation. For most applications of porous organic cages, having a permanent internal cavity in the absence of solvent, a property termed “shape persistency” is critical. Here, we report the development of Graph Neural Networks (GNNs) to predict the shape persistence of organic cages. Graph neural networks are a class of neural networks where the data, in our case that of organic cages, are represented by graphs. The performance of the GNN models was measured against a previously reported computational database of organic cages formed through a range of [4+6] reactions with a variety of reaction chemistries. The reported GNNs have an improved prediction accuracy and transferability compared to random forest predictions. Apart from the improvement in predictive power, we explored the explicability of the GNNs by computing the integrated gradient of the GNN input. The contribution of monomers and molecular fragments to the shape persistence of the organic cages could be quantitatively evaluated with integrated gradient. With the added explicability of the GNNs, it is possible not only to accurately predict the property of organic materials, but also to interpret the predictions of the deep learning models and provide structural insights to the discovery of future materials.

Download Full-text

A Novel Method for Performance Measurement of Public Educational Institutions Using Machine Learning Models

Applied Sciences ◽

10.3390/app11199296 ◽

2021 ◽

Vol 11 (19) ◽

pp. 9296

Author(s):

Talha Mahboob Alam ◽

Mubbashar Mushtaq ◽

Kamran Shaukat ◽

Ibrahim A. Hameed ◽

Muhammad Umer Sarwar ◽

...

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Feature Selection ◽

Artificial Neural Networks ◽

Performance Indicators ◽

Key Performance Indicators ◽

Educational Institutions ◽

Institutional Performance ◽

Learning Models ◽

Machine Learning Models

Lack of education is a major concern in underdeveloped countries because it leads to poor human and economic development. The level of education in public institutions varies across all regions around the globe. Current disparities in access to education worldwide are mostly due to systemic regional differences and the distribution of resources. Previous research focused on evaluating students’ academic performance, but less has been done to measure the performance of educational institutions. Key performance indicators for the evaluation of institutional performance differ from student performance indicators. There is a dire need to evaluate educational institutions’ performance based on their disparities and academic results on a large scale. This study proposes a model to measure institutional performance based on key performance indicators through data mining techniques. Various feature selection methods were used to extract the key performance indicators. Several machine learning models, namely, J48 decision tree, support vector machines, random forest, rotation forest, and artificial neural networks were employed to build an efficient model. The results of the study were based on different factors, i.e., the number of schools in a specific region, teachers, school locations, enrolment, and availability of necessary facilities that contribute to school performance. It was also observed that urban regions performed well compared to rural regions due to the improved availability of educational facilities and resources. The results showed that artificial neural networks outperformed other models and achieved an accuracy of 82.9% when the relief-F based feature selection method was used. This study will help support efforts in governance for performance monitoring, policy formulation, target-setting, evaluation, and reform to address the issues and challenges in education worldwide.

Download Full-text

Benchmarking Graph Neural Networks for Materials Chemistry

10.26434/chemrxiv.13615421.v2 ◽

2021 ◽

Author(s):

Victor Fung ◽

Jiaxin Zhang ◽

Eric Juarez ◽

Bobby Sumpter

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Materials Chemistry ◽

Learning Models ◽

High Data ◽

Hyperparameter Selection ◽

Computational Materials ◽

Conventional Models ◽

Graph Neural Networks ◽

Machine Learning Models

Graph neural networks (GNNs) have received intense interest as a rapidly expanding class of machine learning models remarkably well-suited for materials applications. To date, a number of successful GNNs have been proposed and demonstrated for systems ranging from crystal stability to electronic property prediction and to surface chemistry and heterogeneous catalysis. However, a consistent benchmark of these models remains lacking, hindering the development and consistent evaluation of new models in the materials field. Here, we present a workflow and testing platform, MatDeepLearn, for quickly and reproducibly assessing and comparing GNNs and other machine learning models. We use this platform to optimize and evaluate a selection of top performing GNNs on several representative datasets in computational materials chemistry. From our investigations we note the importance of hyperparameter selection and find roughly similar performances for the top models once optimized. We identify several strengths in GNNs over conventional models in cases with compositionally diverse datasets and in its overall flexibility with respect to inputs, due to learned rather than defined representations. Meanwhile several weaknesses of GNNs are also observed including high data requirements, and suggestions for further improvement for applications in materials chemistry are proposed.

Download Full-text

Framework for TCAD augmented machine learning on multi- I–V characteristics using convolutional neural network and multiprocessing

Journal of Semiconductors ◽

10.1088/1674-4926/42/12/124101 ◽

2021 ◽

Vol 42 (12) ◽

pp. 124101

Author(s):

Thomas Hirtz ◽

Steyn Huurman ◽

He Tian ◽

Yi Yang ◽

Tian-Ling Ren

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Information Technologies ◽

Deep Neural Networks ◽

State Of The Art ◽

Data Driven ◽

Sufficient Data ◽

Learning Models ◽

Simulation Tools ◽

New Information

Abstract In a world where data is increasingly important for making breakthroughs, microelectronics is a field where data is sparse and hard to acquire. Only a few entities have the infrastructure that is required to automate the fabrication and testing of semiconductor devices. This infrastructure is crucial for generating sufficient data for the use of new information technologies. This situation generates a cleavage between most of the researchers and the industry. To address this issue, this paper will introduce a widely applicable approach for creating custom datasets using simulation tools and parallel computing. The multi-I–V curves that we obtained were processed simultaneously using convolutional neural networks, which gave us the ability to predict a full set of device characteristics with a single inference. We prove the potential of this approach through two concrete examples of useful deep learning models that were trained using the generated data. We believe that this work can act as a bridge between the state-of-the-art of data-driven methods and more classical semiconductor research, such as device engineering, yield engineering or process monitoring. Moreover, this research gives the opportunity to anybody to start experimenting with deep neural networks and machine learning in the field of microelectronics, without the need for expensive experimentation infrastructure.

Download Full-text

Train Deep Learning Models using subsurface geological images datasets

10.5194/egusphere-egu21-6385 ◽

2021 ◽

Author(s):

Ramy Abdallah ◽

Clare E. Bond ◽

Robert W.H. Butler

Keyword(s):

Artificial Intelligence ◽

Machine Learning ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Networks ◽

Learning Models ◽

Structural Interpretation ◽

Performance Accuracy ◽

Learning Techniques ◽

Machine Learning Models

Machine learning is being presented as a new solution for a wide range of geoscience problems. Primarily machine learning has been used for 3D seismic data processing, seismic facies analysis and well log data correlation. The rapid development in technology with open-source artificial intelligence libraries and the accessibility of affordable computer graphics processing units (GPU) makes the application of machine learning in geosciences increasingly tractable. However, the application of artificial intelligence in structural interpretation workflows of subsurface datasets is still ambiguous. This study aims to use machine learning techniques to classify images of folds and fold-thrust structures. Here we show that convolutional neural networks (CNNs) as supervised deep learning techniques provide excellent algorithms to discriminate between geological image datasets. Four different datasets of images have been used to train and test the machine learning models. These four datasets are a seismic character dataset with five classes (faults, folds, salt, flat layers and basement), folds types with three classes (buckle, chevron and conjugate), fault types with three classes (normal, reverse and thrust) and fold-thrust geometries with three classes (fault bend fold, fault propagation fold and detachment fold). These image datasets are used to investigate three machine learning models. One Feedforward linear neural network model and two convolutional neural networks models (Convolution 2d layer transforms sequential model and Residual block model (ResNet with 9, 34, and 50 layers)). Validation and testing datasets forms a critical part of testing the model&#8217;s performance accuracy. The ResNet model records the highest performance accuracy score, of the machine learning models tested. Our CNN image classification model analysis provides a framework for applying machine learning to increase structural interpretation efficiency, and shows that CNN classification models can be applied effectively to geoscience problems. The study provides a starting point to apply unsupervised machine learning approaches to sub-surface structural interpretation workflows.

Download Full-text

A Review of Recent Deep Learning Models in COVID-19 Diagnosis

European Journal of Engineering and Technology Research ◽

10.24018/ejers.2021.6.5.2485 ◽

2021 ◽

Vol 6 (5) ◽

pp. 10-15

Author(s):

Ela Bhattacharya ◽

D. Bhattacharya

Keyword(s):

Artificial Intelligence ◽

Neural Networks ◽

Deep Learning ◽

Deep Neural Networks ◽

Test Results ◽

Learning Models ◽

Future Directions ◽

Human Contact ◽

The World ◽

Short Span

COVID-19 has emerged as the latest worrisome pandemic, which is reported to have its outbreak in Wuhan, China. The infection spreads by means of human contact, as a result, it has caused massive infections across 200 countries around the world. Artificial intelligence has likewise contributed to managing the COVID-19 pandemic in various aspects within a short span of time. Deep Neural Networks that are explored in this paper have contributed to the detection of COVID-19 from imaging sources. The datasets, pre-processing, segmentation, feature extraction, classification and test results which can be useful for discovering future directions in the domain of automatic diagnosis of the disease, utilizing artificial intelligence-based frameworks, have been investigated in this paper.

Download Full-text

Chapter 15. Human-Centered Concept Explanations for Neural Networks

10.3233/faia210362 ◽

2021 ◽

Author(s):

Chih-Kuan Yeh ◽

Been Kim ◽

Pradeep Ravikumar

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Case Studies ◽

Real World ◽

Deep Neural Networks ◽

Learning Models ◽

Real World Applications ◽

The Right ◽

Concept Activation ◽

Machine Learning Models

Understanding complex machine learning models such as deep neural networks with explanations is crucial in various applications. Many explanations stem from the model perspective, and may not necessarily effectively communicate why the model is making its predictions at the right level of abstraction. For example, providing importance weights to individual pixels in an image can only express which parts of that particular image is important to the model, but humans may prefer an explanation which explains the prediction by concept-based thinking. In this work, we review the emerging area of concept based explanations. We start by introducing concept explanations including the class of Concept Activation Vectors (CAV) which characterize concepts using vectors in appropriate spaces of neural activations, and discuss different properties of useful concepts, and approaches to measure the usefulness of concept vectors. We then discuss approaches to automatically extract concepts, and approaches to address some of their caveats. Finally, we discuss some case studies that showcase the utility of such concept-based explanations in synthetic settings and real world applications.

Download Full-text