High-Performance Deep Learning Models for Seismic Noise Detection and Quality Control in the Processing Workflow

Recent deep learning models succeed in achieving high accuracy and fast inference time, but they require high-performance computing resources because they have a large number of parameters. However, not all systems have high-performance hardware. Sometimes, a deep learning model needs to be run on edge devices such as IoT devices or smartphones. On edge devices, however, limited computing resources are available and the amount of computation must be reduced to launch the deep learning models. Pruning is one of the well-known approaches for deriving light-weight models by eliminating weights, channels or filters. In this work, we propose “zero-keep filter pruning” for energy-efficient deep neural networks. The proposed method maximizes the number of zero elements in filters by replacing small values with zero and pruning the filter that has the lowest number of zeros. In the conventional approach, the filters that have the highest number of zeros are generally pruned. As a result, through this zero-keep filter pruning, we can have the filters that have many zeros in a model. We compared the results of the proposed method with the random filter pruning and proved that our method shows better performance with many fewer non-zero elements with a marginal drop in accuracy. Finally, we discuss a possible multiplier architecture, zero-skip multiplier circuit, which skips the multiplications with zero to accelerate and reduce energy consumption.

Download Full-text

Use of Endoscopic Images in the Prediction of Submucosal Invasion of Gastric Neoplasms: Automated Deep Learning Model Development and Usability Study

Journal of Medical Internet Research ◽

10.2196/25167 ◽

2021 ◽

Vol 23 (4) ◽

pp. e25167

Author(s):

Chang Seok Bang ◽

Hyun Lim ◽

Hae Min Jeong ◽

Sung Hyeon Hwang

Keyword(s):

Deep Learning ◽

High Performance ◽

Model Development ◽

Gastric Neoplasms ◽

Test Accuracy ◽

Learning Models ◽

Invasion Depth ◽

General Physician ◽

Endoscopic Images ◽

External Test

Background In a previous study, we examined the use of deep learning models to classify the invasion depth (mucosa-confined versus submucosa-invaded) of gastric neoplasms using endoscopic images. The external test accuracy reached 77.3%. However, model establishment is labor intense, requiring high performance. Automated deep learning (AutoDL) models, which enable fast searching of optimal neural architectures and hyperparameters without complex coding, have been developed. Objective The objective of this study was to establish AutoDL models to classify the invasion depth of gastric neoplasms. Additionally, endoscopist–artificial intelligence interactions were explored. Methods The same 2899 endoscopic images that were employed to establish the previous model were used. A prospective multicenter validation using 206 and 1597 novel images was conducted. The primary outcome was external test accuracy. Neuro-T, Create ML Image Classifier, and AutoML Vision were used in establishing the models. Three doctors with different levels of endoscopy expertise were asked to classify the invasion depth of gastric neoplasms for each image without AutoDL support, with faulty AutoDL support, and with best performance AutoDL support in sequence. Results The Neuro-T–based model reached 89.3% (95% CI 85.1%-93.5%) external test accuracy. For the model establishment time, Create ML Image Classifier showed the fastest time of 13 minutes while reaching 82.0% (95% CI 76.8%-87.2%) external test accuracy. While the expert endoscopist's decisions were not influenced by AutoDL, the faulty AutoDL misled the endoscopy trainee and the general physician. However, this was corrected by the support of the best performance AutoDL model. The trainee gained the most benefit from the AutoDL support. Conclusions AutoDL is deemed useful for the on-site establishment of customized deep learning models. An inexperienced endoscopist with at least a certain level of expertise can benefit from AutoDL support.

Download Full-text

Customized automated deep-learning models and endoscopist-artificial intelligence interaction for the prediction of submucosal invasion of gastric neoplasms in endoscopic images: development and usability study (Preprint)

10.2196/preprints.25167 ◽

2020 ◽

Author(s):

Chang Seok Bang ◽

Hyun Lim ◽

Hae Min Jeong ◽

Sung Hyeon Hwang

Keyword(s):

Artificial Intelligence ◽

Deep Learning ◽

High Performance ◽

Gastric Neoplasms ◽

Test Accuracy ◽

Learning Models ◽

Invasion Depth ◽

General Physician ◽

Endoscopic Images ◽

External Test

BACKGROUND Authors previously examined deep-learning models to classify the invasion depth (mucosa-confined vs. submucosa-invaded) of gastric neoplasms using endoscopic images. The external-test accuracy reach 77.3%. However, model establishment is labor-intense, requiring high performance. Automated deep-learning (AutoDL), which enable fast searching of optimal neural architectures and hyperparameters without complex coding, have been developed. OBJECTIVE To establish AutoDL models in classifying the invasion depth of gastric neoplasms. Additionally, endoscopist-artificial intelligence interactions were explored. METHODS The same 2,899 endoscopic images, which were employed to establish the previous model, were used. A prospective multicenter validation using 206 and 1597 novel images was conducted. The primary outcome was external-test accuracy. “Neuro-T,” “Create ML-Image Classifier,” and “AutoML-Vision” were used in establishing the models. Three doctors with different levels of endoscopy expertise were analyzed for each image without AutoDL’s support, with faulty AutoDL’s support, and with best performance AutoDL’s support in sequence. RESULTS Neuro-T-based model reached 89.3% (95% confidence interval: 85.1–93.5%) external-test accuracy. For the model establishment time, Create ML-Image Classifier showed the fastest time of 13 minutes while reaching 82% external-test accuracy. Expert endoscopist decisions were not influenced by AutoDL. The faulty AutoDL has misled the endoscopy trainee and the general physician. However, this was corrected by the support of the best performance AutoDL. The trainee gained the highest benefit from the AutoDL’s support. CONCLUSIONS AutoDL is deemed useful for the on-site establishment of customized deep-learning models. An inexperienced endoscopist with at least a certain level of expertise can benefit from AutoDL support.

Download Full-text

Comparative Evaluations of Human Behavior Recognition Using Deep Learning

Handbook of Research on Multimedia Cyber Security - Advances in Information Security, Privacy, and Ethics ◽

10.4018/978-1-7998-2701-6.ch009 ◽

2020 ◽

pp. 176-189 ◽

Cited By ~ 2

Author(s):

Jia Lu ◽

Wei Qi Yan

Keyword(s):

Deep Learning ◽

Human Behavior ◽

High Performance ◽

Shopping Malls ◽

Learning Models ◽

Behavior Recognition ◽

Model Training ◽

Public Datasets ◽

Human Behavior Recognition ◽

The Cost

With the cost decrease of security monitoring facilities such as cameras, video surveillance has been widely applied to public security and safety such as banks, transportation, shopping malls, etc. which allows police to monitor abnormal events. Through deep learning, authors can achieve high performance of human behavior detection and recognition by using model training and tests. This chapter uses public datasets Weizmann dataset and KTH dataset to train deep learning models. Four deep learning models were investigated for human behavior recognition. Results show that YOLOv3 model is the best one and achieved 96.29% of mAP based on Weizmann dataset and 84.58% of mAP on KTH dataset. The chapter conducts human behavior recognition using deep learning and evaluates the outcomes of different approaches with the support of the datasets.

Download Full-text

Confluence of Artificial Intelligence and High Performance Computing for Accelerated, Scalable and Reproducible Gravitational Wave Detection

10.21203/rs.3.rs-138409/v1 ◽

2021 ◽

Author(s):

Eliu Huerta ◽

Asad Khan ◽

Xiaobo Huang ◽

Minyang Tian ◽

Maksim Levental ◽

...

Keyword(s):

Artificial Intelligence ◽

Black Hole ◽

Deep Learning ◽

Gravitational Wave ◽

High Performance Computing ◽

High Performance ◽

Learning Models ◽

Wave Detection ◽

Binary Black Hole ◽

Performance Computing

Abstract Finding new ways to use artificial intelligence (AI) to accelerate the analysis of gravitational wave data, and ensuring the developed models are easily reusable promises to unlock new opportunities in multi-messenger astrophysics (MMA), and to enable wider use, rigorous validation, and sharing of developed models by the community. In this work, we demonstrate how connecting recently deployed DOE and NSF-sponsored cyberinfrastructure allows for new ways to publish models, and to subsequently deploy these models into applications using computing platforms ranging from laptops to high performance computing clusters. We develop a workflow that connects the \texttt{Data and Learning Hub for Science} (DLHub), a repository for publishing machine learning models, with the Hardware Accelerated Learning (HAL) deep learning computing cluster, using funcX as a universal distributed computing service. We then use this workflow to search for binary black hole gravitational wave signals in open source advanced LIGO data. We find that using this workflow, an ensemble of four openly available deep learning models can be run on HAL and process the entire month of August 2017 of advanced LIGO data in just seven minutes, identifying all four binary black hole mergers previously identified in this dataset, and reporting no misclassifications. This approach, which combines advances in AI, distributed computing, and scientific data infrastructure opens new pathways to conduct reproducible, accelerated, data-driven gravitational wave detection.

Download Full-text

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

10.26434/chemrxiv.12562121 ◽

2020 ◽

Author(s):

Dean Sumner ◽

Jiazhen He ◽

Amol Thakkar ◽

Ola Engkvist ◽

Esben Jannik Bjerrum

Keyword(s):

Neural Networks ◽

Pattern Recognition ◽

Deep Learning ◽

Recurrent Neural Networks ◽

Data Augmentation ◽

State Of The Art ◽

Sequence Similarity ◽

Learning Models ◽

Underlying Network

SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as attentional gain – an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.

Download Full-text

Improving the Accuracy of Protein-Ligand Binding Affinity Prediction by Deep Learning Models: Benchmark and Model

10.26434/chemrxiv.9866912 ◽

2019 ◽

Author(s):

Mohammad Rezaei ◽

Yanjun Li ◽

Xiaolin Li ◽

Chenglong Li

Keyword(s):

Deep Learning ◽

Drug Design ◽

Binding Affinity ◽

Benchmark Dataset ◽

Rational Drug Design ◽

Learning Models ◽

Structure Based Drug Design ◽

Binding Affinity Prediction ◽

Affinity Prediction ◽

Rational Drug

Introduction: The ability to discriminate among ligands binding to the same protein target in terms of their relative binding affinity lies at the heart of structure-based drug design. Any improvement in the accuracy and reliability of binding affinity prediction methods decreases the discrepancy between experimental and computational results. Objectives: The primary objectives were to find the most relevant features affecting binding affinity prediction, least use of manual feature engineering, and improving the reliability of binding affinity prediction using efficient deep learning models by tuning the model hyperparameters. Methods: The binding site of target proteins was represented as a grid box around their bound ligand. Both binary and distance-dependent occupancies were examined for how an atom affects its neighbor voxels in this grid. A combination of different features including ANOLEA, ligand elements, and Arpeggio atom types were used to represent the input. An efficient convolutional neural network (CNN) architecture, DeepAtom, was developed, trained and tested on the PDBbind v2016 dataset. Additionally an extended benchmark dataset was compiled to train and evaluate the models. Results: The best DeepAtom model showed an improved accuracy in the binding affinity prediction on PDBbind core subset (Pearson’s R=0.83) and is better than the recent state-of-the-art models in this field. In addition when the DeepAtom model was trained on our proposed benchmark dataset, it yields higher correlation compared to the baseline which confirms the value of our model. Conclusions: The promising results for the predicted binding affinities is expected to pave the way for embedding deep learning models in virtual screening and rational drug design fields.

Download Full-text

Data science in economics: comprehensive review of advanced machine learning and deep learning methods

10.31232/osf.io/4pxq2 ◽

2020 ◽

Author(s):

Saeed Nosratabadi ◽

Amir Mosavi ◽

Puhong Duan ◽

Pedram Ghamisi ◽

Ferdinand Filip ◽

...

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Data Science ◽

State Of The Art ◽

Science Methods ◽

Learning Models ◽

Diverse Range ◽

Hybrid Machine ◽

Economics Research

This paper provides a state-of-the-art investigation of advances in data science in emerging economic applications. The analysis was performed on novel data science methods in four individual classes of deep learning models, hybrid deep learning models, hybrid machine learning, and ensemble models. Application domains include a wide and diverse range of economics research from the stock market, marketing, and e-commerce to corporate banking and cryptocurrency. Prisma method, a systematic literature review methodology, was used to ensure the quality of the survey. The findings reveal that the trends follow the advancement of hybrid models, which, based on the accuracy metric, outperform other learning algorithms. It is further expected that the trends will converge toward the advancements of sophisticated hybrid deep learning models.

Download Full-text

Development of the automated information system «clean district» for control of cleaning of streets in the city of Krasnoyarsk

Informatization and communication ◽

10.34219/2078-8320-2020-11-2-77-83 ◽

2020 ◽

pp. 77-83

Author(s):

A. Popov ◽

O.N. Lopateeva ◽

A.K. Ovsyankin ◽

M. M. Satsuk ◽

A. A. Artyshko ◽

...

Keyword(s):

Information System ◽

Quality Control ◽

High Performance ◽

Operational Performance ◽

Development Environment ◽

Integrated Development ◽

Central District ◽

Effective Performance ◽

Automated Information System ◽

The City

Among the measures aimed at the effective performance of public services in a modern urban environment, one of the main is the quality control and efficiency of the work performed. Timely street cleaning is hampered by several groups of problems, including the lack of a single automated information system (AIS) control of the work performed. In this regard, there is a need to improve and automate this area. This approach will allow you to combine high performance due to the speed of the system and effective quality control of street cleaning. The purpose of this work is the study and analysis of existing information systems (is), allowing to automate the process of quality control and operational performance of the above tasks. On the basis of the conducted researches, to develop is, having coordinated with the customer (administration of the Central district of Krasnoyarsk) requirements and functionality which allow to automate this process.This article presents the main aspects of the design and software solutions for the implementation of the algorithm in the form of AIS, designed to automate the process of monitoring the cleanliness of streets in the city. The development of AIS was conducted in the PhpStorm integrated development environment in the PHP programming language.

Download Full-text