ORCA-SLANG: An Automatic Multi-Stage Semi-Supervised Deep Learning Framework for Large-Scale Killer Whale Call Type Identification

2021 ◽  
Author(s):  
Christian Bergler ◽  
Manuel Schmitt ◽  
Andreas Maier ◽  
Helena Symonds ◽  
Paul Spong ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Killer Whale ◽  
Call Type ◽  
Learning Framework ◽  
Multi Stage

scholarly journals FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Christian Bergler ◽  
Alexander Gebhard ◽  
Jared R. Towers ◽  
Leonid Butyrev ◽  
Gary J. Sutton ◽  
...  
Keyword(s):  
Deep Learning ◽  
North America ◽  
Killer Whale ◽  
Individual Recognition ◽  
Killer Whales ◽  
Photo Identification ◽  
Learning Framework ◽  
Multi Stage ◽  
The Individual ◽  
Identification Techniques

AbstractBiometric identification techniques such as photo-identification require an array of unique natural markings to identify individuals. From 1975 to present, Bigg’s killer whales have been photo-identified along the west coast of North America, resulting in one of the largest and longest-running cetacean photo-identification datasets. However, data maintenance and analysis are extremely time and resource consuming. This study transfers the procedure of killer whale image identification into a fully automated, multi-stage, deep learning framework, entitled FIN-PRINT. It is composed of multiple sequentially ordered sub-components. FIN-PRINT is trained and evaluated on a dataset collected over an 8-year period (2011–2018) in the coastal waters off western North America, including 121,000 human-annotated identification images of Bigg’s killer whales. At first, object detection is performed to identify unique killer whale markings, resulting in 94.4% recall, 94.1% precision, and 93.4% mean-average-precision (mAP). Second, all previously identified natural killer whale markings are extracted. The third step introduces a data enhancement mechanism by filtering between valid and invalid markings from previous processing levels, achieving 92.8% recall, 97.5%, precision, and 95.2% accuracy. The fourth and final step involves multi-class individual recognition. When evaluated on the network test set, it achieved an accuracy of 92.5% with 97.2% top-3 unweighted accuracy (TUA) for the 100 most commonly photo-identified killer whales. Additionally, the method achieved an accuracy of 84.5% and a TUA of 92.9% when applied to the entire 2018 image collection of the 100 most common killer whales. The source code of FIN-PRINT can be adapted to other species and will be publicly available.

An Enhanced Multi-Stage Deep Learning Framework for Detecting Malicious Activities From Autonomous Vehicles

2021 ◽  
pp. 1-10
Author(s):  
Izhar Ahmed Khan ◽  
Nour Moustafa ◽  
Dechang Pi ◽  
Waqas Haider ◽  
Bentian Li ◽  
...  
Keyword(s):  
Deep Learning ◽  
Autonomous Vehicles ◽  
Learning Framework ◽  
Multi Stage

scholarly journals Mol-BERT: An Effective Molecular Representation with BERT for Molecular Property Prediction

2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Juncai Li ◽  
Xiaofei Jiang
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Large Scale ◽  
Molecular Data ◽  
Molecular Property ◽  
Property Prediction ◽  
Learning Framework ◽  
Learning Techniques ◽  
Potential Benefits ◽  
Current Sequence

Molecular property prediction is an essential task in drug discovery. Most computational approaches with deep learning techniques either focus on designing novel molecular representation or combining with some advanced models together. However, researchers pay fewer attention to the potential benefits in massive unlabeled molecular data (e.g., ZINC). This task becomes increasingly challenging owing to the limitation of the scale of labeled data. Motivated by the recent advancements of pretrained models in natural language processing, the drug molecule can be naturally viewed as language to some extent. In this paper, we investigate how to develop the pretrained model BERT to extract useful molecular substructure information for molecular property prediction. We present a novel end-to-end deep learning framework, named Mol-BERT, that combines an effective molecular representation with pretrained BERT model tailored for molecular property prediction. Specifically, a large-scale prediction BERT model is pretrained to generate the embedding of molecular substructures, by using four million unlabeled drug SMILES (i.e., ZINC 15 and ChEMBL 27). Then, the pretrained BERT model can be fine-tuned on various molecular property prediction tasks. To examine the performance of our proposed Mol-BERT, we conduct several experiments on 4 widely used molecular datasets. In comparison to the traditional and state-of-the-art baselines, the results illustrate that our proposed Mol-BERT can outperform the current sequence-based methods and achieve at least 2% improvement on ROC-AUC score on Tox21, SIDER, and ClinTox dataset.

scholarly journals Progressive System: A Deep-Learning Framework for Real-Time Data in Industrial Production

Processes ◽  
2020 ◽  
Vol 8 (6) ◽  
pp. 649
Author(s):  
Yifeng Liu ◽  
Wei Zhang ◽  
Wenhao Du
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Large Scale ◽  
Quality Data ◽  
Time Data ◽  
High Quality ◽  
Real Time System ◽  
High Quality Data ◽  
Learning Framework ◽  
Data Accumulation

Deep learning based on a large number of high-quality data plays an important role in many industries. However, deep learning is hard to directly embed in the real-time system, because the data accumulation of the system depends on real-time acquisitions. However, the analysis tasks of such systems need to be carried out in real time, which makes it impossible to complete the analysis tasks by accumulating data for a long time. In order to solve the problems of high-quality data accumulation, high timeliness of the data analysis, and difficulty in embedding deep-learning algorithms directly in real-time systems, this paper proposes a new progressive deep-learning framework and conducts experiments on image recognition. The experimental results show that the proposed framework is effective and performs well and can reach a conclusion similar to the deep-learning framework based on large-scale data.

scholarly journals Multi-Stage Meta-Learning for Few-Shot with Lie Group Network Constraint

2020 ◽  
Author(s):  
Fang Dong ◽  
Fanzhang Li
Keyword(s):  
Deep Learning ◽  
Gradient Descent ◽  
Large Scale ◽  
Common Knowledge ◽  
Stiefel Manifold ◽  
Training Process ◽  
Multi Stage ◽  
Training Approach ◽  
Meta Learning ◽  
Novel Method

Deep learning has achieved lots of successes in many fields, but when trainable sample are extremely limited, deep learning often under or overfitting to few samples. Meta-learning was proposed to solve difficulties in few-shot learning and fast adaptive areas. Meta-learner learns to remember some common knowledge by training on large scale tasks sampled from a certain data distribution to equip generalization when facing unseen new tasks. Due to the limitation of samples, most approaches only use shallow neural network to avoid overfitting and reduce the difficulty of training process, that causes the waste of many extra information when adapting to unseen tasks. Euclidean space-based gradient descent also make meta-learner's update inaccurate. These issues cause many meta-learning model hard to extract feature from samples and update network parameters. In this paper, we propose a novel method by using multi-stage joint training approach to post the bottleneck during adapting process. To accelerate adapt procedure, we also constraint network to Stiefel manifold, thus meta-learner could perform more stable gradient descent in limited steps. Experiment on mini-ImageNet shows that our method reaches better accuracy under 5-way 1-shot and 5-way 5-shot conditions.

scholarly journals DeepT3 2.0: improving type III secreted effector predictions by an integrative deep learning framework

2021 ◽  
Vol 3 (4) ◽  
Author(s):  
Runyu Jing ◽  
Tingke Wen ◽  
Chengxiang Liao ◽  
Li Xue ◽  
Fengjuan Liu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Amino Acid Sequences ◽  
Secretion Systems ◽  
Gram Negative ◽  
Type Iii ◽  
Learning Framework ◽  
Genome Wide ◽  
Type Iii Secretion Systems ◽  
Animal Pathogens

Abstract Type III secretion systems (T3SSs) are bacterial membrane-embedded nanomachines that allow a number of humans, plant and animal pathogens to inject virulence factors directly into the cytoplasm of eukaryotic cells. Export of effectors through T3SSs is critical for motility and virulence of most Gram-negative pathogens. Current computational methods can predict type III secreted effectors (T3SEs) from amino acid sequences, but due to algorithmic constraints, reliable and large-scale prediction of T3SEs in Gram-negative bacteria remains a challenge. Here, we present DeepT3 2.0 (http://advintbioinforlab.com/deept3/), a novel web server that integrates different deep learning models for genome-wide predicting T3SEs from a bacterium of interest. DeepT3 2.0 combines various deep learning architectures including convolutional, recurrent, convolutional-recurrent and multilayer neural networks to learn N-terminal representations of proteins specifically for T3SE prediction. Outcomes from the different models are processed and integrated for discriminating T3SEs and non-T3SEs. Because it leverages diverse models and an integrative deep learning framework, DeepT3 2.0 outperforms existing methods in validation datasets. In addition, the features learned from networks are analyzed and visualized to explain how models make their predictions. We propose DeepT3 2.0 as an integrated and accurate tool for the discovery of T3SEs.

scholarly journals Large-Scale Detection and Categorization of Oil Spills from SAR Images with Deep Learning

2020 ◽  
Vol 12 (14) ◽  
pp. 2260 ◽  
Author(s):  
Filippo Maria Bianchi ◽  
Martine M. Espeseth ◽  
Njål Borch
Keyword(s):  
Deep Learning ◽  
Oil Spill ◽  
Large Scale ◽  
Oil Spills ◽  
Sar Images ◽  
Oil Spill Detection ◽  
Learning Framework ◽  
Large Scale Data ◽  
Human Operators ◽  
Scale Detection

We propose a deep-learning framework to detect and categorize oil spills in synthetic aperture radar (SAR) images at a large scale. Through a carefully designed neural network model for image segmentation trained on an extensive dataset, we obtain state-of-the-art performance in oil spill detection, achieving results that are comparable to results produced by human operators. We also introduce a classification task, which is novel in the context of oil spill detection in SAR. Specifically, after being detected, each oil spill is also classified according to different categories of its shape and texture characteristics. The classification results provide valuable insights for improving the design of services for oil spill monitoring by world-leading providers. Finally, we present our operational pipeline and a visualization tool for large-scale data, which allows detection and analysis of the historical occurrence of oil spills worldwide.

scholarly journals Learning to Design From Humans: Imitating Human Designers Through Deep Learning

2019 ◽  
Vol 141 (11) ◽  
Author(s):  
Ayush Raina ◽  
Christopher McComb ◽  
Jonathan Cagan
Keyword(s):  
Deep Learning ◽  
Problem Solving ◽  
Large Scale ◽  
Performance Metrics ◽  
Design Strategies ◽  
Learning Framework ◽  
Human Data ◽  
Design Generation ◽  
And Performance ◽  
Computational Resources

Abstract Humans as designers have quite versatile problem-solving strategies. Computer agents on the other hand can access large-scale computational resources to solve certain design problems. Hence, if agents can learn from human behavior, a synergetic human-agent problem-solving team can be created. This paper presents an approach to extract human design strategies and implicit rules, purely from historical human data, and use that for design generation. A two-step framework that learns to imitate human design strategies from observation is proposed and implemented. This framework makes use of deep learning constructs to learn to generate designs without any explicit information about objective and performance metrics. The framework is designed to interact with the problem through a visual interface as humans did when solving the problem. It is trained to imitate a set of human designers by observing their design state sequences without inducing problem-specific modeling bias or extra information about the problem. Furthermore, an end-to-end agent is developed that uses this deep learning framework as its core in conjunction with image processing to map pixel-to-design moves as a mechanism to generate designs. Finally, the designs generated by a computational team of these agents are then compared with actual human data for teams solving a truss design problem. Results demonstrate that these agents are able to create feasible and efficient truss designs without guidance, showing that this methodology allows agents to learn effective design strategies.

Sign in / Sign up

Export Citation Format

Share Document