Combining Deep Learning and Qualitative Spatial Reasoning to Learn Complex Structures from Sparse Examples with Noise

Many modern machine learning approaches require vast amounts of training data to learn new concepts; conversely, human learning often requires few examples—sometimes only one—from which the learner can abstract structural concepts. We present a novel approach to introducing new spatial structures to an AI agent, combining deep learning over qualitative spatial relations with various heuristic search algorithms. The agent extracts spatial relations from a sparse set of noisy examples of block-based structures, and trains convolutional and sequential models of those relation sets. To create novel examples of similar structures, the agent begins placing blocks on a virtual table, uses a CNN to predict the most similar complete example structure after each placement, an LSTM to predict the most likely set of remaining moves needed to complete it, and recommends one using heuristic search. We verify that the agent learned the concept by observing its virtual block-building activities, wherein it ranks each potential subsequent action toward building its learned concept. We empirically assess this approach with human participants’ ratings of the block structures. Initial results and qualitative evaluations of structures generated by the trained agent show where it has generalized concepts from the training data, which heuristics perform best within the search space, and how we might improve learning and execution.

Download Full-text

Generative Adversarial Networks for Data Augmentation in Structural Adhesive Inspection

Applied Sciences ◽

10.3390/app11073086 ◽

2021 ◽

Vol 11 (7) ◽

pp. 3086

Author(s):

Ricardo Silva Peres ◽

Miguel Azevedo ◽

Sara Oleiro Araújo ◽

Magno Guedes ◽

Fábio Miranda ◽

...

Keyword(s):

Deep Learning ◽

Data Augmentation ◽

Manufacturing Sector ◽

Data Availability ◽

Training Data ◽

Generative Adversarial Networks ◽

Learning Approaches ◽

Adversarial Networks ◽

Novel Approach ◽

Structural Adhesive

The technological advances brought forth by the Industry 4.0 paradigm have renewed the disruptive potential of artificial intelligence in the manufacturing sector, building the data-driven era on top of concepts such as Cyber–Physical Systems and the Internet of Things. However, data availability remains a major challenge for the success of these solutions, particularly concerning those based on deep learning approaches. Specifically in the quality inspection of structural adhesive applications, found commonly in the automotive domain, defect data with sufficient variety, volume and quality is generally costly, time-consuming and inefficient to obtain, jeopardizing the viability of such approaches due to data scarcity. To mitigate this, we propose a novel approach to generate synthetic training data for this application, leveraging recent breakthroughs in training generative adversarial networks with limited data to improve the performance of automated inspection methods based on deep learning, especially for imbalanced datasets. Preliminary results in a real automotive pilot cell show promise in this direction, with the approach being able to generate realistic adhesive bead images and consequently object detection models showing improved mean average precision at different thresholds when trained on the augmented dataset. For reproducibility purposes, the model weights, configurations and data encompassed in this study are made publicly available.

Download Full-text

Deep Learning of Appearance Affinity for Multi-Object Tracking and Re-Identification: A Comparative View

Electronics ◽

10.3390/electronics9111757 ◽

2020 ◽

Vol 9 (11) ◽

pp. 1757

Author(s):

María J. Gómez-Silva ◽

Arturo de la Escalera ◽

José M. Armingol

Keyword(s):

Deep Learning ◽

Object Tracking ◽

Loss Function ◽

Neural Model ◽

Training Data ◽

Learning Approaches ◽

The Core ◽

Triplet Loss ◽

Affinity Model

Recognizing the identity of a query individual in a surveillance sequence is the core of Multi-Object Tracking (MOT) and Re-Identification (Re-Id) algorithms. Both tasks can be addressed by measuring the appearance affinity between people observations with a deep neural model. Nevertheless, the differences in their specifications and, consequently, in the characteristics and constraints of the available training data for each one of these tasks, arise from the necessity of employing different learning approaches to attain each one of them. This article offers a comparative view of the Double-Margin-Contrastive and the Triplet loss function, and analyzes the benefits and drawbacks of applying each one of them to learn an Appearance Affinity model for Tracking and Re-Identification. A batch of experiments have been conducted, and their results support the hypothesis concluded from the presented study: Triplet loss function is more effective than the Contrastive one when an Re-Id model is learnt, and, conversely, in the MOT domain, the Contrastive loss can better discriminate between pairs of images rendering the same person or not.

Download Full-text

Real-Time Automated Classification of Sky Conditions Using Deep Learning and Edge Computing

Remote Sensing ◽

10.3390/rs13193859 ◽

2021 ◽

Vol 13 (19) ◽

pp. 3859

Author(s):

Joby M. Prince Czarnecki ◽

Sathishkumar Samiappan ◽

Meilun Zhou ◽

Cary Daniel McCraine ◽

Louis L. Wasson

Keyword(s):

Neural Network ◽

Deep Learning ◽

Image Quality ◽

Convolutional Neural Network ◽

Precision Agriculture ◽

Edge Computing ◽

Training Data ◽

Learning Approaches ◽

Sky Conditions

The radiometric quality of remotely sensed imagery is crucial for precision agriculture applications because estimations of plant health rely on the underlying quality. Sky conditions, and specifically shadowing from clouds, are critical determinants in the quality of images that can be obtained from low-altitude sensing platforms. In this work, we first compare common deep learning approaches to classify sky conditions with regard to cloud shadows in agricultural fields using a visible spectrum camera. We then develop an artificial-intelligence-based edge computing system to fully automate the classification process. Training data consisting of 100 oblique angle images of the sky were provided to a convolutional neural network and two deep residual neural networks (ResNet18 and ResNet34) to facilitate learning two classes, namely (1) good image quality expected, and (2) degraded image quality expected. The expectation of quality stemmed from the sky condition (i.e., density, coverage, and thickness of clouds) present at the time of the image capture. These networks were tested using a set of 13,000 images. Our results demonstrated that ResNet18 and ResNet34 classifiers produced better classification accuracy when compared to a convolutional neural network classifier. The best overall accuracy was obtained by ResNet34, which was 92% accurate, with a Kappa statistic of 0.77. These results demonstrate a low-cost solution to quality control for future autonomous farming systems that will operate without human intervention and supervision.

Download Full-text

Morphological Estimation of Cellularity on Neo-Adjuvant Treated Breast Cancer Histological Images

Journal of Imaging ◽

10.3390/jimaging6100101 ◽

2020 ◽

Vol 6 (10) ◽

pp. 101

Author(s):

Mauricio Alberto Ortega-Ruiz ◽

Cefa Karabağ ◽

Victor García Garduño ◽

Constantino Carlos Reyes-Aldasoro

Keyword(s):

Breast Cancer ◽

Deep Learning ◽

Morphological Features ◽

Training Data ◽

Morphological Operations ◽

Morphological Parameters ◽

Learning Approaches ◽

Residual Cancer Burden ◽

Histological Images ◽

Treated Breast

This paper describes a methodology that extracts key morphological features from histological breast cancer images in order to automatically assess Tumour Cellularity (TC) in Neo-Adjuvant treatment (NAT) patients. The response to NAT gives information on therapy efficacy and it is measured by the residual cancer burden index, which is composed of two metrics: TC and the assessment of lymph nodes. The data consist of whole slide images (WSIs) of breast tissue stained with Hematoxylin and Eosin (H&E) released in the 2019 SPIE Breast Challenge. The methodology proposed is based on traditional computer vision methods (K-means, watershed segmentation, Otsu’s binarisation, and morphological operations), implementing colour separation, segmentation, and feature extraction. Correlation between morphological features and the residual TC after a NAT treatment was examined. Linear regression and statistical methods were used and twenty-two key morphological parameters from the nuclei, epithelial region, and the full image were extracted. Subsequently, an automated TC assessment that was based on Machine Learning (ML) algorithms was implemented and trained with only selected key parameters. The methodology was validated with the score assigned by two pathologists through the intra-class correlation coefficient (ICC). The selection of key morphological parameters improved the results reported over other ML methodologies and it was very close to deep learning methodologies. These results are encouraging, as a traditionally-trained ML algorithm can be useful when limited training data are available preventing the use of deep learning approaches.

Download Full-text

A general approach for improving deep learning-based medical relation extraction using a pre-trained model and fine-tuning

Database ◽

10.1093/database/baz116 ◽

2019 ◽

Vol 2019 ◽

Cited By ~ 2

Author(s):

Tao Chen ◽

Mingfen Wu ◽

Hexi Li

Keyword(s):

Deep Learning ◽

Large Scale ◽

Relation Extraction ◽

Training Model ◽

Biomedical Literature ◽

Training Data ◽

Fine Tuning ◽

Learning Approaches ◽

Additional Time ◽

Clinical Records

Abstract The automatic extraction of meaningful relations from biomedical literature or clinical records is crucial in various biomedical applications. Most of the current deep learning approaches for medical relation extraction require large-scale training data to prevent overfitting of the training model. We propose using a pre-trained model and a fine-tuning technique to improve these approaches without additional time-consuming human labeling. Firstly, we show the architecture of Bidirectional Encoder Representations from Transformers (BERT), an approach for pre-training a model on large-scale unstructured text. We then combine BERT with a one-dimensional convolutional neural network (1d-CNN) to fine-tune the pre-trained model for relation extraction. Extensive experiments on three datasets, namely the BioCreative V chemical disease relation corpus, traditional Chinese medicine literature corpus and i2b2 2012 temporal relation challenge corpus, show that the proposed approach achieves state-of-the-art results (giving a relative improvement of 22.2, 7.77, and 38.5% in F1 score, respectively, compared with a traditional 1d-CNN classifier). The source code is available at https://github.com/chentao1999/MedicalRelationExtraction.

Download Full-text

Enzymatic Weight Update Algorithm for DNA-Based Molecular Learning

Molecules ◽

10.3390/molecules24071409 ◽

2019 ◽

Vol 24 (7) ◽

pp. 1409 ◽

Cited By ~ 1

Author(s):

Christina Baek ◽

Sang-Woo Lee ◽

Beom-Jin Lee ◽

Dong-Hyun Kwak ◽

Byoung-Tak Zhang

Keyword(s):

Machine Learning ◽

Dna Sequences ◽

Dna Nanotechnology ◽

Search Space ◽

Molecular Computing ◽

Training Data ◽

Molecular Systems ◽

Novel Approach ◽

Biological Substrates

Recent research in DNA nanotechnology has demonstrated that biological substrates can be used for computing at a molecular level. However, in vitro demonstrations of DNA computations use preprogrammed, rule-based methods which lack the adaptability that may be essential in developing molecular systems that function in dynamic environments. Here, we introduce an in vitro molecular algorithm that ‘learns’ molecular models from training data, opening the possibility of ‘machine learning’ in wet molecular systems. Our algorithm enables enzymatic weight update by targeting internal loop structures in DNA and ensemble learning, based on the hypernetwork model. This novel approach allows massively parallel processing of DNA with enzymes for specific structural selection for learning in an iterative manner. We also introduce an intuitive method of DNA data construction to dramatically reduce the number of unique DNA sequences needed to cover the large search space of feature sets. By combining molecular computing and machine learning the proposed algorithm makes a step closer to developing molecular computing technologies for future access to more intelligent molecular systems.

Download Full-text

BUILDING GENERALIZATION USING DEEP LEARNING

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xlii-4-565-2018 ◽

2018 ◽

Vol XLII-4 ◽

pp. 565-572 ◽

Cited By ~ 4

Author(s):

M. Sester ◽

Y. Feng ◽

F. Thiemann

Keyword(s):

Computer Vision ◽

Deep Learning ◽

Physical Reality ◽

Training Data ◽

Data Sets ◽

Learning Approaches ◽

Depth Analysis ◽

Map Series ◽

Training Examples ◽

Future Work

<p><strong>Abstract.</strong> Cartographic generalization is a problem, which poses interesting challenges to automation. Whereas plenty of algorithms have been developed for the different sub-problems of generalization (e.g. simplification, displacement, aggregation), there are still cases, which are not generalized adequately or in a satisfactory way. The main problem is the interplay between different operators. In those cases the benchmark is the human operator, who is able to design an aesthetic and correct representation of the physical reality.</p><p>Deep Learning methods have shown tremendous success for interpretation problems for which algorithmic methods have deficits. A prominent example is the classification and interpretation of images, where deep learning approaches outperform the traditional computer vision methods. In both domains &ndash; computer vision and cartography &ndash; humans are able to produce a solution; a prerequisite for this is, that there is the possibility to generate many training examples for the different cases. Thus, the idea in this paper is to employ Deep Learning for cartographic generalizations tasks, especially for the task of building generalization. An advantage of this task is the fact that many training data sets are available from given map series. The approach is a first attempt using an existing network.</p><p>In the paper, the details of the implementation will be reported, together with an in depth analysis of the results. An outlook on future work will be given.</p>

Download Full-text

Review of Deep Learning Methods in Robotic Grasp Detection

Multimodal Technologies and Interaction ◽

10.3390/mti2030057 ◽

2018 ◽

Vol 2 (3) ◽

pp. 57 ◽

Cited By ~ 40

Author(s):

Shehan Caldera ◽

Alexander Rassau ◽

Douglas Chai

Keyword(s):

Deep Learning ◽

Language Processing ◽

General Purpose ◽

Training Data ◽

Learning Approaches ◽

Automated Driving ◽

Learning Methods ◽

Robotic Vision ◽

Object A ◽

Robotic Grasp

For robots to attain more general-purpose utility, grasping is a necessary skill to master. Such general-purpose robots may use their perception abilities to visually identify grasps for a given object. A grasp describes how a robotic end-effector can be arranged to securely grab an object and successfully lift it without slippage. Traditionally, grasp detection requires expert human knowledge to analytically form the task-specific algorithm, but this is an arduous and time-consuming approach. During the last five years, deep learning methods have enabled significant advancements in robotic vision, natural language processing, and automated driving applications. The successful results of these methods have driven robotics researchers to explore the use of deep learning methods in task-generalised robotic applications. This paper reviews the current state-of-the-art in regards to the application of deep learning methods to generalised robotic grasping and discusses how each element of the deep learning approach has improved the overall performance of robotic grasp detection. Several of the most promising approaches are evaluated and the most suitable for real-time grasp detection is identified as the one-shot detection method. The availability of suitable volumes of appropriate training data is identified as a major obstacle for effective utilisation of the deep learning approaches, and the use of transfer learning techniques is proposed as a potential mechanism to address this. Finally, current trends in the field and future potential research directions are discussed.

Download Full-text

Extensibility of U-Net Neural Network Model for Hydrographic Feature Extraction and Implications for Hydrologic Modeling

Remote Sensing ◽

10.3390/rs13122368 ◽

2021 ◽

Vol 13 (12) ◽

pp. 2368

Author(s):

Lawrence V. Stanislawski ◽

Ethan J. Shavers ◽

Shaowen Wang ◽

Zhe Jiang ◽

E. Lynn Usery ◽

...

Keyword(s):

Feature Extraction ◽

Deep Learning ◽

Surface Water ◽

High Performance ◽

Land Development ◽

Training Data ◽

Topographic Map ◽

Feature Maps ◽

Flow Routing ◽

Novel Approach

Accurate maps of regional surface water features are integral for advancing ecologic, atmospheric and land development studies. The only comprehensive surface water feature map of Alaska is the National Hydrography Dataset (NHD). NHD features are often digitized representations of historic topographic map blue lines and may be outdated. Here we test deep learning methods to automatically extract surface water features from airborne interferometric synthetic aperture radar (IfSAR) data to update and validate Alaska hydrographic databases. U-net artificial neural networks (ANN) and high-performance computing (HPC) are used for supervised hydrographic feature extraction within a study area comprised of 50 contiguous watersheds in Alaska. Surface water features derived from elevation through automated flow-routing and manual editing are used as training data. Model extensibility is tested with a series of 16 U-net models trained with increasing percentages of the study area, from about 3 to 35 percent. Hydrography is predicted by each of the models for all watersheds not used in training. Input raster layers are derived from digital terrain models, digital surface models, and intensity images from the IfSAR data. Results indicate about 15 percent of the study area is required to optimally train the ANN to extract hydrography when F1-scores for tested watersheds average between 66 and 68. Little benefit is gained by training beyond 15 percent of the study area. Fully connected hydrographic networks are generated for the U-net predictions using a novel approach that constrains a D-8 flow-routing approach to follow U-net predictions. This work demonstrates the ability of deep learning to derive surface water feature maps from complex terrain over a broad area.

Download Full-text

Simulation-based Data Augmentation for the Quality Inspection of Structural Adhesive with Deep Learning

10.36227/techrxiv.14287334 ◽

2021 ◽

Author(s):

Ricardo Peres ◽

Magno Guedes ◽

Fábio Miranda ◽

José Barata

Keyword(s):

Deep Learning ◽

Data Augmentation ◽

Data Availability ◽

Training Data ◽

Learning Approaches ◽

Automotive Parts ◽

Novel Method ◽

The Cost ◽

Structural Adhesive ◽

Data Context

<div>The advent of Industry 4.0 has shown the tremendous transformative potential of combining artificial intelligence, cyber-physical systems and Internet of Things concepts in industrial settings. Despite this, data availability is still a major roadblock for the successful adoption of data-driven solutions, particularly concerning deep learning approaches in manufacturing. Specifically in the quality control domain, annotated defect data can often be costly, time-consuming and inefficient to obtain, potentially compromising the viability of deep learning approaches due to data scarcity. In this context, we propose a novel method for generating annotated synthetic training data for automated quality inspections of structural adhesive applications, validated in an industrial cell for automotive parts. Our approach greatly reduces the cost of training deep learning models for this task, while simultaneously improving their performance in a scarce manufacturing data context with imbalanced training sets by 3.1% ([email protected]). Additional results can be seen at https://git.io/Jtc4b.</div>

Download Full-text