scholarly journals Three-Dimensional Structural Geological Modeling Using Graph Neural Networks

2021 ◽  
Author(s):  
Michael Hillier ◽  
Florian Wellmann ◽  
Boyan Brodaric ◽  
Eric de Kemp ◽  
Ernst Schetselaar
Keyword(s):  
Neural Networks ◽  
Three Dimensional ◽  
Classification Problem ◽  
Future Research ◽  
Data Sampling ◽  
Neural Network Approach ◽  
Geological Unit ◽  
Point Data ◽  
Complex Settings ◽  
Graph Neural Networks

AbstractThree-dimensional structural geomodels are increasingly being used for a wide variety of scientific and societal purposes. Most advanced methods for generating these models are implicit approaches, but they suffer limitations in the types of interpolation constraints permitted, which can lead to poor modeling in structurally complex settings. A geometric deep learning approach, using graph neural networks, is presented in this paper as an alternative to classical implicit interpolation that is driven by a learning through training paradigm. The graph neural network approach consists of a developed architecture utilizing unstructured meshes as graphs on which coupled implicit and discrete geological unit modeling is performed, with the latter treated as a classification problem. The architecture generates three-dimensional structural models constrained by scattered point data, sampling geological units and interfaces as well as planar and linear orientations. The modeling capacity of the architecture for representing geological structures is demonstrated from its application on two diverse case studies. The benefits of the approach are (1) its ability to provide an expressive framework for incorporating interpolation constraints using loss functions and (2) its capacity to deal with both continuous and discrete properties simultaneously. Furthermore, a framework is established for future research for which additional geological constraints can be integrated into the modeling process.

Using Graph Neural Networks for 3-D Structural Geological Modelling

2021 ◽  
Author(s):  
Michael Hillier ◽  
Florian Wellmann ◽  
Boyan Brodaric ◽  
Eric de Kemp ◽  
Ernst Schetselaar
Keyword(s):  
Neural Networks ◽  
Scalar Fields ◽  
Loss Functions ◽  
Data Sampling ◽  
Geological Modelling ◽  
Geological Unit ◽  
Geological Features ◽  
Point Data ◽  
Graph Neural Networks ◽  
Geological Units

<p>A new approach for constrained 3-D structural geological modelling using Graph Neural Networks (GNN) has been developed that is driven by a learning through training paradigm. Graph neural networks are an emerging deep learning model for graph structured data which can produce vector embeddings of graph elements including nodes, edges, and graphs themselves, useful for various learning objectives. In this work our graphs represent unstructured volumetric meshes. Our developed GNN architecture can generate spatially interpolated implicit scalar fields and discrete geological unit predictions on graph nodes (e.g. mesh vertices) to construct 3-D structural models. Interpolations are constrained by scattered point data sampling geological units, interfaces, as well as linear and planar orientation measurements. Interpolation constraints are incorporated into the neural architecture using loss functions associated with each constraint type that measure the error between the network’s predictions and data observations. This presentation will describe key concepts involved within this approach including vector embeddings, spatial-based convolutions on graphs, and loss functions for structural geological features. In addition, several modelling results will be given that demonstrate the capabilities and potential of GNNs for representing geological structures.</p>

MetaLearning with Graph Neural Networks

2021 ◽  
Vol 23 (2) ◽  
pp. 13-22
Author(s):  
Debmalya Mandal ◽  
Sourav Medya ◽  
Brian Uzzi ◽  
Charu Aggarwal
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Future Research ◽  
Open Problems ◽  
Research Directions ◽  
Graph Problems ◽  
Meta Learning ◽  
Comprehensive Survey ◽  
Future Research Directions ◽  
Graph Neural Networks

Graph Neural Networks (GNNs), a generalization of deep neural networks on graph data have been widely used in various domains, ranging from drug discovery to recommender systems. However, GNNs on such applications are limited when there are few available samples. Meta-learning has been an important framework to address the lack of samples in machine learning, and in recent years, researchers have started to apply meta-learning to GNNs. In this work, we provide a comprehensive survey of different metalearning approaches involving GNNs on various graph problems showing the power of using these two approaches together. We categorize the literature based on proposed architectures, shared representations, and applications. Finally, we discuss several exciting future research directions and open problems.

Graph-based knowledge tracing: Modeling student proficiency using graph neural networks

2021 ◽  
pp. 1-16
Author(s):  
Hiromi Nakagawa ◽  
Yusuke Iwasawa ◽  
Yutaka Matsuo
Keyword(s):  
Neural Networks ◽  
Student Performance ◽  
Classification Problem ◽  
Computer Assisted ◽  
Graph Structure ◽  
Inductive Bias ◽  
Additional Information ◽  
Knowledge Tracing ◽  
Open Datasets ◽  
Graph Neural Networks

Recent advancements in computer-assisted learning systems have caused an increase in the research in knowledge tracing, wherein student performance is predicted over time. Student coursework can potentially be structured as a graph. Incorporating this graph-structured nature into a knowledge tracing model as a relational inductive bias can improve its performance; however, previous methods, such as deep knowledge tracing, did not consider such a latent graph structure. Inspired by the recent successes of graph neural networks (GNNs), we herein propose a GNN-based knowledge tracing method, i.e., graph-based knowledge tracing. Casting the knowledge structure as a graph enabled us to reformulate the knowledge tracing task as a time-series node-level classification problem in the GNN. As the knowledge graph structure is not explicitly provided in most cases, we propose various implementations of the graph structure. Empirical validations on two open datasets indicated that our method could potentially improve the prediction of student performance and demonstrated more interpretable predictions compared to those of the previous methods, without the requirement of any additional information.

scholarly journals Graph Neural Networks: Taxonomy, Advances, and Trends

2022 ◽  
Vol 13 (1) ◽  
pp. 1-54
Author(s):  
Yu Zhou ◽  
Haixia Zheng ◽  
Xin Huang ◽  
Shufeng Hao ◽  
Dengao Li ◽  
...  
Keyword(s):  
Neural Networks ◽  
Real World ◽  
Research Community ◽  
Future Research ◽  
Research Directions ◽  
Comprehensive Review ◽  
Future Research Directions ◽  
Graph Neural Networks ◽  
Low Dimensional

Graph neural networks provide a powerful toolkit for embedding real-world graphs into low-dimensional spaces according to specific tasks. Up to now, there have been several surveys on this topic. However, they usually lay emphasis on different angles so that the readers cannot see a panorama of the graph neural networks. This survey aims to overcome this limitation and provide a systematic and comprehensive review on the graph neural networks. First of all, we provide a novel taxonomy for the graph neural networks, and then refer to up to 327 relevant literatures to show the panorama of the graph neural networks. All of them are classified into the corresponding categories. In order to drive the graph neural networks into a new stage, we summarize four future research directions so as to overcome the challenges faced. It is expected that more and more scholars can understand and exploit the graph neural networks and use them in their research community.

scholarly journals Graph Neural Networks with Multiple Feature Extraction Paths for Chemical Property Estimation

Molecules ◽  
2021 ◽  
Vol 26 (11) ◽  
pp. 3125
Author(s):  
Sho Ishida ◽  
Tomo Miyazaki ◽  
Yoshihiro Sugaya ◽  
Shinichiro Omachi
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Feature Extraction ◽  
Chemical Property ◽  
Structural Information ◽  
Three Dimensional ◽  
Property Estimation ◽  
Multiple Feature ◽  
Multiple Structures ◽  
Graph Neural Networks

Feature extraction is essential for chemical property estimation of molecules using machine learning. Recently, graph neural networks have attracted attention for feature extraction from molecules. However, existing methods focus only on specific structural information, such as node relationship. In this paper, we propose a novel graph convolutional neural network that performs feature extraction with simultaneously considering multiple structures. Specifically, we propose feature extraction paths specialized in node, edge, and three-dimensional structures. Moreover, we propose an attention mechanism to aggregate the features extracted by the paths. The attention aggregation enables us to select useful features dynamically. The experimental results showed that the proposed method outperformed previous methods.

scholarly journals Graph Convolutional Networks by Architecture Search for PolSAR Image Classification

2021 ◽  
Vol 13 (7) ◽  
pp. 1404
Author(s):  
Hongying Liu ◽  
Derong Xu ◽  
Tianwen Zhu ◽  
Fanhua Shang ◽  
Yuanyuan Liu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Large Scale ◽  
Spatial Relations ◽  
Classification Problem ◽  
Search Method ◽  
Sample Distribution ◽  
Convolutional Networks ◽  
Training Samples ◽  
Graph Neural Networks

Classification of polarimetric synthetic aperture radar (PolSAR) images has achieved good results due to the excellent fitting ability of neural networks with a large number of training samples. However, the performance of most convolutional neural networks (CNNs) degrades dramatically when only a few labeled training samples are available. As one well-known class of semi-supervised learning methods, graph convolutional networks (GCNs) have gained much attention recently to address the classification problem with only a few labeled samples. As the number of layers grows in the network, the parameters dramatically increase. It is challenging to determine an optimal architecture manually. In this paper, we propose a neural architecture search method based GCN (ASGCN) for the classification of PolSAR images. We construct a novel graph whose nodes combines both the physical features and spatial relations between pixels or samples to represent the image. Then we build a new searching space whose components are empirically selected from some graph neural networks for architecture search and develop the differentiable architecture search method to construction our ASGCN. Moreover, to address the training of large-scale images, we present a new weighted mini-batch algorithm to reduce the computing memory consumption and ensure the balance of sample distribution, and also analyze and compare with other similar training strategies. Experiments on several real-world PolSAR datasets show that our method has improved the overall accuracy as much as 3.76% than state-of-the-art methods.

scholarly journals From Auto-encoders to Capsule Networks: A Survey

2021 ◽  
Vol 229 ◽  
pp. 01048
Author(s):  
Omaima El Alaoui-Elfels ◽  
Taoufiq Gadi
Keyword(s):  
Neural Networks ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Three Dimensional ◽  
Future Research ◽  
Two Dimensional ◽  
One Dimensional ◽  
Deep Learning Algorithm ◽  
New Generation ◽  
Open Issues

Convolutional Neural Networks are a very powerful Deep Learning algorithm used in image processing, object classification and segmentation. They are very robust in extracting features from data and largely used in several domains. Nonetheless, they require a large number of training datasets and relations between features get lost in the Max-pooling step, which can lead to a wrong classification. Capsule Networks (CapsNets) were introduced to overcome these limitations by extracting features and their pose using capsules instead of neurons. This technique shows an impressive performance in one-dimensional, two-dimensional and three-dimensional datasets as well as in sparse datasets. In this paper, we present an initial understanding of CapsNets, their concept, structure and learning algorithm. We introduce the progress made by CapsNets from their introduction in 2011 until 2020. We compare different CapsNets series to demonstrate strengths and challenges. Finally, we quote different implementations of Capsule Networks and show their robustness in a variety of domains. This survey provides the state-of-the-art of Capsule Networks and allows other researchers to get a clear view of this new field. Besides, we discuss the open issues and the promising directions of future research, which may lead to a new generation of CapsNets.

scholarly journals Predicting The Risk and Timing of Bipolar Disorder In Offspring of Bipolar Parents: Exploring The Utility of a Neural Network Approach

2020 ◽  
Author(s):  
Alysha Cooper ◽  
Julie Horrocks ◽  
Sarah Margaret Goodday ◽  
Charles Keown-Stoneman ◽  
Anne Duffy
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
At Risk ◽  
Bipolar Disorder ◽  
High Risk ◽  
Logistic Model ◽  
Predictive Performance ◽  
Future Research ◽  
Routine Practice ◽  
Neural Network Approach

Abstract BackgroundBipolar disorder onset peaks over early adulthood and confirmed family history is a robust risk factor. However, penetrance within families varies and most children of bipolar parents will not develop the illness. Individualized risk prediction would be helpful for identifying those young people most at risk and to inform targeted intervention. Using prospectively collected data from the Canadian Flourish High-Risk Offspring cohort study available in routine practice, we explored the use of a neural network, known as the Partial Logistic Artificial Neural Network (PLANN) to predict the time to diagnosis of bipolar spectrum disorders Results Overall, for predictive performance, PLANN outperformed the more traditional logistic model for one year, three year and five-year predictions. PLANN was better able to discriminate or rank individuals based on their risk of developing bipolar disorder, better able to predict the probability of developing bipolar disorder and had higher accuracy than the logistic model. ConclusionsThis evaluation of PLANN is a useful step in the investigation of using neural networks as tools in the prediction of diagnosis of mental health for at-risk individuals and demonstrated the potential that neural networks have in this field. Future research is needed to replicate these findings in a separate high-risk sample.

Video Classification Using 3D Convolutional Neural Network

2021 ◽  
pp. 1-18
Author(s):  
K. Jairam Naik ◽  
Annukriti Soni
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Spatial Information ◽  
Three Dimensional ◽  
Classification Problem ◽  
Video Classification ◽  
Convolutional Networks ◽  
3D Cnn ◽  
Deep Learning Neural Network

Since video includes both temporal and spatial features, it has become a fascinating classification problem. Each frame within a video holds important information called spatial information, as does the context of that frame relative to the frames before it in time called temporal information. Several methods have been invented for video classification, but each one is suffering from its own drawback. One of such method is called convolutional neural networks (CNN) model. It is a category of deep learning neural network model that can turn directly on the underdone inputs. However, such models are recently limited to handling two-dimensional inputs only. This chapter implements a three-dimensional convolutional neural networks (CNN) model for video classification to analyse the classification accuracy gained using the 3D CNN model. The 3D convolutional networks are preferred for video classification since they inherently apply convolutions in the 3D space.

scholarly journals From Auto-encoders to Capsule Networks: A Survey

2021 ◽  
Vol 229 ◽  
pp. 01003
Author(s):  
Omaima El Alaoui-Elfels ◽  
Taoufiq Gadi
Keyword(s):  
Neural Networks ◽  
Image Processing ◽  
Deep Learning ◽  
Learning Algorithm ◽  
Three Dimensional ◽  
Future Research ◽  
Two Dimensional ◽  
One Dimensional ◽  
New Generation ◽  
Open Issues

Convolutional Neural Networks are a very powerful Deep Learning structure used in image processing, object classification and segmentation. They are very robust in extracting features from data and largely used in several domains. Nonetheless, they require a large number of training datasets and relations between features get lost in the Max-pooling step, which can lead to a wrong classification. Capsule Networks(CapsNets) were introduced to overcome these limitations by extracting features and their pose using capsules instead of neurons. This technique shows an impressive performance in one-dimensional, two-dimensional and three-dimensional datasets as well as in sparse datasets. In this paper, we present an initial understanding of CapsNets, their concept, structure and learning algorithm. We introduce the progress made by CapsNets from their introduction in 2011 until 2020. We compare different CapsNets series architectures to demonstrate strengths and challenges. Finally, we quote different implementations of Capsule Networks and show their robustness in a variety of domains. This survey provides the state-of-theartof Capsule Networks and allows other researchers to get a clear view of this new field. Besides, we discuss the open issues and the promising directions of future research, which may lead to a new generation of CapsNets.

Sign in / Sign up

Export Citation Format

Share Document