Neural Networks for Adaptive Processing of Structured Data

2001 ◽  
pp. 5-12 ◽  
Author(s):  
Alessandro Sperduti
Keyword(s):  
Neural Networks ◽  
Structured Data ◽  
Adaptive Processing

scholarly journals Multi-View Attribute Graph Convolution Networks for Clustering

2020 ◽  
Author(s):  
Jiafeng Cheng ◽  
Qianqian Wang ◽  
Zhiqiang Tao ◽  
Deyan Xie ◽  
Quanxue Gao
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Graph Embedding ◽  
Structured Data ◽  
Attention Networks ◽  
Graph Data ◽  
Graph Reconstruction ◽  
Node Attributes ◽  
Graph Neural Networks ◽  
Geometric Relationship

Graph neural networks (GNNs) have made considerable achievements in processing graph-structured data. However, existing methods can not allocate learnable weights to different nodes in the neighborhood and lack of robustness on account of neglecting both node attributes and graph reconstruction. Moreover, most of multi-view GNNs mainly focus on the case of multiple graphs, while designing GNNs for solving graph-structured data of multi-view attributes is still under-explored. In this paper, we propose a novel Multi-View Attribute Graph Convolution Networks (MAGCN) model for the clustering task. MAGCN is designed with two-pathway encoders that map graph embedding features and learn the view-consistency information. Specifically, the first pathway develops multi-view attribute graph attention networks to reduce the noise/redundancy and learn the graph embedding features for each multi-view graph data. The second pathway develops consistent embedding encoders to capture the geometric relationship and probability distribution consistency among different views, which adaptively finds a consistent clustering embedding space for multi-view attributes. Experiments on three benchmark graph datasets show the superiority of our method compared with several state-of-the-art algorithms.

Twenty-five years of information extraction

2019 ◽  
Vol 25 (06) ◽  
pp. 677-692
Author(s):  
Ralph Grishman
Keyword(s):  
Neural Networks ◽  
Information Extraction ◽  
Data Base ◽  
Information Content ◽  
Structured Data ◽  
Essential Step ◽  
The Past ◽  
Unstructured Text ◽  
The Us ◽  
Us Government

AbstractInformation extraction is the process of converting unstructured text into a structured data base containing selected information from the text. It is an essential step in making the information content of the text usable for further processing. In this paper, we describe how information extraction has changed over the past 25 years, moving from hand-coded rules to neural networks, with a few stops on the way. We connect these changes to research advances in NLP and to the evaluations organized by the US Government.

scholarly journals A self-organizing map for adaptive processing of structured data

2003 ◽  
Vol 14 (3) ◽  
pp. 491-505 ◽  
Author(s):  
M. Hagenbuchner ◽  
A. Sperduti ◽  
Ah Chung Tsoi
Keyword(s):  
Structured Data ◽  
Adaptive Processing ◽  
Self Organizing Map ◽  
Self Organizing

scholarly journals Fast Quaternion Product Units for Learning Disentangled Representations in SO(3)

2022 ◽  
Author(s):  
Shaofei Qin ◽  
Xuan Zhang ◽  
Hongteng Xu ◽  
Yi Xu
Keyword(s):  
Neural Networks ◽  
Real World ◽  
Message Passing ◽  
Point Clouds ◽  
Rotation Group ◽  
Structured Data ◽  
Data Indexing ◽  
Basic Module ◽  
Real Neuron ◽  
Rotation Groups

Real-world 3D structured data like point clouds and skeletons often can be represented as data in a 3D rotation group (denoted as $\mathbb{SO}(3)$). However, most existing neural networks are tailored for the data in the Euclidean space, which makes the 3D rotation data not closed under their algebraic operations and leads to sub-optimal performance in 3D-related learning tasks. To resolve the issues caused by the above mismatching between data and model, we propose a novel non-real neuron model called \textit{quaternion product unit} (QPU) to represent data on 3D rotation groups. The proposed QPU leverages quaternion algebra and the law of the 3D rotation group, representing 3D rotation data as quaternions and merging them via a weighted chain of Hamilton products. We demonstrate that the QPU mathematically maintains the $\mathbb{SO}(3)$ structure of the 3D rotation data during the inference process and disentangles the 3D representations into ``rotation-invariant'' features and ``rotation-equivariant'' features, respectively. Moreover, we design a fast QPU to accelerate the computation of QPU. The fast QPU applies a tree-structured data indexing process, and accordingly, leverages the power of parallel computing, which reduces the computational complexity of QPU in a single thread from $\mathcal{O}(N)$ to $\mathcal {O}(\log N)$. Taking the fast QPU as a basic module, we develop a series of quaternion neural networks (QNNs), including quaternion multi-layer perceptron (QMLP), quaternion message passing (QMP), and so on. In addition, we make the QNNs compatible with conventional real-valued neural networks and applicable for both skeletons and point clouds. Experiments on synthetic and real-world 3D tasks show that the QNNs based on our fast QPUs are superior to state-of-the-art real-valued models, especially in the scenarios requiring the robustness to random rotations.<br>

scholarly journals Graph Universal Adversarial Attacks: A Few Bad Actors Ruin Graph Learning Models

2021 ◽  
Author(s):  
Xiao Zang ◽  
Yi Xie ◽  
Jie Chen ◽  
Bo Yuan
Keyword(s):  
Neural Networks ◽  
Success Rate ◽  
Deep Neural Networks ◽  
Model Performance ◽  
Graph Model ◽  
Structured Data ◽  
Graph Structure ◽  
Learning Models ◽  
Security Threat ◽  
Anchor Nodes

Deep neural networks, while generalize well, are known to be sensitive to small adversarial perturbations. This phenomenon poses severe security threat and calls for in-depth investigation of the robustness of deep learning models. With the emergence of neural networks for graph structured data, similar investigations are urged to understand their robustness. It has been found that adversarially perturbing the graph structure and/or node features may result in a significant degradation of the model performance. In this work, we show from a different angle that such fragility similarly occurs if the graph contains a few bad-actor nodes, which compromise a trained graph neural network through flipping the connections to any targeted victim. Worse, the bad actors found for one graph model severely compromise other models as well. We call the bad actors ``anchor nodes'' and propose an algorithm, named GUA, to identify them. Thorough empirical investigations suggest an interesting finding that the anchor nodes often belong to the same class; and they also corroborate the intuitive trade-off between the number of anchor nodes and the attack success rate. For the dataset Cora which contains 2708 nodes, as few as six anchor nodes will result in an attack success rate higher than 80% for GCN and other three models.

Skeleton-Based Action Recognition with Joint Coordinates as Feature Using Neural Oblivious Decision Ensembles

2021 ◽  
Author(s):  
Fakhrul Aniq Hakimi Nasrul ’Alam ◽  
Mohd. Ibrahim Shapiai ◽  
Uzma Batool ◽  
Ahmad Kamal Ramli ◽  
Khairil Ashraf Elias
Keyword(s):  
Neural Networks ◽  
Action Recognition ◽  
Recurrent Neural Networks ◽  
State Of The Art ◽  
Structured Data ◽  
Joint Coordinates ◽  
Label Prediction ◽  
Within Subjects ◽  
High Degree ◽  
Behaviour Recognition

Recognition of human behavior is critical in video monitoring, human-computer interaction, video comprehension, and virtual reality. The key problem with behaviour recognition in video surveillance is the high degree of variation between and within subjects. Numerous studies have suggested background-insensitive skeleton-based as the proven detection technique. The present state-of-the-art approaches to skeleton-based action recognition rely primarily on Recurrent Neural Networks (RNN) and Convolution Neural Networks (CNN). Both methods take dynamic human skeleton as the input to the network. We chose to handle skeleton data differently, relying solely on its skeleton joint coordinates as the input. The skeleton joints’ positions are defined in (x, y) coordinates. In this paper, we investigated the incorporation of the Neural Oblivious Decision Ensemble (NODE) into our proposed action classifier network. The skeleton is extracted using a pose estimation technique based on the Residual Network (ResNet). It extracts the 2D skeleton of 18 joints for each detected body. The joint coordinates of the skeleton are stored in a table in the form of rows and columns. Each row represents the position of the joints. The structured data are fed into NODE for label prediction. With the proposed network, we obtain 97.5% accuracy on RealWorld (HAR) dataset. Experimental results show that the proposed network outperforms one the state-of-the-art approaches by 1.3%. In conclusion, NODE is a promising deep learning technique for structured data analysis as compared to its machine learning counterparts such as the GBDT packages; Catboost, and XGBoost.

scholarly journals Message Passing Attention Networks for Document Understanding

2020 ◽  
Vol 34 (05) ◽  
pp. 8544-8551 ◽  
Author(s):  
Giannis Nikolentzos ◽  
Antoine Tixier ◽  
Michalis Vazirgiannis
Keyword(s):  
Neural Networks ◽  
Text Classification ◽  
Message Passing ◽  
State Of The Art ◽  
Structured Data ◽  
Attention Networks ◽  
Document Understanding ◽  
Standard Text ◽  
Graph Neural Networks ◽  
The Impact

Graph neural networks have recently emerged as a very effective framework for processing graph-structured data. These models have achieved state-of-the-art performance in many tasks. Most graph neural networks can be described in terms of message passing, vertex update, and readout functions. In this paper, we represent documents as word co-occurrence networks and propose an application of the message passing framework to NLP, the Message Passing Attention network for Document understanding (MPAD). We also propose several hierarchical variants of MPAD. Experiments conducted on 10 standard text classification datasets show that our architectures are competitive with the state-of-the-art. Ablation studies reveal further insights about the impact of the different components on performance. Code is publicly available at: https://github.com/giannisnik/mpad.

scholarly journals Fast Quaternion Product Units for Learning Disentangled Representations in SO(3)

2022 ◽  
Author(s):  
Shaofei Qin ◽  
Xuan Zhang ◽  
Hongteng Xu ◽  
Yi Xu
Keyword(s):  
Neural Networks ◽  
Real World ◽  
Message Passing ◽  
Point Clouds ◽  
Rotation Group ◽  
Structured Data ◽  
Data Indexing ◽  
Basic Module ◽  
Real Neuron ◽  
Rotation Groups

Real-world 3D structured data like point clouds and skeletons often can be represented as data in a 3D rotation group (denoted as $\mathbb{SO}(3)$). However, most existing neural networks are tailored for the data in the Euclidean space, which makes the 3D rotation data not closed under their algebraic operations and leads to sub-optimal performance in 3D-related learning tasks. To resolve the issues caused by the above mismatching between data and model, we propose a novel non-real neuron model called \textit{quaternion product unit} (QPU) to represent data on 3D rotation groups. The proposed QPU leverages quaternion algebra and the law of the 3D rotation group, representing 3D rotation data as quaternions and merging them via a weighted chain of Hamilton products. We demonstrate that the QPU mathematically maintains the $\mathbb{SO}(3)$ structure of the 3D rotation data during the inference process and disentangles the 3D representations into ``rotation-invariant'' features and ``rotation-equivariant'' features, respectively. Moreover, we design a fast QPU to accelerate the computation of QPU. The fast QPU applies a tree-structured data indexing process, and accordingly, leverages the power of parallel computing, which reduces the computational complexity of QPU in a single thread from $\mathcal{O}(N)$ to $\mathcal {O}(\log N)$. Taking the fast QPU as a basic module, we develop a series of quaternion neural networks (QNNs), including quaternion multi-layer perceptron (QMLP), quaternion message passing (QMP), and so on. In addition, we make the QNNs compatible with conventional real-valued neural networks and applicable for both skeletons and point clouds. Experiments on synthetic and real-world 3D tasks show that the QNNs based on our fast QPUs are superior to state-of-the-art real-valued models, especially in the scenarios requiring the robustness to random rotations.<br>

Sign in / Sign up

Export Citation Format

Share Document