Utilizing High-Dimensional Neural Networks for Pseudo-orthogonalization of Memory Patterns

2014 ◽  
pp. 527-534
Author(s):  
Toshifumi Minemoto ◽  
Teijiro Isokawa ◽  
Haruhiko Nishimura ◽  
Nobuyuki Matsui
Keyword(s):  
Neural Networks ◽  
High Dimensional

Neural networks trained with high-dimensional functions approximation data in high-dimensional space

2021 ◽  
pp. 1-12
Author(s):  
Jian Zheng ◽  
Jianfeng Wang ◽  
Yanping Chen ◽  
Shuping Chen ◽  
Jingjin Chen ◽  
...  
Keyword(s):  
Neural Networks ◽  
Dimensional Space ◽  
Data Distribution ◽  
High Dimensional ◽  
Sufficient Information ◽  
Sufficient Data ◽  
High Dimensional Space ◽  
Positive Effects ◽  
The Neural Networks ◽  
Using Data

Neural networks can approximate data because of owning many compact non-linear layers. In high-dimensional space, due to the curse of dimensionality, data distribution becomes sparse, causing that it is difficulty to provide sufficient information. Hence, the task becomes even harder if neural networks approximate data in high-dimensional space. To address this issue, according to the Lipschitz condition, the two deviations, i.e., the deviation of the neural networks trained using high-dimensional functions, and the deviation of high-dimensional functions approximation data, are derived. This purpose of doing this is to improve the ability of approximation high-dimensional space using neural networks. Experimental results show that the neural networks trained using high-dimensional functions outperforms that of using data in the capability of approximation data in high-dimensional space. We find that the neural networks trained using high-dimensional functions more suitable for high-dimensional space than that of using data, so that there is no need to retain sufficient data for neural networks training. Our findings suggests that in high-dimensional space, by tuning hidden layers of neural networks, this is hard to have substantial positive effects on improving precision of approximation data.

scholarly journals Efficient approximation of solutions of parametric linear transport equations by ReLU DNNs

2021 ◽  
Vol 47 (1) ◽  
Author(s):  
Fabian Laakmann ◽  
Philipp Petersen
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Initial Conditions ◽  
Activation Function ◽  
Transport Equations ◽  
High Dimensional ◽  
Linear Transport ◽  
Approximation Rates ◽  
Curse Of Dimension ◽  
Efficient Approximation

AbstractWe demonstrate that deep neural networks with the ReLU activation function can efficiently approximate the solutions of various types of parametric linear transport equations. For non-smooth initial conditions, the solutions of these PDEs are high-dimensional and non-smooth. Therefore, approximation of these functions suffers from a curse of dimension. We demonstrate that through their inherent compositionality deep neural networks can resolve the characteristic flow underlying the transport equations and thereby allow approximation rates independent of the parameter dimension.

Nonlinear reduction of high-dimensional dynamical systems via neural networks

1994 ◽  
Vol 72 (12) ◽  
pp. 1822-1825 ◽  
Author(s):  
Michael Kirby ◽  
Rick Miranda
Keyword(s):  
Neural Networks ◽  
Dynamical Systems ◽  
High Dimensional

Effective approximation of high-dimensional space using neural networks

2021 ◽  
Author(s):  
Jian Zheng ◽  
Jianfeng Wang ◽  
Yanping Chen ◽  
Shuping Chen ◽  
Jingjin Chen ◽  
...  
Keyword(s):  
Neural Networks ◽  
Dimensional Space ◽  
High Dimensional ◽  
High Dimensional Space ◽  
Effective Approximation

scholarly journals Clustered Echo State Networks for Signal Observation and Frequency Filtering

2020 ◽  
Author(s):  
Laércio Oliveira Junior ◽  
Florian Stelzer ◽  
Liang Zhao
Keyword(s):  
Dynamical System ◽  
Neural Networks ◽  
Input Signal ◽  
Recurrent Neural Networks ◽  
High Dimensional ◽  
Frequency Filtering ◽  
Echo State Networks ◽  
Chaotic Signals ◽  
Network Topologies ◽  
Dimensional Dynamical System

Echo State Networks (ESNs) are recurrent neural networks that map an input signal to a high-dimensional dynamical system, called reservoir, and possess adaptive output weights. The output weights are trained such that the ESN’s output signal fits the desired target signal. Classical reservoirs are sparse and randomly connected networks. In this article, we investigate the effect of different network topologies on the performance of ESNs. Specifically, we use two types of networks to construct clustered reservoirs of ESN: the clustered Erdös–Rényi and the clustered Barabási-Albert network model. Moreover, we compare the performance of these clustered ESNs (CESNs) and classical ESNs with the random reservoir by employing them to two different tasks: frequency filtering and the reconstruction of chaotic signals. By using a clustered topology, one can achieve a significant increase in the ESN’s performance.

Sign in / Sign up

Export Citation Format

Share Document