Modeling higher order adaptivity of a network by multilevel network reification

2020 ◽  
Vol 8 (S1) ◽  
pp. S110-S144 ◽  
Author(s):  
Jan Treur
Keyword(s):  
Network Architecture ◽  
Network Models ◽  
Higher Order ◽  
Second Order ◽  
Adaptation Level ◽  
Network Adaptation ◽  
Base Level ◽  
Generic Architecture ◽  
Adaptation Processes ◽  
Base Network

AbstractIn network models for real-world domains, often network adaptation has to be addressed by incorporating certain network adaptation principles. In some cases, also higher order adaptation occurs: the adaptation principles themselves also change over time. To model such multilevel adaptation processes, it is useful to have some generic architecture. Such an architecture should describe and distinguish the dynamics within the network (base level), but also the dynamics of the network itself by certain adaptation principles (first-order adaptation level), and also the adaptation of these adaptation principles (second-order adaptation level), and may be still more levels of higher order adaptation. This paper introduces a multilevel network architecture for this, based on the notion network reification. Reification of a network occurs when a base network is extended by adding explicit states representing the characteristics of the structure of the base network. It will be shown how this construction can be used to explicitly represent network adaptation principles within a network. When the reified network is itself also reified, also second-order adaptation principles can be explicitly represented. The multilevel network reification construction introduced here is illustrated for an adaptive adaptation principle from social science for bonding based on homophily and one for metaplasticity in Cognitive Neuroscience.

Invariant Object Recognition Using Higher-Order Neural Networks, Line-Segment Spectra and Multi-Resolution Training

1997 ◽  
Vol 08 (03) ◽  
pp. 251-262
Author(s):  
Marijke F. Augusteijn ◽  
Michael C. Winterbottom
Keyword(s):  
Line Segment ◽  
Network Architecture ◽  
Network Size ◽  
Higher Order ◽  
Second Order ◽  
Image Size ◽  
Training Time ◽  
Training Approach ◽  
Invariant Object Recognition ◽  
Improved Performance

A second-order neural network architecture is introduced that achieves invariant recognition with respect to an object's position and orientation in an image. This network does not show the combinatorial growth in network size as image size is increased, which is commonly observed in higher-order architectures. A new concept called an object's line-segment spectrum is introduced. It is argued that the weights of the second-order architecture are determined by these line-segments. Training time then becomes a function of object size rather than image size. The network is tested on the 26 capital letters of the alphabet. It is shown that in this application a multi-resolution training approach leads to reduced training time and improved performance.

Optimal control of higher order differential inclusions with functional constraints

2020 ◽  
Vol 26 ◽  
pp. 37 ◽  
Author(s):  
Elimhan N. Mahmudov
Keyword(s):  
Optimal Control ◽  
Optimality Conditions ◽  
Differential Inclusion ◽  
Differential Inclusions ◽  
Higher Order ◽  
Second Order ◽  
Sufficient Optimality Conditions ◽  
Functional Constraints ◽  
Complementary Slackness ◽  
Complementary Slackness Conditions

The present paper studies the Mayer problem with higher order evolution differential inclusions and functional constraints of optimal control theory (PFC); to this end first we use an interesting auxiliary problem with second order discrete-time and discrete approximate inclusions (PFD). Are proved necessary and sufficient conditions incorporating the Euler–Lagrange inclusion, the Hamiltonian inclusion, the transversality and complementary slackness conditions. The basic concept of obtaining optimal conditions is locally adjoint mappings and equivalence results. Then combining these results and passing to the limit in the discrete approximations we establish new sufficient optimality conditions for second order continuous-time evolution inclusions. This approach and results make a bridge between optimal control problem with higher order differential inclusion (PFC) and constrained mathematical programming problems in finite-dimensional spaces. Formulation of the transversality and complementary slackness conditions for second order differential inclusions play a substantial role in the next investigations without which it is hardly ever possible to get any optimality conditions; consequently, these results are generalized to the problem with an arbitrary higher order differential inclusion. Furthermore, application of these results is demonstrated by solving some semilinear problem with second and third order differential inclusions.

scholarly journals Preserving the Shape of Functions by Applying Multidimensional Schoenberg-Type Operators

Symmetry ◽  
2021 ◽  
Vol 13 (6) ◽  
pp. 1016
Author(s):  
Camelia Liliana Moldovan ◽  
Radu Păltănea
Keyword(s):  
Applied Sciences ◽  
Degree Of Approximation ◽  
Higher Order ◽  
Second Order ◽  
Dimensional Case ◽  
Moduli Of Continuity ◽  
One Dimensional ◽  
Multidimensional Generalization ◽  
The One

The paper presents a multidimensional generalization of the Schoenberg operators of higher order. The new operators are powerful tools that can be used for approximation processes in many fields of applied sciences. The construction of these operators uses a symmetry regarding the domain of definition. The degree of approximation by sequences of such operators is given in terms of the first and the second order moduli of continuity. Extending certain results obtained by Marsden in the one-dimensional case, the property of preservation of monotonicity and convexity is proved.

scholarly journals Higher order Hamiltonian Monte Carlo sampling for cosmological large-scale structure analysis

2021 ◽  
Vol 502 (3) ◽  
pp. 3976-3992
Author(s):  
Mónica Hernández-Sánchez ◽  
Francisco-Shu Kitaura ◽  
Metin Ata ◽  
Claudio Dalla Vecchia
Keyword(s):  
Fourth Order ◽  
Large Scale ◽  
Equations Of Motion ◽  
Large Scale Structure ◽  
Real Space ◽  
Higher Order ◽  
Second Order ◽  
Scale Structure ◽  
Integration Schemes ◽  
Reconstruction Methods

ABSTRACT We investigate higher order symplectic integration strategies within Bayesian cosmic density field reconstruction methods. In particular, we study the fourth-order discretization of Hamiltonian equations of motion (EoM). This is achieved by recursively applying the basic second-order leap-frog scheme (considering the single evaluation of the EoM) in a combination of even numbers of forward time integration steps with a single intermediate backward step. This largely reduces the number of evaluations and random gradient computations, as required in the usual second-order case for high-dimensional cases. We restrict this study to the lognormal-Poisson model, applied to a full volume halo catalogue in real space on a cubical mesh of 1250 h−1 Mpc side and 2563 cells. Hence, we neglect selection effects, redshift space distortions, and displacements. We note that those observational and cosmic evolution effects can be accounted for in subsequent Gibbs-sampling steps within the COSMIC BIRTH algorithm. We find that going from the usual second to fourth order in the leap-frog scheme shortens the burn-in phase by a factor of at least ∼30. This implies that 75–90 independent samples are obtained while the fastest second-order method converges. After convergence, the correlation lengths indicate an improvement factor of about 3.0 fewer gradient computations for meshes of 2563 cells. In the considered cosmological scenario, the traditional leap-frog scheme turns out to outperform higher order integration schemes only when considering lower dimensional problems, e.g. meshes with 643 cells. This gain in computational efficiency can help to go towards a full Bayesian analysis of the cosmological large-scale structure for upcoming galaxy surveys.

scholarly journals Higher-order Recursion Schemes and Collapsible Pushdown Automata: Logical Properties

2021 ◽  
Vol 22 (2) ◽  
pp. 1-37
Author(s):  
Christopher H. Broadbent ◽  
Arnaud Carayol ◽  
C.-H. Luke Ong ◽  
Olivier Serre
Keyword(s):  
Model Checking ◽  
Free Variable ◽  
Higher Order ◽  
Second Order ◽  
Effective Solution ◽  
Parity Games ◽  
Transition Graphs ◽  
Second Order Logic ◽  
Pushdown Automata ◽  
Monadic Second Order Logic

This article studies the logical properties of a very general class of infinite ranked trees, namely, those generated by higher-order recursion schemes. We consider, for both monadic second-order logic and modal -calculus, three main problems: model-checking, logical reflection (a.k.a. global model-checking, that asks for a finite description of the set of elements for which a formula holds), and selection (that asks, if exists, for some finite description of a set of elements for which an MSO formula with a second-order free variable holds). For each of these problems, we provide an effective solution. This is obtained, thanks to a known connection between higher-order recursion schemes and collapsible pushdown automata and on previous work regarding parity games played on transition graphs of collapsible pushdown automata.

scholarly journals Pointwise estimates for higher order convexity preserving polynomial approximation

1994 ◽  
Vol 36 (2) ◽  
pp. 213-233 ◽  
Author(s):  
Jia-Ding Cao ◽  
Heinz H. Gonska
Keyword(s):  
Polynomial Approximation ◽  
Convex Functions ◽  
Higher Order ◽  
Second Order ◽  
Convolution Type ◽  
Shape Preservation ◽  
Moduli Of Continuity ◽  
Pointwise Estimates ◽  
Higher Order Convexity ◽  
Boolean Sum

AbstractDeVore-Gopengauz-type operators have attracted some interest over the recent years. Here we investigate their relationship to shape preservation. We construct certain positive convolution-type operators Hn, s, j which leave the cones of j-convex functions invariant and give Timan-type inequalities for these. We also consider Boolean sum modifications of the operators Hn, s, j show that they basically have the same shape preservation behavior while interpolating at the endpoints of [−1, 1], and also satisfy Telyakovskiῐ- and DeVore-Gopengauz-type inequalities involving the first and second order moduli of continuity, respectively. Our results thus generalize related results by Lorentz and Zeller, Shvedov, Beatson, DeVore, Yu and Leviatan.

HIGHER ORDER COMPLEXITY OF TIME SERIES

2004 ◽  
Vol 14 (08) ◽  
pp. 2979-2990 ◽  
Author(s):  
FANJI GU ◽  
ENHUA SHEN ◽  
XIN MENG ◽  
YANG CAO ◽  
ZHIJIE CAI
Keyword(s):  
Time Series ◽  
Mental Arithmetic ◽  
Higher Order ◽  
Second Order ◽  
Stationary Time Series ◽  
Eeg Signals ◽  
Eyes Closed ◽  
Logistic Mapping ◽  
Original Time ◽  
Different Order

A concept of higher order complexity is proposed in this letter. If a randomness-finding complexity [Rapp & Schmah, 2000] is taken as the complexity measure, the first-order complexity is suggested to be a measure of randomness of the original time series, while the second-order complexity is a measure of its degree of nonstationarity. A different order is associated with each different aspect of complexity. Using logistic mapping repeatedly, some quasi-stationary time series were constructed, the nonstationarity degree of which could be expected theoretically. The estimation of the second-order complexity of these time series shows that the second-order complexities do reflect the degree of nonstationarity and thus can be considered as its indicator. It is also shown that the second-order complexities of the EEG signals from subjects doing mental arithmetic are significantly higher than those from subjects in deep sleep or resting with eyes closed.

Spatial Variability Aware Deep Neural Networks (SVANN): A General Approach

2021 ◽  
Vol 12 (6) ◽  
pp. 1-21
Author(s):  
Jayant Gupta ◽  
Carl Molnar ◽  
Yiqun Xie ◽  
Joe Knight ◽  
Shashi Shekhar
Keyword(s):  
Neural Network ◽  
Spatial Variability ◽  
Network Architecture ◽  
Network Models ◽  
Neural Network Architecture ◽  
Neural Network Models ◽  
Climatic Zones ◽  
The Neural Network ◽  
Plant Hardiness ◽  
Interpretation Model

Spatial variability is a prominent feature of various geographic phenomena such as climatic zones, USDA plant hardiness zones, and terrestrial habitat types (e.g., forest, grasslands, wetlands, and deserts). However, current deep learning methods follow a spatial-one-size-fits-all (OSFA) approach to train single deep neural network models that do not account for spatial variability. Quantification of spatial variability can be challenging due to the influence of many geophysical factors. In preliminary work, we proposed a spatial variability aware neural network (SVANN-I, formerly called SVANN ) approach where weights are a function of location but the neural network architecture is location independent. In this work, we explore a more flexible SVANN-E approach where neural network architecture varies across geographic locations. In addition, we provide a taxonomy of SVANN types and a physics inspired interpretation model. Experiments with aerial imagery based wetland mapping show that SVANN-I outperforms OSFA and SVANN-E performs the best of all.

scholarly journals A data-driven neural network architecture for sentiment analysis

2019 ◽  
Vol 53 (1) ◽  
pp. 2-19 ◽  
Author(s):  
Erion Çano ◽  
Maurizio Morisio
Keyword(s):  
Neural Network ◽  
Sentiment Analysis ◽  
Network Architecture ◽  
Network Models ◽  
Data Sets ◽  
Feature Maps ◽  
Neural Network Architecture ◽  
Neural Network Models ◽  
Content Type ◽  
Max Pooling

Purpose The fabulous results of convolution neural networks in image-related tasks attracted attention of text mining, sentiment analysis and other text analysis researchers. It is, however, difficult to find enough data for feeding such networks, optimize their parameters, and make the right design choices when constructing network architectures. The purpose of this paper is to present the creation steps of two big data sets of song emotions. The authors also explore usage of convolution and max-pooling neural layers on song lyrics, product and movie review text data sets. Three variants of a simple and flexible neural network architecture are also compared. Design/methodology/approach The intention was to spot any important patterns that can serve as guidelines for parameter optimization of similar models. The authors also wanted to identify architecture design choices which lead to high performing sentiment analysis models. To this end, the authors conducted a series of experiments with neural architectures of various configurations. Findings The results indicate that parallel convolutions of filter lengths up to 3 are usually enough for capturing relevant text features. Also, max-pooling region size should be adapted to the length of text documents for producing the best feature maps. Originality/value Top results the authors got are obtained with feature maps of lengths 6–18. An improvement on future neural network models for sentiment analysis could be generating sentiment polarity prediction of documents using aggregation of predictions on smaller excerpt of the entire text.

Sign in / Sign up

Export Citation Format

Share Document