Recurrent Neural Hidden Markov Model for High-order Transition

We propose a method to pay attention to high-order relations among latent states to improve the conventional HMMs that focus only on the latest latent state, since they assume Markov property. To address the high-order relations, we apply an RNN to each sequence of latent states, because the RNN can represent the information of an arbitrary-length sequence with their cell: a fixed-size vector. However, the simplest way, which provides all latent sequences explicitly for the RNN, is intractable due to the combinatorial explosion of the search space of latent states. Thus, we modify the RNN to represent the history of latent states from the beginning of the sequence to the current state with a fixed number of RNN cells whose number is equal to the number of possible states. We conduct experiments on unsupervised POS tagging and synthetic datasets. Experimental results show that the proposed method achieves better performance than previous methods. In addition, the results on the synthetic dataset indicate that the proposed method can capture the high-order relations.

Hypergraph Convolution on Nodes-Hyperedges Network for Semi-Supervised Node Classification

Hypergraphs have shown great power in representing high-order relations among entities, and lots of hypergraph-based deep learning methods have been proposed to learn informative data representations for the node classification problem. However, most of these deep learning approaches do not take full consideration of either the hyperedge information or the original relationships among nodes and hyperedges. In this article, we present a simple yet effective semi-supervised node classification method named Hypergraph Convolution on Nodes-Hyperedges network, which performs filtering on both nodes and hyperedges as well as recovers the original hypergraph with the least information loss. Instead of only reducing the cross-entropy loss over the labeled samples as most previous approaches do, we additionally consider the hypergraph reconstruction loss as prior information to improve prediction accuracy. As a result, by taking both the cross-entropy loss on the labeled samples and the hypergraph reconstruction loss into consideration, we are able to achieve discriminative latent data representations for training a classifier. We perform extensive experiments on the semi-supervised node classification problem and compare the proposed method with state-of-the-art algorithms. The promising results demonstrate the effectiveness of the proposed method.

Kendall Transformation: A Robust Representation of Continuous Data for Information Theory

Kendall transformation is a conversion of an ordered feature into a vector of pairwise order relations between individual values. This way, it preserves ranking of observations and represents it in a categorical form. Such transformation allows for generalisation of methods requiring strictly categorical input, especially in the limit of small number of observations, when discretisation becomes problematic.In particular, many approaches of information theory can be directly applied to Kendall-transformed continuous data without relying on differential entropy or any additional parameters. Moreover, by filtering information to this contained in ranking, Kendall transformation leads to a better robustness at a reasonable cost of dropping sophisticated interactions which are anyhow unlikely to be correctly estimated. In bivariate analysis, Kendall transformation can be related to popular non-parametric methods, showing the soundness of the approach.The paper also demonstrates its efficiency in multivariate problems, as well as provides an example analysis of a real-world data.

Graph Entropy Based on Strong Coloring of Uniform Hypergraphs

The classical graph entropy based on the vertex coloring proposed by Mowshowitz depends on a graph. In fact, a hypergraph, as a generalization of a graph, can express complex and high-order relations such that it is often used to model complex systems. Being different from the classical graph entropy, we extend this concept to a hypergraph. Then, we define the graph entropy based on the vertex strong coloring of a hypergraph. Moreover, some tightly upper and lower bounds of such graph entropies as well as the corresponding extremal hypergraphs are obtained.

A Note on the w-Pseudo-Orders in Ordered (Semi)Hyperrings

In this work, we study the notion of w-pseudo-order on an ordered (semi)hyperring and give some explicit examples. In addition, we give some examples to compare weak pseudo-order relations with pseudo-order relations. Finally, we construct ordered (semi)hyperrings using regular relations.

Social Recommendation System Based on Hypergraph Attention Network

In recent years, due to the rise of online social platforms, social networks have more and more influence on our daily life, and social recommendation system has become one of the important research directions of recommendation system research. Because the graph structure in social networks and graph neural networks has strong representation capabilities, the application of graph neural networks in social recommendation systems has become more and more extensive, and it has also shown good results. Although graph neural networks have been successfully applied in social recommendation systems, their performance may still be limited in practical applications. The main reason is that they can only take advantage of pairs of user relations but cannot capture the higher-order relations between users. We propose a model that applies the hypergraph attention network to the social recommendation system (HASRE) to solve this problem. Specifically, we take the hypergraph’s ability to model high-order relations to capture high-order relations between users. However, because the influence of the users’ friends is different, we use the graph attention mechanism to capture the users’ attention to different friends and adaptively model selection information for the user. In order to verify the performance of the recommendation system, this paper carries out analysis experiments on three data sets related to the recommendation system. The experimental results show that HASRE outperforms the state-of-the-art method and can effectively improve the accuracy of recommendation.

Leaping across the mental canyon: Higher-order long-distance analogical retrieval

Previous experiments have shown that a comparison of two written narratives highlights theirshared relational structure, which in turn facilitates the retrieval of analogous narratives from the past (e.g., Gentner, Loewenstein, Thompson, & Forbus, 2009). However, analogical retrieval occurs across domains that appear more conceptually distant than merely different narratives, and the deepest analogies use matches in higher-order relational structure. The present study investigated whether comparison can facilitate analogical retrieval of higher-order relations across written narratives and abstract symbolic problems. Participants read stories which became retrieval targets after a delay, cued by either analogous stories or letter-strings. In Experiment 1 we replicated Gentner et al. who used narrative retrieval cues, and also found preliminary evidence for retrieval between narrative and symbolic domains. In Experiment 2 we found clear evidence that a comparison of analogous letter-string problems facilitated the retrieval of source stories with analogous higher-order relations. Experiment 3 replicated the retrieval results of Experiment 2 but with a longer delay between encoding and recall, and a greater number of distractor source stories. These experiments offer support for the schema induction account of analogical retrieval (Gentner et al., 2009) and show that the schemas abstracted from comparison of narratives can be transferred to non-semantic symbolic domains.

Generalized Ordinal Patterns and the KS-Entropy

Ordinal patterns classifying real vectors according to the order relations between their components are an interesting basic concept for determining the complexity of a measure-preserving dynamical system. In particular, as shown by C. Bandt, G. Keller and B. Pompe, the permutation entropy based on the probability distributions of such patterns is equal to Kolmogorov–Sinai entropy in simple one-dimensional systems. The general reason for this is that, roughly speaking, the system of ordinal patterns obtained for a real-valued “measuring arrangement” has high potential for separating orbits. Starting from a slightly different approach of A. Antoniouk, K. Keller and S. Maksymenko, we discuss the generalizations of ordinal patterns providing enough separation to determine the Kolmogorov–Sinai entropy. For defining these generalized ordinal patterns, the idea is to substitute the basic binary relation ≤ on the real numbers by another binary relation. Generalizing the former results of I. Stolz and K. Keller, we establish conditions that the binary relation and the dynamical system have to fulfill so that the obtained generalized ordinal patterns can be used for estimating the Kolmogorov–Sinai entropy.