Embedding Hierarchical Structures for Venue Category Representation

Venue categories used in location-based social networks often exhibit a hierarchical structure, together with the category sequences derived from users’ check-ins. The two data modalities provide a wealth of information for us to capture the semantic relationships between those categories. To understand the venue semantics, existing methods usually embed venue categories into low-dimensional spaces by modeling the linear context (i.e., the positional neighbors of the given category) in check-in sequences. However, the hierarchical structure of venue categories, which inherently encodes the relationships between categories, is largely untapped. In this article, we propose a venue C ategory E mbedding M odel named Hier-CEM , which generates a latent representation for each venue category by embedding the Hier archical structure of categories and utilizing multiple types of context. Specifically, we investigate two kinds of hierarchical context based on any given venue category hierarchy and show how to model them together with the linear context collaboratively. We apply Hier-CEM to three tasks on two real check-in datasets collected from Foursquare. Experimental results show that Hier-CEM is better at capturing both semantic and sequential information inherent in venues than state-of-the-art embedding methods.

Sequential-Knowledge-Aware Next POI Recommendation: A Meta-Learning Approach

Accurately recommending the next point of interest (POI) has become a fundamental problem with the rapid growth of location-based social networks. However, sparse, imbalanced check-in data and diverse user check-in patterns pose severe challenges for POI recommendation tasks. Knowledge-aware models are known to be primary in leveraging these problems. However, as most knowledge graphs are constructed statically, sequential information is yet integrated. In this work, we propose a meta-learned sequential-knowledge-aware recommender (Meta-SKR), which utilizes sequential, spatio-temporal, and social knowledge to recommend the next POI for a location-based social network user. The framework mainly contains four modules. First, in the graph construction module, a novel type of knowledge graph—the sequential knowledge graph, which is sensitive to the check-in order of POIs—is built to model users’ check-in patterns. To deal with the problem of data sparsity, a meta-learning module based on latent embedding optimization is then introduced to generate user-conditioned parameters of the subsequent sequential-knowledge-aware embedding module, where representation vectors of entities (nodes) and relations (edges) are learned. In this embedding module, gated recurrent units are adapted to distill intra- and inter-sequential knowledge graph information. We also design a novel knowledge-aware attention mechanism to capture information surrounding a given node. Finally, POI recommendation is provided by inferring potential links of knowledge graphs in the prediction module. Evaluations on three real-world check-in datasets show that Meta-SKR can achieve high recommendation accuracy even with sparse data.

Experimental Evaluation of Malware Family Classification Methods from Sequential Information of TLS-Encrypted Traffic

In parallel with the rapid adoption of transport layer security (TLS), malware has utilized the encrypted communication channel provided by TLS to hinder detection from network traffic. To this end, recent research efforts are directed toward malware detection and malware family classification for TLS-encrypted traffic. However, amongst their feature sets, the proposals to utilize the sequential information of each TLS session has not been properly evaluated, especially in the context of malware family classification. In this context, we propose a systematic framework to evaluate the state-of-the-art malware family classification methods for TLS-encrypted traffic in a controlled environment and discuss the advantages and limitations of the methods comprehensively. In particular, our experimental results for the 10 representations and classifier combinations show that the graph-based representation for the sequential information achieves better performance regardless of the evaluated classification algorithms. With our framework and findings, researchers can design better machine learning based classifiers.

A Sequential Graph Neural Network for Short Text Classification

Short text classification is an important problem of natural language processing (NLP), and graph neural networks (GNNs) have been successfully used to solve different NLP problems. However, few studies employ GNN for short text classification, and most of the existing graph-based models ignore sequential information (e.g., word orders) in each document. In this work, we propose an improved sequence-based feature propagation scheme, which fully uses word representation and document-level word interaction and overcomes the limitations of textual features in short texts. On this basis, we utilize this propagation scheme to construct a lightweight model, sequential GNN (SGNN), and its extended model, ESGNN. Specifically, we build individual graphs for each document in the short text corpus based on word co-occurrence and use a bidirectional long short-term memory network (Bi-LSTM) to extract the sequential features of each document; therefore, word nodes in the document graph retain contextual information. Furthermore, two different simplified graph convolutional networks (GCNs) are used to learn word representations based on their local structures. Finally, word nodes combined with sequential information and local information are incorporated as the document representation. Extensive experiments on seven benchmark datasets demonstrate the effectiveness of our method.

Sequence learning attenuates cortical responses in both frontal and perceptual cortices in early infancy

Prior work using a variety of imaging modalities has found that the frontal lobe is involved in higher-order sequential and statistical learning in young infants. Separate lines of work have found evidence of modulation of posterior sensory cortices during and after learning tasks. How do these processes relate together? Here, we build from a well-regarded EEG task that found evidence that the frontal lobe of young infants tracked higher-order sequential information (Basirat et al., 2014) and ask whether posterior perceptual cortices respond differentially to predictable vs. unpredictable sequences as well. First, replicating and extending past work, we found evidence of frontal lobe involvement in this task. Second, consistent with our hypotheses, we found that there is a corresponding attenuation of neural responses in the posterior perceptual cortices (temporal and occipital) to predictable compared to unpredictable audiovisual sequences. This study provides convergent evidence that the frontal lobe is crucial for higher-level learning in young infants but that it likely works as part of a large, distributed network of regions to modulate infant neural responses as a result of learning. Overall, this work challenges the view that the infant brain is not dynamic and disconnected, lacking in long-range neural connections. Instead, this paper reveals patterns of a highly dynamic and interconnected infant brain that change rapidly as a result of new, learnable experiences.

Session centered Recommendation Utilizing Future Contexts in Social Media

Session centered recommender systems has emerged as an interesting and challenging topic amid researchers during the past few years. In order to make a prediction in the sequential data, prevailing approaches utilize either left to right design autoregressive or data augmentation methods. As these approaches are used to utilize the sequential information pertaining to user conduct, the information about the future context of an objective interaction is totally ignored while making prediction. As a matter of fact, we claim that during the course of training, the future data after the objective interaction are present and this supplies indispensable signal on preferences of users and if utilized can increase the quality of recommendation. It is a subtle task to incorporate future contexts into the process of training, as the rules of machine learning are not followed and can result in loss of data. Therefore, in order to solve this problem, we suggest a novel encoder decoder prototype termed as space filling centered Recommender (SRec), which is used to train the encoder and decoder utilizing space filling approach. Particularly, an incomplete sequence is taken into consideration by the encoder as input (few items are absent) and then decoder is used to predict these items which are absent initially based on the encoded interpretation. The general SRec prototype is instantiated by us employing convolutional neural network (CNN) by giving emphasis on both e ciency and accuracy. The empirical studies and investigation on two real world datasets are conducted by us including short, medium and long sequences, which exhibits that SRec performs better than traditional sequential recommendation approaches.

Soft Periodic Convolutional Recurrent Network for Spatiotemporal Climate Forecast

Many machine-learning applications and methods are emerging to solve problems associated with spatiotemporal climate forecasting; however, a prediction algorithm that considers only short-range sequential information may not be adequate to deal with periodic patterns such as seasonality. In this paper, we adopt a Periodic Convolutional Recurrent Network (Periodic-CRN) model to employ the periodicity component in our proposals of the periodic representation dictionary (PRD). Phase shifts and non-stationarity of periodicity are the key components in the model to support. Specifically, we propose a Soft Periodic-CRN (SP-CRN) with three proposals of utilizing periodicity components: nearby-time (PRD-1), periodic-depth (PRD-2), and periodic-depth differencing (PRD-3) representation to improve climate forecasting accuracy. We experimented on geopotential height at 300 hPa (ZH300) and sea surface temperature (SST) datasets of ERA-Interim. The results showed the superiority of PRD-1 plus or minus one month of a prior cycle to capture the phase shift. In addition, PRD-3 considered only the depth of one differencing periodic cycle (i.e., the previous year) can significantly improve the prediction accuracy of ZH300 and SST. The mixed method of PRD-1, and PRD-3 (SP-CRN-1+3) showed a competitive or slight improvement over their base models. By adding the metadata component to indicate the month with one-hot encoding to SP-CRN-1+3, the prediction result was a drastic improvement. The results showed that the proposed method could learn four years of periodicity from the data, which may relate to the El Niño–Southern Oscillation (ENSO) cycle.

Disorganization of Oscillatory Activity in Animal Models of Schizophrenia

Schizophrenia is a chronic, debilitating disorder with diverse symptomatology, including disorganized cognition and behavior. Despite considerable research effort, we have only a limited understanding of the underlying brain dysfunction. In this article, we review the potential role of oscillatory circuits in the disorder with a particular focus on the hippocampus, a region that encodes sequential information across time and space, as well as the frontal cortex. Several mechanistic explanations of schizophrenia propose that a loss of oscillatory synchrony between and within these brain regions may underlie some of the symptoms of the disorder. We describe how these oscillations are affected in several animal models of schizophrenia, including models of genetic risk, maternal immune activation (MIA) models, and models of NMDA receptor hypofunction. We then critically discuss the evidence for disorganized oscillatory activity in these models, with a focus on gamma, sharp wave ripple, and theta activity, including the role of cross-frequency coupling as a synchronizing mechanism. Finally, we focus on phase precession, which is an oscillatory phenomenon whereby individual hippocampal place cells systematically advance their firing phase against the background theta oscillation. Phase precession is important because it allows sequential experience to be compressed into a single 120 ms theta cycle (known as a ‘theta sequence’). This time window is appropriate for the induction of synaptic plasticity. We describe how disruption of phase precession could disorganize sequential processing, and thereby disrupt the ordered storage of information. A similar dysfunction in schizophrenia may contribute to cognitive symptoms, including deficits in episodic memory, working memory, and future planning.