A Neural Approach for Modeling Continuous Time Sequences with Intermittent Observations

A large fraction of data generated via human activities such as online purchases, health records, spatial mobility etc. can be represented as continuous-time event sequences (CTES) i.e. sequences of discrete events over a continuous time. Learning neural models over CTES is a non-trivial task as it involves modeling the ever-increasing event timestamps, inter-event time gaps, event types, and the influences between different events within and across different sequences. Moreover, existing sequence modeling techniques consider a complete observation scenario i.e. the event sequence being modeled is completely observed with no missing events – an ideal setting that is rarely applicable in real-world applications. In this paper, we highlight our approach[8] for modeling CTES with intermittent observations. Buoyed by the recent success of neural marked temporal point processes (MTPP) for modeling the generative distribution of CTES, we provide a novel unsupervised model and inference method for learning MTPP in presence of event sequences with missing events. Specifically, we first model the generative processes of observed events and missing events using two MTPP, where the missing events are represented as latent random variables. Then, we devise an unsupervised training method that jointly learns both the MTPP using variational inference. Experiments across real-world datasets show that our modeling framework outperforms state-of-the-art techniques for future event prediction and imputation. This work appeared in AISTATS 2021.

Comparison and aggregation of event sequences across ten cohorts to describe the consensus biomarker evolution in Alzheimer's disease

Background: Previous models of Alzheimer's disease (AD) progression were primarily hypothetical or based on data originating from single cohort studies. However, cohort datasets are subject to specific inclusion and exclusion criteria that influence the signals observed in their collected data. Furthermore, each study measures only a subset of AD-relevant variables. To gain a comprehensive understanding of AD progression, the heterogeneity and robustness of estimated progression patterns must be understood, and complementary information contained in cohort datasets be leveraged. Methods: We compared ten event-based models that we fit to ten independent AD cohort datasets. Additionally, we designed and applied a novel rank aggregation algorithm that combines partially overlapping, individual event sequences into a meta-sequence containing the complementary information from each cohort. Results: We observed overall consistency across the ten event-based model sequences, despite variance in the positioning of mainly imaging variables. The changes described in the aggregated meta-sequence are broadly consistent with current understanding of AD progression, starting with cerebrospinal fluid amyloid beta, followed by memory impairment, tauopathy, FDG-PET, and ultimately brain deterioration and impairment of visual memory. Conclusion: Overall, the event-based models demonstrated similar and robust disease cascades across independent AD cohorts. Aggregation of data-driven results can combine complementary strengths and information of patient-level datasets. Accordingly, the derived meta-sequence draws a more complete picture of AD pathology compared to models relying on single cohorts.

Bio-inspired central pattern generator for development of Cartesian motor skills for a Quadruped Robot

Central Pattern Generator (CPG) plays a significant role in the generation of diverse and stable gaits patterns for animals as well as controlling their locomotion. The main contributions of this paper are the ability to develop the Cartesian motor skills and coordinating legs of the quadruped robot for gait adaption and its nominal characteristics with CPG approach. Primary, a predefined relationship between an excitation signal and essential parameters of the CPG design is programmed. Next, the coordinated oscillator's rhythmic patterns by CPG and accordingly output gait diagrams for each foot of the robot are attained. Then, these desirable features such as predictive modulation and programming the gait event sequences including leg-lifting sequences and step length, duration of the time of each footstep within a gait, coordination of swing and stance phases of all legs are calculated in terms of different spatio_temporal vectors. Furthermore, a novel Cartesian footstep basis function is designed based on the robot characteristics and consequently, the associated spatio-temporal vectors can be inserted to it, which caused to spanning the space of possible gait timing in Cartesian space. Next, Cartesian footstep planner can be computed the swing foot trajectories in workspace along movement axes and then according to these footholds and feet placement, ZMP (Zero Moment Point) reference trajectory will be calculated and obtained. Therefore, COG (Center of Gravity) trajectory can be computed by designing a preview controller on the basis of the desired ZMP trajectory. Finally, to demonstrate the effectiveness of the proposed algorithm, it is implemented on a quadruped robot on both simulation or experimental implementations and the results are compared and discussed with other references.

Novel objects with causal schemas elicit selective responses in tool- and hand-selective lateral occipito-temporal cortex

Tool-selective lateral occipito-temporal cortex (LOTC) responds preferentially to images of tools (hammers, brushes) relative to non-tool objects (clocks, shoes). What drives these responses? Tools have elongated shapes and are more likely to have motor associations, but another essential property is that they exert causal effects on the environment. We tested whether LOTC would respond to novel objects associated with a tool-canonical schema in which their actions cause other events. To do so, we taught male and female human participants about novel objects embedded in animated event sequences, which varied in the temporal order of their events. Causer objects moved prior to the appearance of an environmental event (e.g., stars) while Reactor objects moved after an identical event; objects were matched on shape and motor association. During fMRI, participants viewed still images of these novel objects. We localized tool-selective LOTC and non-tool-selective parahippocampal cortex (PHC) by contrasting neural responses to images of familiar tools and non-tools. We found that LOTC responded more to Causers than Reactors; this effect was absent and weaker in right PHC. We also localized responses to images of hands, which elicit overlapping responses with tools. Across inferior temporal cortex, voxels’ tool and hand selectivity positively predicted a preferential response to Causers, and non-tool selectivity negatively so. We conclude that a causal schema typical of tools is sufficient to drive LOTC, and more generally, that preferential responses to domains across the temporal lobe may reflect the relational event structures typical of those domains.

Supervised sequential pattern mining of event sequences in sport to identify important patterns of play: An application to rugby union

Given a set of sequences comprised of time-ordered events, sequential pattern mining is useful to identify frequent subsequences from different sequences or within the same sequence. However, in sport, these techniques cannot determine the importance of particular patterns of play to good or bad outcomes, which is often of greater interest to coaches and performance analysts. In this study, we apply a recently proposed supervised sequential pattern mining algorithm called safe pattern pruning (SPP) to 490 labelled event sequences representing passages of play from one rugby team’s matches in the 2018 Japan Top League season. We obtain patterns that are the most discriminative between scoring and non-scoring outcomes from both the team’s and opposition teams’ perspectives using SPP, and compare these with the most frequent patterns obtained with well-known unsupervised sequential pattern mining algorithms when applied to subsets of the original dataset, split on the label. From our obtained results, line breaks, successful line-outs, regained kicks in play, repeated phase-breakdown play, and failed exit plays by the opposition team were found to be the patterns that discriminated most between the team scoring and not scoring. Opposition team line breaks, errors made by the team, opposition team line-outs, and repeated phase-breakdown play by the opposition team were found to be the patterns that discriminated most between the opposition team scoring and not scoring. It was also found that, probably because of the supervised nature and pruning/safe-screening mechanisms of SPP, compared to the patterns obtained by the unsupervised methods, those obtained by SPP were more sophisticated in terms of containing a greater variety of events, and when interpreted, the SPP-obtained patterns would also be more useful for coaches and performance analysts.

A Unifying Framework for Analysis of Spatial-Temporal Event Sequence Similarity and Its Applications

Measures of similarity or differences between data objects are applied frequently in geography, biology, computer science, linguistics, logic, business analytics, and statistics, among other fields. This work focuses on event sequence similarity among event sequences extracted from time series observed at spatially deployed monitoring locations with the aim of enhancing the understanding of process similarity over time and geospatial locations. We present a framework for a novel matrix-based spatiotemporal event sequence representation that unifies punctual and interval-based representation of events. This unified representation of spatiotemporal event sequences (STES) supports different event data types and provides support for data mining and sequence classification and clustering. The similarity measure is based on the Jaccard index with temporal order constraints and accommodates different event data types. The approach is demonstrated through simulated data examples and the performance of the similarity measures is evaluated with a k-nearest neighbor algorithm (k-NN) classification test on synthetic datasets. As a case study, we demonstrate the use of these similarity measures in a spatiotemporal analysis of event sequences extracted from space time series of a water quality monitoring system.

Emotional arousal ripples across time to bind subsequent episodes in memory

Time unfolds continuously, yet our memories are stored as discrete episodes. Prior work shows that fluctuations between stability and change in an ongoing neutral context facilitates this formation of distinct and memorable events. However, less is known about how shifting emotional states influence these memory processes, despite ample evidence that emotion has a robust influence on non-temporal aspects of episodic memory. Here, we examined if emotional stimuli influence temporal memory for recent event sequences. Participants encoded lists of neutral object images while listening to pure auditory tones. At regular intervals within each list, participants heard emotional positive, negative, or neutral sounds, which served as ‘emotional event boundaries’ that divided each sequence into discrete auditory events. Temporal order memory was tested for neutral item pairs that either spanned an emotional sound (‘boundary-spanning’) or encountered within the same auditory event (‘same-context’). We found that highly arousing boundaries had opposite effects on binding ongoing versus subsequent sequential representations in memory. Specifically, highly arousing emotional sounds tended to lead to worse temporal order memory for boundary-spanning item pairs. By contrast, they led to better temporal order memory for same-context item pairs in the next event. Both of these arousal effects were specific to negative sounds. The carryover effect of negative arousal was also strongest for item pairs encountered closest to the boundary and diminished as the event unfolded. These findings suggest that temporally dynamic emotional states support the temporal integration of mnemonic events, which may contribute to the hyper-episodic nature of negative emotional memories.

Paleoearthquakes Constrained by Stratigraphic Sequences of Different Drainages Since Late Pleistocene: A Case Study Along the Gulang Fault, NE Tibetan Plateau

Paleoearthquake data obtained from fault trenching are essential for rebuilding the rupture history and understanding the rupture behavior of active faults. However, due to the lack of attention to stratigraphic sequences, the usual multiple trench constraining method may result in uncertainties of paleoearthquake sequences. In this study, we proposed an improved constraining method to generate stratigraphic sequences from multiple trenches of different drainages to obtain a paleoearthquake sequence of the Gulang fault. Single-trench stratigraphic sequences were built up by nineteen trenches excavated along the fault. Based on stratigraphic characteristics, we found the strata sedimented around the fault were derived from five drainages. The single-trench sequences were divided into five drainages to establish the composite sequence of multiple trenches through the correlation of stratigraphic units. Meanwhile, we used high-quality event indicators to pick out very likely earthquakes. Coupled with the dating samples, the events were used to determine the earthquake horizons in the composite sequence and to constrain the numbers and ages of events in each drainage. After combining the event sequences, six paleoearthquakes were determined along the Gulang fault since the late Pleistocene. Their occurrence timings are 13,700–10,400, 10,400–10,200, 8,560–7,295, 5,825–4,810, 4,285–3,200, and 2,615–2,240 a B.P. And their different rupture scenarios indicate that the fault might be composed of two rupture segments.