Improved Path Planning for Indoor Patrol Robot Based on Deep Reinforcement Learning

To solve the problems of poor exploration ability and convergence speed of traditional deep reinforcement learning in the navigation task of the patrol robot under indoor specified routes, an improved deep reinforcement learning algorithm based on Pan/Tilt/Zoom(PTZ) image information was proposed in this paper. The obtained symmetric image information and target position information are taken as the input of the network, the speed of the robot is taken as the output of the next action, and the circular route with boundary is taken as the test. The improved reward and punishment function is designed to improve the convergence speed of the algorithm and optimize the path so that the robot can plan a safer path while avoiding obstacles first. Compared with Deep Q Network(DQN) algorithm, the convergence speed after improvement is shortened by about 40%, and the loss function is more stable.

Goal-oriented predictive representations in the human hippocampus

Recent work in cognitive and systems neuroscience has suggested that the hippocampus might support planning, imagination, and navigation by forming "cognitive maps" that capture the structure of physical spaces, tasks, and situations. Critically, navigation involves planning within a context and disambiguating similar contexts to reach a goal. We examined hippocampal activity patterns in humans during a goal-directed navigation task to examine how contextual and goal information are incorporated in the construction and execution of navigational plans. Results demonstrate that, during planning, the hippocampus carries a context-specific representation of a future goal. Importantly, this effect could not be explained by stimulus or spatial information alone. During navigation, we observed reinstatement of activity patterns in the hippocampus ahead of participants' required actions, which was strongest for behaviorally relevant points in the sequence. These results suggest that, rather than simply representing overlapping associations, hippocampal activity patterns are powerfully shaped by context and goals.

Identifying Interaction Patterns of Tangible Co-Adaptations in Human-Robot Team Behaviors

As robots become more ubiquitous, they will increasingly need to behave as our team partners and smoothly adapt to the (adaptive) human team behaviors to establish successful patterns of collaboration over time. A substantial amount of adaptations present themselves through subtle and unconscious interactions, which are difficult to observe. Our research aims to bring about awareness of co-adaptation that enables team learning. This paper presents an experimental paradigm that uses a physical human-robot collaborative task environment to explore emergent human-robot co-adaptions and derive the interaction patterns (i.e., the targeted awareness of co-adaptation). The paradigm provides a tangible human-robot interaction (i.e., a leash) that facilitates the expression of unconscious adaptations, such as “leading” (e.g., pulling the leash) and “following” (e.g., letting go of the leash) in a search-and-navigation task. The task was executed by 18 participants, after which we systematically annotated videos of their behavior. We discovered that their interactions could be described by four types of adaptive interactions: stable situations, sudden adaptations, gradual adaptations and active negotiations. From these types of interactions we have created a language of interaction patterns that can be used to describe tacit co-adaptation in human-robot collaborative contexts. This language can be used to enable communication between collaborating humans and robots in future studies, to let them share what they learned and support them in becoming aware of their implicit adaptations.

Acquisition of a spatial navigation task in the padding pool induces an increase of GABA level in the hippocampus of Swiss mice

The balance between excitatory and inhibitory glutamatergic and GABAergic systems, respectively, is crucial for the maintenance of complex cognitive functions such as learning. Using Swiss mice as experimental model, the aims of the present study were to evaluate cognitive performance in a shallow water maze (SWM) and the effects of training in this spatial navigation task on hippocampal GABA and glutamate levels. In addition, correlations between neurochemical and behavioural data, and between glutamate and GABA levels were assessed. Forty-six three-month-old mice were divided into three groups: Learning, n=18: animals submitted to the SWM task; Active, n=14: animals exposed to the SWM, without the demand of performing a cognitive task and Control, n=14: the animals were kept in the vivarium without contact with the SWM. There was significant training effect indicating that the Learning group animals have learned the task. Regarding neurochemical data, the findings of the present work show for the first time that the task learning process in SWM has a significant effect on GABA levels in the hippocampus. The relationship between the two neurotransmitters, observed in the control animals, was adjusted by a significant increase in hippocampal GABA levels caused by the spatial training performed by the animals from the Learning group. However, the relationship observed in control condition is disrupted by a subsequent exposure to the maze in the absence of a spatial cognitive demand, as was the case of the Active group. These data open new perspectives to explore the involvement of the inhibitory and excitatory systems in the molecular mechanisms associated with different types and steps of learning processes.

Temporal Sensitivity and Latency During Teleoperation: Using Track Clearance to Understand Errors in Future Projection

Objective This study investigates the role of individual differences in time perception on task performance during teleoperation with latency. Background Long distance teleoperation induces latency, causing performance issues for the operator. Previous research demonstrated that individual differences in time perception predicted performance on a similar task, having participants navigate a radio controlled (RC) car around a track. This work extends the relationship into routes of varying course width to test whether differences in time perception predict movement over-/underestimation. Method Participants completed a time estimation task and a route navigation task while experiencing latency. In the time estimation task, participants estimated the duration of multiple visual stimuli (2 s or less). In the route navigation task, participants moved a virtual cube across a route. Each trial varied in the amount of latency and the amount of horizontal clearance in the track (4–10 m for a 1.2-m-long/wide cube). Results The results showed fairly consistent latency by time estimation and latency by clearance interaction effects on a wide set of trial-level variables, such as completion time, and action-level performance variables, such as time spent moving per move event. However, the results were not consistently in the predicted direction. Conclusion Results suggest that clearance and timing affect performance across latency, at both the overall level (i.e., trial completion time) and at the action level (time spent moving). An open question remains as to how these contextual factors affect movement strategy selection.

Effects of Spatial Memory Processing on Hippocampal Ripples

Human High-Frequency-Oscillations (HFO) in the ripple band are oscillatory brain activity in the frequency range between 80 and 250 Hz. HFOs may comprise different subgroups that either play a role in physiologic or pathologic brain functions. An exact differentiation between physiologic and pathologic HFOs would help elucidate their relevance for cognitive and epileptogenic brain mechanisms, but the criteria for differentiating between physiologic and pathologic HFOs remain controversial. In particular, the separation of pathologic HFOs from physiologic HFOs could improve the identification of epileptogenic brain regions during the pre-surgical evaluation of epilepsy patients. In this study, we performed intracranial electroencephalography recordings from the hippocampus of epilepsy patients before, during, and after the patients completed a spatial navigation task. We isolated hippocampal ripples from the recordings and categorized the ripples into the putative pathologic group iesRipples, when they coincided with interictal spikes, and the putative physiologic group isolRipples, when they did not coincide with interictal spikes. We found that the occurrence of isolRipples significantly decreased during the task as compared to periods before and after the task. The rate of iesRipples was not modulated by the task. In patients who completed the spatial navigation task on two consecutive days, we furthermore examined the occurrence of ripples in the intervening night. We found that the rate of ripples that coincided with sleep spindles and were therefore putatively physiologic correlated with the performance improvement on the spatial navigation task, whereas the rate of all ripples did not show this relationship. Together, our results suggest that the differentiation of HFOs into putative physiologic and pathologic subgroups may help identify their role for spatial memory and memory consolidation processes. Conversely, excluding putative physiologic HFOs from putative pathologic HFOs may improve the HFO-based identification of epileptogenic brain regions in future studies.

Age-related impairment of navigation and strategy in virtual star maze

Background Although it is well known that aging impairs navigation performance, the underlying mechanisms remain largely unknown. Egocentric strategy requires navigators to remember a series of body-turns without relying on the relationship between environmental cues. Previous study suggested that the egocentric strategy, compared with non-egocentric strategy, was relatively unimpaired during aging. In this study, we aimed to examine strategy use during virtual navigation task and the underlying cognitive supporting mechanisms in older adults. Methods Thirty young adults and thirty-one older adults were recruited from the local community. This study adapted star maze paradigm using non-immersive virtual environment. Participants moved freely in a star maze with adequate landmarks, and were requested to find a fixed destination. After 9 learning trials, participants were probed in the same virtual star maze but with no salient landmarks. Participants were classified as egocentric or non-egocentric strategy group according to their response in the probe trial. Results The results revealed that older adults adopting egocentric strategy completed the navigation task as accurate as young adults, whereas older adults using non-egocentric strategy completed the navigation task with more detours and lower accuracy. The relatively well-maintained egocentric strategy in older adults was related to better visuo-spatial ability. Conclusions Visuo-spatial ability might play an important role in navigation accuracy and navigation strategy of older adults. This study demonstrated the potential value of the virtual star maze in evaluating navigation strategy and visuo-spatial ability in older adults.

Age-related impairment of navigation and strategy in virtual reality star maze

Background: Although it is well known that aging impairs navigation performance, the underlying mechanisms remain largely unknown. Egocentric strategy requires navigators to remember a series of body-turns without relying on the relationship between environmental cues. Previous study suggested that the egocentric strategy, compared with non-egocentric strategy, was relatively unimpaired during aging. In this study, we aimed to examine strategy use during virtual navigation task and the underlying cognitive supporting mechanisms in older adults.Methods: Thirty young adults and thirty-one older adults were recruited from the local community. This study adapted star maze paradigm using non-immersive virtual environment. Participants moved freely in a star maze with adequate landmarks, and were requested to find a fixed destination. After 9 learning trials, participants were probed in the same virtual star maze but with no salient landmarks. Participants were classified as egocentric or non-egocentric strategy group according to their response in the probe trial. Results: The results revealed that older adults adopting egocentric strategy completed the navigation task as accurate as young adults, whereas older adults using non-egocentric strategy completed the navigation task with more detours and lower accuracy. The relatively well-maintained egocentric strategy in older adults was related to better visuo-spatial ability.Conclusions: Visuo-spatial ability might play an important role in navigation accuracy and navigation strategy of older adults. This study demonstrated the potential value of the virtual star maze in evaluating navigation strategy and visuo-spatial ability in older adults.