Sequencing of multi-robot behaviors using reinforcement learning

Given a collection of parameterized multi-robot controllers associated with individual behaviors designed for particular tasks, this paper considers the problem of how to sequence and instantiate the behaviors for the purpose of completing a more complex, overarching mission. In addition, uncertainties about the environment or even the mission specifications may require the robots to learn, in a cooperative manner, how best to sequence the behaviors. In this paper, we approach this problem by using reinforcement learning to approximate the solution to the computationally intractable sequencing problem, combined with an online gradient descent approach to selecting the individual behavior parameters, while the transitions among behaviors are triggered automatically when the behaviors have reached a desired performance level relative to a task performance cost. To illustrate the effectiveness of the proposed method, it is implemented on a team of differential-drive robots for solving two different missions, namely, convoy protection and object manipulation.

Brain–computer interfaces for human gait restoration

In this review article, we present more than a decade of our work on the development of brain–computer interface (BCI) systems for the restoration of walking following neurological injuries such as spinal cord injury (SCI) or stroke. Most of this work has been in the domain of non-invasive electroencephalogram-based BCIs, including interfacing our system with a virtual reality environment and physical prostheses. Real-time online tests are presented to demonstrate the ability of able-bodied subjects as well as those with SCI to purposefully operate our BCI system. Extensions of this work are also presented and include the development of a portable low-cost BCI suitable for at-home use, our ongoing efforts to develop a fully implantable BCI for the restoration of walking and leg sensation after SCI, and our novel BCI-based therapy for stroke rehabilitation.

Discrete-time inverse linear quadratic optimal control over finite time-horizon under noisy output measurements

In this paper, the problem of inverse quadratic optimal control over finite time-horizon for discrete-time linear systems is considered. Our goal is to recover the corresponding quadratic objective function using noisy observations. First, the identifiability of the model structure for the inverse optimal control problem is analyzed under relative degree assumption and we show the model structure is strictly globally identifiable. Next, we study the inverse optimal control problem whose initial state distribution and the observation noise distribution are unknown, yet the exact observations on the initial states are available. We formulate the problem as a risk minimization problem and approximate the problem using empirical average. It is further shown that the solution to the approximated problem is statistically consistent under the assumption of relative degrees. We then study the case where the exact observations on the initial states are not available, yet the observation noises are known to be white Gaussian distributed and the distribution of the initial state is also Gaussian (with unknown mean and covariance). EM-algorithm is used to estimate the parameters in the objective function. The effectiveness of our results are demonstrated by numerical examples.

The dorsolateral pre-frontal cortex bi-polar error-related potential in a locked-in patient implanted with a daily use brain–computer interface

While brain computer interfaces (BCIs) offer the potential of allowing those suffering from loss of muscle control to once again fully engage with their environment by bypassing the affected motor system and decoding user intentions directly from brain activity, they are prone to errors. One possible avenue for BCI performance improvement is to detect when the BCI user perceives the BCI to have made an unintended action and thus take corrective actions. Error-related potentials (ErrPs) are neural correlates of error awareness and as such can provide an indication of when a BCI system is not performing according to the user’s intentions. Here, we investigate the brain signals of an implanted BCI user suffering from locked-in syndrome (LIS) due to late-stage ALS that prevents her from being able to speak or move but not from using her BCI at home on a daily basis to communicate, for the presence of error-related signals. We first establish the presence of an ErrP originating from the dorsolateral pre-frontal cortex (dLPFC) in response to errors made during a discrete feedback task that mimics the click-based spelling software she uses to communicate. Then, we show that this ErrP can also be elicited by cursor movement errors in a continuous BCI cursor control task. This work represents a first step toward detecting ErrPs during the daily home use of a communications BCI.