Control of Magnetic Manipulator Using Reinforcement Learning Based on Incrementally Adapted Local Linear Models

Reinforcement learning (RL) agents can learn to control a nonlinear system without using a model of the system. However, having a model brings benefits, mainly in terms of a reduced number of unsuccessful trials before achieving acceptable control performance. Several modelling approaches have been used in the RL domain, such as neural networks, local linear regression, or Gaussian processes. In this article, we focus on techniques that have not been used much so far: symbolic regression (SR), based on genetic programming and local modelling. Using measured data, symbolic regression yields a nonlinear, continuous-time analytic model. We benchmark two state-of-the-art methods, SNGP (single-node genetic programming) and MGGP (multigene genetic programming), against a standard incremental local regression method called RFWR (receptive field weighted regression). We have introduced modifications to the RFWR algorithm to better suit the low-dimensional continuous-time systems we are mostly dealing with. The benchmark is a nonlinear, dynamic magnetic manipulation system. The results show that using the RL framework and a suitable approximation method, it is possible to design a stable controller of such a complex system without the necessity of any haphazard learning. While all of the approximation methods were successful, MGGP achieved the best results at the cost of higher computational complexity. Index Terms–AI-based methods, local linear regression, nonlinear systems, magnetic manipulation, model learning for control, optimal control, reinforcement learning, symbolic regression.

A state-space approach for longitudinal outcomes: An application to neuropsychological outcomes

Longitudinal assessments are crucial in evaluating the disease state and trajectory in patients with neurodegenerative diseases. Neuropsychological outcomes measured over time often have a non-linear trajectory with autocorrelated residuals and a skewed distribution. We propose the adjusted local linear trend model, an extended state-space model in lieu of the commonly used linear mixed-effects model in modeling longitudinal neuropsychological outcomes. Our contributed model has the capability to utilize information from the stochasticity of the data while accounting for subject-specific trajectories with the inclusion of covariates and unequally spaced time intervals. The first step of model fitting involves a likelihood maximization step to estimate the unknown variances in the model before parsing these values into the Kalman filter and Kalman smoother recursive algorithms. Results from simulation studies showed that the adjusted local linear trend model is able to attain lower bias, lower standard errors, and high power, particularly in short longitudinal studies with equally spaced time intervals, as compared to the linear mixed-effects model. The adjusted local linear trend model also outperforms the linear mixed-effects model when data is missing completely at random, missing at random, and, in certain cases, even in data with missing not at random.

Experimental investigation of the dynamics of a slider-crank mechanism with local linear force input

Conventional implementation of slider-crank mechanisms result in high loads transmitted through the mechanical structure, inhibiting the design of compact and oil-free machines. Therefore, this research proposes to step away from the conventional, i.e. rotative, actuation and to investigate local linear actuation on the slider-component directly, while maintaining the kinematic link of the slider-crank configuration. In this work the local linear actuating principle is evaluated experimentally where the goal is to obtain a continuous movement of the slider mechanism where Top Dead Centre & Bottom Dead Centre are reached and to minimise the loads transmitted through the mechanical structure. The non-isochronous transient behaviour of a slider-crank mechanism loaded with a spring-damper element is detailed as well as the optimal working conditions at steady state to achieve a reduced loading of the kinematic structure. By matching the operating frequency and resonance frequency of the system, a reduction of the loads transmitted through the system by 63% of the nominal spring load can be achieved. Further experimental (and multibody mechanical) investigation on the influence of flywheel exposes a clear trade-off between the sensitivity of the system and the transmission of the actuation force through the kinematic link.

Local Linear Scale Factors in Map Projections of an Ellipsoid

The main problem in cartography is that it is not possible to map/project/transform a spherical or ellipsoidal surface into a plane without distortions. The distortions of areas, angles, and/or distances are immanent to all maps. It is known that scale changes from point to point, and at certain points, the scale usually depends on the direction. The local linear scale factor c is one of the most important indicators of distortion distribution in the theory of map projections. It is not possible to find out the values of the local linear scale factor c in directions of coordinate axes x and y immediately from the definition of c. To solve this problem, in this paper, we derive new formulae for the calculation of c for a rotational ellipsoid. In addition, we derive the formula for computing c in any direction defined by dx and dy. We also considered the position and magnitude of the extreme values of c and derived new formulae for a rotational ellipsoid.