Control of Magnetic Manipulator Using Reinforcement Learning Based on Incrementally Adapted Local Linear Models

Reinforcement learning (RL) agents can learn to control a nonlinear system without using a model of the system. However, having a model brings benefits, mainly in terms of a reduced number of unsuccessful trials before achieving acceptable control performance. Several modelling approaches have been used in the RL domain, such as neural networks, local linear regression, or Gaussian processes. In this article, we focus on techniques that have not been used much so far: symbolic regression (SR), based on genetic programming and local modelling. Using measured data, symbolic regression yields a nonlinear, continuous-time analytic model. We benchmark two state-of-the-art methods, SNGP (single-node genetic programming) and MGGP (multigene genetic programming), against a standard incremental local regression method called RFWR (receptive field weighted regression). We have introduced modifications to the RFWR algorithm to better suit the low-dimensional continuous-time systems we are mostly dealing with. The benchmark is a nonlinear, dynamic magnetic manipulation system. The results show that using the RL framework and a suitable approximation method, it is possible to design a stable controller of such a complex system without the necessity of any haphazard learning. While all of the approximation methods were successful, MGGP achieved the best results at the cost of higher computational complexity. Index Terms–AI-based methods, local linear regression, nonlinear systems, magnetic manipulation, model learning for control, optimal control, reinforcement learning, symbolic regression.

Integrating magnetic capabilities to intracellular chips for cell trapping

Current microtechnologies have shown plenty of room inside a living cell for silicon chips. Microchips as barcodes, biochemical sensors, mechanical sensors and even electrical devices have been internalized into living cells without interfering their cell viability. However, these technologies lack from the ability to trap and preconcentrate cells in a specific region, which are prerequisites for cell separation, purification and posterior studies with enhanced sensitivity. Magnetic manipulation of microobjects, which allows a non-contacting method, has become an attractive and promising technique at small scales. Here, we show intracellular Ni-based chips with magnetic capabilities to allow cell enrichment. As a proof of concept of the potential to integrate multiple functionalities on a single device of this technique, we combine coding and magnetic manipulation capabilities in a single device. Devices were found to be internalized by HeLa cells without interfering in their viability. We demonstrated the tagging of a subpopulation of cells and their subsequent magnetic trapping with internalized barcodes subjected to a force up to 2.57 pN (for magnet-cells distance of 4.9 mm). The work opens the venue for future intracellular chips that integrate multiple functionalities with the magnetic manipulation of cells.