scholarly journals Lightweight Reinforcement Algorithms for autonomous, scalable intra-cortical Brain Machine Interfaces

2020 ◽  
Author(s):  
Shoeb Shaikh ◽  
Rosa So ◽  
Tafadzwa Sibindi ◽  
Camilo Libedinsky ◽  
Arindam Basu
Keyword(s):  
State Of The Art ◽  
Single Layer ◽  
Open Loop ◽  
Feedback Signal ◽  
Moderate Increase ◽  
Loop Analysis ◽  
Brain Machine Interfaces ◽  
Discrete State ◽  
Binary Feedback ◽  
Recording Channels

AbstractIntra-cortical Brain Machine Interfaces (iBMIs) with wireless capability could scale the number of recording channels by integrating an intention decoder to reduce data rates. However, the need for frequent retraining due to neural signal non-stationarity is a big impediment. This paper presents an alternate paradigm of online reinforcement learning (RL) with a binary evaluative feedback in iBMIs to tackle this issue. This paradigm eliminates time-consuming calibration procedures. Instead, it relies on updating the model on a sequential sample-by-sample basis based on an instantaneous evaluative binary feedback signal. However, batch updates of weight in popular deep networks is very resource consuming and incompatible with constraints of an implant. In this work, using offline open-loop analysis on pre-recorded data, we show application of a simple RL algorithm - Banditron -in discrete-state iBMIs and compare it against previously reported state of the art RL algorithms – Hebbian RL, Attention gated RL, deep Q-learning. Owing to its simplistic single-layer architecture, Banditron is found to yield at least two orders of magnitude of reduction in power dissipation compared to state of the art RL algorithms. At the same time, post-hoc analysis performed on four pre-recorded experimental datasets procured from the motor cortex of two non-human primates performing joystick-based movement-related tasks indicate Banditron performing significantly better than state of the art RL algorithms by at least 5%, 10%, 7% and 7% in experiments 1, 2, 3 and 4 respectively. Furthermore, we propose a non-linear variant of Banditron, Banditron-RP, which gives an average improvement of 6%, 2% in decoding accuracy in experiments 2,4 respectively with only a moderate increase in power consumption.

scholarly journals Towards Autonomous Intra-cortical Brain Machine Interfaces: Applying Bandit Algorithms for Online Reinforcement Learning

2020 ◽  
Author(s):  
Shoeb Shaikh ◽  
Rosa So ◽  
Tafadzwa Sibindi ◽  
Camilo Libedinsky ◽  
Arindam Basu
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Discrete Control ◽  
Brain Machine Interface ◽  
Control Task ◽  
Brain Machine Interfaces ◽  
Discrete State ◽  
Supervised Methods ◽  
Machine Interface

AbstractThis paper presents application of Banditron - an online reinforcement learning algorithm (RL) in a discrete state intra-cortical Brain Machine Interface (iBMI) setting. We have analyzed two datasets from non-human primates (NHPs) - NHP A and NHP B each performing a 4-option discrete control task over a total of 8 days. Results show average improvements of ≈ 15%, 6% in NHP A and 15%, 21% in NHP B over state of the art algorithms - Hebbian Reinforcement Learning (HRL) and Attention Gated Reinforcement Learning (AGREL) respectively. Apart from yielding a superior decoding performance, Banditron is also the most computationally friendly as it requires two orders of magnitude less multiply-and-accumulate operations than HRL and AGREL. Furthermore, Banditron provides average improvements of at least 40%, 15% in NHPs A, B respectively compared to popularly employed supervised methods - LDA, SVM across test days. These results pave the way towards an alternate paradigm of temporally robust hardware friendly reinforcement learning based iBMIs.

scholarly journals RLC-GNN: An Improved Deep Architecture for Spatial-Based Graph Neural Network with Application to Fraud Detection

2021 ◽  
Vol 11 (12) ◽  
pp. 5656
Author(s):  
Yufan Zeng ◽  
Jiashan Tang
Keyword(s):  
Numerical Experiments ◽  
State Of The Art ◽  
Single Layer ◽  
Fraud Detection ◽  
Layer By Layer ◽  
Residual Structure ◽  
Detection Algorithms ◽  
Deep Architecture ◽  
Graph Neural Networks ◽  
Node Embeddings

Graph neural networks (GNNs) have been very successful at solving fraud detection tasks. The GNN-based detection algorithms learn node embeddings by aggregating neighboring information. Recently, CAmouflage-REsistant GNN (CARE-GNN) is proposed, and this algorithm achieves state-of-the-art results on fraud detection tasks by dealing with relation camouflages and feature camouflages. However, stacking multiple layers in a traditional way defined by hop leads to a rapid performance drop. As the single-layer CARE-GNN cannot extract more information to fix the potential mistakes, the performance heavily relies on the only one layer. In order to avoid the case of single-layer learning, in this paper, we consider a multi-layer architecture which can form a complementary relationship with residual structure. We propose an improved algorithm named Residual Layered CARE-GNN (RLC-GNN). The new algorithm learns layer by layer progressively and corrects mistakes continuously. We choose three metrics—recall, AUC, and F1-score—to evaluate proposed algorithm. Numerical experiments are conducted. We obtain up to 5.66%, 7.72%, and 9.09% improvements in recall, AUC, and F1-score, respectively, on Yelp dataset. Moreover, we also obtain up to 3.66%, 4.27%, and 3.25% improvements in the same three metrics on the Amazon dataset.

scholarly journals Single-Layer Line-Fed Broadband Microstrip Patch Antenna on Thin Substrates

Electronics ◽  
2020 ◽  
Vol 10 (1) ◽  
pp. 37
Author(s):  
Roberto Vincenti Gatti ◽  
Riccardo Rossi ◽  
Marco Dionigi
Keyword(s):  
Patch Antenna ◽  
State Of The Art ◽  
Low Cost ◽  
Single Layer ◽  
Microstrip Antennas ◽  
Microstrip Patch Antenna ◽  
Layer Line ◽  
Limited Bandwidth ◽  
Microstrip Patch ◽  
Effective Design

In this work, the issue of limited bandwidth typical of microstrip antennas realized on a single thin substrate is addressed. A simple yet effective design approach is proposed based on the combination of traditional single-resonance patch geometries. Two novel shaped microstrip patch antenna elements with an inset feed are presented. Despite being printed on a single-layer substrate with reduced thickness, both radiators are characterized by a broadband behavior. The antennas are prototyped with a low-cost and fast manufacturing process, and measured results validate the simulations. State-of-the-art performance is obtained when compared to the existing literature, with measured fractional bandwidths of 3.71% and 6.12% around 10 GHz on a 0.508-mm-thick Teflon-based substrate. The small feeding line width could be an appealing feature whenever such radiating elements are to be used in array configurations.

scholarly journals Open‐Loop Analysis of the Renin‐Angiotensin System in the Dog

1971 ◽  
Vol 28 (5) ◽  
pp. 568-581 ◽  
Author(s):  
A. W. COWLEY ◽  
J. P. MILLER ◽  
A. C. GUYTON
Keyword(s):  
Renin Angiotensin System ◽  
Open Loop ◽  
Loop Analysis ◽  
Angiotensin System ◽  
Renin Angiotensin

Outlook of the Dynamic Behavior of Closed-Loop Control through Open-Loop Analysis for Intensified Separation Processes

2018 ◽  
Vol 57 (49) ◽  
pp. 16795-16808
Author(s):  
Julián Cabrera-Ruiz ◽  
César Ramírez-Márquez ◽  
Shinji Hasebe ◽  
Salvador Hernández ◽  
J. Rafael Alcántara Avila
Keyword(s):  
Dynamic Behavior ◽  
Closed Loop ◽  
Open Loop ◽  
Closed Loop Control ◽  
Loop Analysis ◽  
Separation Processes ◽  
Loop Control

scholarly journals Towards a Multi-Layered Phishing Detection

Sensors ◽  
2020 ◽  
Vol 20 (16) ◽  
pp. 4540
Author(s):  
Kieran Rendall ◽  
Antonia Nisioti ◽  
Alexios Mylonas
Keyword(s):  
Machine Learning ◽  
State Of The Art ◽  
Detection System ◽  
Single Layer ◽  
Supervised Machine Learning ◽  
Data Driven ◽  
Feature Sets ◽  
Phishing Attacks ◽  
Production Environments ◽  
Phishing Detection

Phishing is one of the most common threats that users face while browsing the web. In the current threat landscape, a targeted phishing attack (i.e., spear phishing) often constitutes the first action of a threat actor during an intrusion campaign. To tackle this threat, many data-driven approaches have been proposed, which mostly rely on the use of supervised machine learning under a single-layer approach. However, such approaches are resource-demanding and, thus, their deployment in production environments is infeasible. Moreover, most previous works utilise a feature set that can be easily tampered with by adversaries. In this paper, we investigate the use of a multi-layered detection framework in which a potential phishing domain is classified multiple times by models using different feature sets. In our work, an additional classification takes place only when the initial one scores below a predefined confidence level, which is set by the system owner. We demonstrate our approach by implementing a two-layered detection system, which uses supervised machine learning to identify phishing attacks. We evaluate our system with a dataset consisting of active phishing attacks and find that its performance is comparable to the state of the art.

Open Loop Force Control of Piezo-Actuated Stick-Slip Drives

2011 ◽  
Vol 1 (1) ◽  
pp. 1-19 ◽  
Author(s):  
Christoph Edeler ◽  
Sergej Fatikow
Keyword(s):  
Force Control ◽  
State Of The Art ◽  
Open Loop ◽  
Force Generation ◽  
New Method ◽  
Stick Slip ◽  
Micro Actuator ◽  
Friction Models ◽  
Application Fields

In this paper a new method to generate forces with stick-slip micro drives is described. The forces are generated if the runner of the stick-slip drive operates against an obstacle. It is shown that the generated force can be varied selectively without additional sensors and that virtually any force between zero and a limiting force given by certain parameters can be generated. For the investigated micro actuator this force is typically in the range up to hundreds of mN. For this reason, the method has the potential to expand the application fields of stick-slip positioners. After the presentation of the testbed containing the measured linear axis, measurements showing the principle and important parameters are discussed. Furthermore, it is shown that the force generation can be qualitatively simulated using state-of-the-art friction models. Finally, the results are discussed and an outlook is given.

Dynamic Nonlinear Feedback Control Applied to Improve Butanol Production by Clostridium acetobutylicum

2017 ◽  
Vol 15 (6) ◽  
Author(s):  
H. I Velázquez-Sánchez ◽  
G. Lara-Cisneros ◽  
R. Femat ◽  
R. Aguilar-López
Keyword(s):  
Clostridium Acetobutylicum ◽  
Numerical Experiments ◽  
Closed Loop ◽  
Open Loop ◽  
Feedback Signal ◽  
Butanol Production ◽  
Nonlinear Feedback ◽  
Operational Conditions ◽  
Continuous Bioreactor ◽  
Control Scheme

Abstract The goal of this work is to present a closed-loop operational strategy in order to improve the butanol production in an anaerobic continuous bioreactor for the called Acetone-Butanol-Ethanol (ABE) process. The proposed control scheme considers a class of feedback signal which includes a nonlinear bounded function of the regulation error. The control scheme is applied to a phenomenological unstructured kinetic model obtained from an experimental and metabolic study of butanol production by Clostridium acetobutylicum, which allows the proposed structure to predict several operational conditions from batch and continuous regimes. Numerical experiments using the proposed model considering continuous operation were performed in order to find a feasible operating region for maximum butanol production at open-loop regime. The proposed methodology is applied to regulate the product concentration, manipulating the dilution rate to lead to a higher butanol productivity. The closed-loop behaviour of the bioreactor is analysed, finding that the proposed controller minimizes the response time of the system and allows it to achieve a productivity gain of 55 % over open-loop operation. Further numerical experiments show the satisfactory closed-loop performance of the proposed methodology in comparison with a PI controller.

scholarly journals Control of a 3-RRR Planar Parallel Robot Using Fractional Order PID Controller

2020 ◽  
Vol 17 (6) ◽  
pp. 822-836
Author(s):  
Auday Al-Mayyahi ◽  
Ammar A. Aldair ◽  
Chris Chatwin
Keyword(s):  
Fractional Order ◽  
Material Handling ◽  
State Of The Art ◽  
Kinematic Model ◽  
Parallel Robot ◽  
Inverse Kinematic ◽  
Open Loop ◽  
Parallel Robots ◽  
The State ◽  
Revolute Joints

Abstract3-RRR planar parallel robots are utilized for solving precise material-handling problems in industrial automation applications. Thus, robust and stable control is required to deliver high accuracy in comparison to the state of the art. The operation of the mechanism is achieved based on three revolute (3-RRR) joints which are geometrically designed using an open-loop spatial robotic platform. The inverse kinematic model of the system is derived and analyzed by using the geometric structure with three revolute joints. The main variables in our design are the platform base positions, the geometry of the joint angles, and links of the 3-RRR planar parallel robot. These variables are calculated based on Cayley-Menger determinants and bilateration to determine the final position of the platform when moving and placing objects. Additionally, a proposed fractional order proportional integral derivative (FOPID) is optimized using the bat optimization algorithm to control the path tracking of the center of the 3-RRR planar parallel robot. The design is compared with the state of the art and simulated using the Matlab environment to validate the effectiveness of the proposed controller. Furthermore, real-time implementation has been tested to prove that the design performance is practical.

Surface-confined single-layer covalent organic frameworks: design, synthesis and application

2020 ◽  
Vol 49 (7) ◽  
pp. 2020-2038 ◽  
Author(s):  
Daling Cui ◽  
Dmitrii F. Perepichka ◽  
Jennifer M. MacLeod ◽  
Federico Rosei
Keyword(s):  
State Of The Art ◽  
Single Layer ◽  
The State ◽  
Covalent Organic Frameworks ◽  
Design Synthesis ◽  
Guest Chemistry

This review describes the state of the art of surface-confined single-layer covalent organic frameworks, focusing on reticular design, synthesis approaches, and exploring applications in host/guest chemistry.

Sign in / Sign up

Export Citation Format

Share Document