scholarly journals Real-time synthesis of imagined speech processes from minimally invasive recordings of neural activity

2021 ◽  
Vol 4 (1) ◽  
Author(s):  
Miguel Angrick ◽  
Maarten C. Ottenhoff ◽  
Lorenz Diener ◽  
Darius Ivucic ◽  
Gabriel Ivucic ◽  
...  
Keyword(s):  
Real Time ◽  
Neural Activity ◽  
Communication Channel ◽  
Closed Loop ◽  
Open Loop ◽  
Depth Electrodes ◽  
Whispered Speech ◽  
Frontal Activity ◽  
Synthesis Approach

AbstractSpeech neuroprosthetics aim to provide a natural communication channel to individuals who are unable to speak due to physical or neurological impairments. Real-time synthesis of acoustic speech directly from measured neural activity could enable natural conversations and notably improve quality of life, particularly for individuals who have severely limited means of communication. Recent advances in decoding approaches have led to high quality reconstructions of acoustic speech from invasively measured neural activity. However, most prior research utilizes data collected during open-loop experiments of articulated speech, which might not directly translate to imagined speech processes. Here, we present an approach that synthesizes audible speech in real-time for both imagined and whispered speech conditions. Using a participant implanted with stereotactic depth electrodes, we were able to reliably generate audible speech in real-time. The decoding models rely predominately on frontal activity suggesting that speech processes have similar representations when vocalized, whispered, or imagined. While reconstructed audio is not yet intelligible, our real-time synthesis approach represents an essential step towards investigating how patients will learn to operate a closed-loop speech neuroprosthesis based on imagined speech.

scholarly journals Real-time Synthesis of Imagined Speech Processes from Minimally Invasive Recordings of Neural Activity

2020 ◽  
Author(s):  
Miguel Angrick ◽  
Maarten Ottenhoff ◽  
Lorenz Diener ◽  
Darius Ivucic ◽  
Gabriel Ivucic ◽  
...  
Keyword(s):  
Real Time ◽  
Neural Activity ◽  
Communication Channel ◽  
Open Loop ◽  
Human In The Loop ◽  
Depth Electrodes ◽  
Whispered Speech ◽  
Frontal Activity ◽  
Synthesis Approach

ABSTRACTSpeech neuroprosthetics aim to provide a natural communication channel to individuals who are unable to speak due to physical or neurological impairments. Real-time synthesis of acoustic speech directly from measured neural activity could enable natural conversations and significantly improve quality of life, particularly for individuals who have severely limited means of communication. Recent advances in decoding approaches have led to high quality reconstructions of acoustic speech from invasively measured neural activity. However, most prior research utilizes data collected during open-loop experiments of articulated speech, which neglects the critical human-in-the-loop aspect of a practical speech neuroprosthetic.Here we present an approach that synthesizes audible speech in real-time for both imagined and whispered speech conditions. Using a participant implanted with stereotactic depth electrodes, we were able to reliably generate audible speech in real-time. The decoding models rely predominately on frontal activity suggesting that speech processes have similar representations when vocalized, whispered, or imagined. Our real-time synthesis approach represents an essential step towards investigating how patients will learn to operate a closed-loop speech neuroprosthesis, as well as the development of techniques that incorporate co-adaptation of the user and system for optimized performance.

scholarly journals Converting MST to TSP Path by Branch Elimination

2020 ◽  
Vol 11 (1) ◽  
pp. 177
Author(s):  
Pasi Fränti ◽  
Teemu Nenonen ◽  
Mingchuan Yuan
Keyword(s):  
Spanning Tree ◽  
Closed Loop ◽  
Minimum Spanning Tree ◽  
Travelling Salesman Problem ◽  
Open Loop ◽  
Travelling Salesman ◽  
Elimination Algorithm ◽  
Number Of Iterations ◽  
Number Of Branches

Travelling salesman problem (TSP) has been widely studied for the classical closed loop variant but less attention has been paid to the open loop variant. Open loop solution has property of being also a spanning tree, although not necessarily the minimum spanning tree (MST). In this paper, we present a simple branch elimination algorithm that removes the branches from MST by cutting one link and then reconnecting the resulting subtrees via selected leaf nodes. The number of iterations equals to the number of branches (b) in the MST. Typically, b << n where n is the number of nodes. With O-Mopsi and Dots datasets, the algorithm reaches gap of 1.69% and 0.61 %, respectively. The algorithm is suitable especially for educational purposes by showing the connection between MST and TSP, but it can also serve as a quick approximation for more complex metaheuristics whose efficiency relies on quality of the initial solution.

scholarly journals A novel device for real-time measurement and manipulation of licking behavior in head-fixed mice

2018 ◽  
Vol 120 (6) ◽  
pp. 2975-2987 ◽  
Author(s):  
Brice Williams ◽  
Anderson Speed ◽  
Bilal Haider
Keyword(s):  
Real Time ◽  
Neural Activity ◽  
Closed Loop ◽  
Neural Circuits ◽  
Neural Circuit ◽  
Visual Behavior ◽  
Sensory Stimuli ◽  
Neural Recordings ◽  
Control Signals ◽  
Sensory Motor

The mouse has become an influential model system for investigating the mammalian nervous system. Technologies in mice enable recording and manipulation of neural circuits during tasks where they respond to sensory stimuli by licking for liquid rewards. Precise monitoring of licking during these tasks provides an accessible metric of sensory-motor processing, particularly when combined with simultaneous neural recordings. There are several challenges in designing and implementing lick detectors during head-fixed neurophysiological experiments in mice. First, mice are small, and licking behaviors are easily perturbed or biased by large sensors. Second, neural recordings during licking are highly sensitive to electrical contact artifacts. Third, submillisecond lick detection latencies are required to generate control signals that manipulate neural activity at appropriate time scales. Here we designed, characterized, and implemented a contactless dual-port device that precisely measures directional licking in head-fixed mice performing visual behavior. We first determined the optimal characteristics of our detector through design iteration and then quantified device performance under ideal conditions. We then tested performance during head-fixed mouse behavior with simultaneous neural recordings in vivo. We finally demonstrate our device’s ability to detect directional licks and generate appropriate control signals in real time to rapidly suppress licking behavior via closed-loop inhibition of neural activity. Our dual-port detector is cost effective and easily replicable, and it should enable a wide variety of applications probing the neural circuit basis of sensory perception, motor action, and learning in normal and transgenic mouse models. NEW & NOTEWORTHY Mice readily learn tasks in which they respond to sensory cues by licking for liquid rewards; tasks that involve multiple licking responses allow study of neural circuits underlying decision making and sensory-motor integration. Here we design, characterize, and implement a novel dual-port lick detector that precisely measures directional licking in head-fixed mice performing visual behavior, enabling simultaneous neural recording and closed-loop manipulation of licking.

When Causality Does Not Imply Correlation: More Spadework at the Foundations of Scientific Psychology

2011 ◽  
Vol 108 (3) ◽  
pp. 943-954 ◽  
Author(s):  
Richard S. Marken ◽  
Brittany Horth
Keyword(s):  
Sensory Input ◽  
Closed Loop ◽  
Causal Model ◽  
Open Loop ◽  
Tracking Performance ◽  
Loop Model ◽  
Surprising Result ◽  
Scientific Psychology ◽  
Mouse Movements

Experimental research in psychology is based on an open-loop causal model which assumes that sensory input causes behavioral output. This model was tested in a tracking experiment where participants were asked to control a cursor, keeping it aligned with a target by moving a mouse to compensate for disturbances of differing difficulty. Since cursor movements (inputs) are the only observable cause of mouse movements (outputs), the open-loop model predicts that there will be a correlation between input and output that increases as tracking performance improves. In fact, the correlation between sensory input and motor output is very low regardless of the quality of tracking performance; causality, in terms of the effect of input on output, does not seem to imply correlation in this situation. This surprising result can be explained by a closed-loop model which assumes that input is causing output while output is causing input.

scholarly journals Generic characterization of electrical test benches for AC- and HVDC-connected wind power plants

2020 ◽  
Vol 5 (2) ◽  
pp. 561-575
Author(s):  
Behnam Nouri ◽  
Ömer Göksu ◽  
Vahan Gevorgian ◽  
Poul Ejnar Sørensen
Keyword(s):  
Real Time ◽  
Wind Power ◽  
Wind Turbines ◽  
Closed Loop ◽  
Short Circuit ◽  
Open Loop ◽  
Device Under Test ◽  
Generic Structure ◽  
Electrical Test ◽  
Grid Connection

Abstract. The electrical test and assessment of wind turbines go hand in hand with standards and network connection requirements. In this paper, the generic structure of advanced electrical test benches, including grid emulator or controllable grid interface, wind torque emulator, and device under test, is proposed to harmonize state-of-the-art test sites. On the other hand, modern wind turbines are under development towards new features, concerning grid-forming, black-start, and frequency support capabilities as well as harmonic stability and control interaction considerations, to secure the robustness and stability of renewable-energy-based power systems. Therefore, it is necessary to develop new and revised test standards and methodologies to address the new features of wind turbines. This paper proposes a generic test structure within two main groups, including open-loop and closed-loop tests. The open-loop tests include the IEC 61400-21-1 standard tests as well as the additional proposed test options for the new capabilities of wind turbines, which replicate grid connection compliance tests using open-loop references for the grid emulator. In addition, the closed-loop tests evaluate the device under test as part of a virtual wind power plant and perform real-time simulations considering the grid dynamics. The closed-loop tests concern grid connection topologies consisting of AC and HVDC, as well as different electrical characteristics, including impedance, short-circuit ratio, inertia, and background harmonics. The proposed tests can be implemented using available advanced test benches by adjusting their control systems. The characteristics of a real power system can be emulated by a grid emulator coupled with real-time digital simulator systems through a high-bandwidth power-hardware-in-the-loop interface.

scholarly journals Design Considerations for Long Term Non-invasive Brain Computer Interface Training With Tetraplegic CYBATHLON Pilot

2021 ◽  
Vol 15 ◽  
Author(s):  
Neethu Robinson ◽  
Tushar Chouhan ◽  
Ernest Mihelj ◽  
Paulina Kratka ◽  
Frédéric Debraine ◽  
...  
Keyword(s):  
Real Time ◽  
Closed Loop ◽  
Brain Computer Interface ◽  
Brain Activation ◽  
Classification Performance ◽  
Open Loop ◽  
Computer Interface ◽  
Single Subject ◽  
User Engagement

Several studies in the recent past have demonstrated how Brain Computer Interface (BCI) technology can uncover the neural mechanisms underlying various tasks and translate them into control commands. While a multitude of studies have demonstrated the theoretic potential of BCI, a point of concern is that the studies are still confined to lab settings and mostly limited to healthy, able-bodied subjects. The CYBATHLON 2020 BCI race represents an opportunity to further develop BCI design strategies for use in real-time applications with a tetraplegic end user. In this study, as part of the preparation to participate in CYBATHLON 2020 BCI race, we investigate the design aspects of BCI in relation to the choice of its components, in particular, the type of calibration paradigm and its relevance for long-term use. The end goal was to develop a user-friendly and engaging interface suited for long-term use, especially for a spinal-cord injured (SCI) patient. We compared the efficacy of conventional open-loop calibration paradigms with real-time closed-loop paradigms, using pre-trained BCI decoders. Various indicators of performance were analyzed for this study, including the resulting classification performance, game completion time, brain activation maps, and also subjective feedback from the pilot. Our results show that the closed-loop calibration paradigms with real-time feedback is more engaging for the pilot. They also show an indication of achieving better online median classification performance as compared to conventional calibration paradigms (p = 0.0008). We also observe that stronger and more localized brain activation patterns are elicited in the closed-loop paradigm in which the experiment interface closely resembled the end application. Thus, based on this longitudinal evaluation of single-subject data, we demonstrate that BCI-based calibration paradigms with active user-engagement, such as with real-time feedback, could help in achieving better user acceptability and performance.

scholarly journals CLoSES: A platform for closed-loop intracranial stimulation in humans

2020 ◽  
Author(s):  
Rina Zelmann ◽  
Angelique C. Paulk ◽  
Ishita Basu ◽  
Anish Sarma ◽  
Ali Yousefi ◽  
...  
Keyword(s):  
Real Time ◽  
Closed Loop ◽  
Therapeutic Modality ◽  
Patient Specific ◽  
Cognitive State ◽  
Intracranial Stimulation ◽  
Decision Algorithm ◽  
Depth Electrodes ◽  
Neuropsychiatric Diseases ◽  
Epilepsy Monitoring

AbstractTargeted interrogation of brain networks through invasive brain stimulation has become an increasingly important research tool as well as a therapeutic modality. The majority of work with this emerging capability has been focused on open-loop approaches. Closed-loop techniques, however, could improve neuromodulatory therapies and research investigations by optimizing stimulation approaches using neurally informed, personalized targets. Specifically, closed-loop direct electrical stimulation tests in humans performed during semi-chronic electrode implantation in patients with refractory epilepsy could help deepen our understanding of basic research questions as well as the mechanisms and treatment solutions for many neuropsychiatric diseases.However, implementing closed-loop systems is challenging. In particular, during intracranial epilepsy monitoring, electrodes are implanted exclusively for clinical reasons. Thus, detection and stimulation sites must be participant- and task-specific. In addition, the system must run in parallel with clinical systems, integrate seamlessly with existing setups, and ensure safety features. A robust, yet flexible platform is required to perform different tests in a single participant and to comply with clinical settings.In order to investigate closed-loop stimulation for research and therapeutic use, we developed a Closed-Loop System for Electrical Stimulation (CLoSES) that computes neural features which are then used in a decision algorithm to trigger stimulation in near real-time. To summarize CLoSES, intracranial EEG signals are acquired, band-pass filtered, and local and network features are continuously computed. If target features are detected (e.g. above a preset threshold for certain duration), stimulation is triggered. An added benefit is the flexibility of CLoSES. Not only could the system trigger stimulation while detecting real-time neural features, but we incorporated a pipeline wherein we used an encoder/decoder model to estimate a hidden cognitive state from the neural features. Other features include randomly timed stimulation, which percentage of biomarker detections produce stimulation, and safety refractory periods.CLoSES has been successfully used in twelve patients with implanted depth electrodes in the epilepsy monitoring unit during cognitive tasks, spindle detection during sleep, and epileptic activity detection. CLoSES provides a flexible platform to implement a variety of closed-loop experimental paradigms in humans. We anticipate that probing neural dynamics and interaction between brain states and stimulation responses with CLoSES will lead to novel insights into the mechanism of normal and pathological brain activity, the discovery and evaluation of potential electrographic biomarkers of neurological and psychiatric disorders, and the development and testing of patient-specific stimulation targets and control signals before implanting a therapeutic device.

scholarly journals Marked point process filter for clusterless and adaptive encoding-decoding of multiunit activity

2018 ◽  
Author(s):  
Kensuke Arai ◽  
Daniel F. Liu ◽  
Loren M. Frank ◽  
Uri T. Eden
Keyword(s):  
Real Time ◽  
Point Process ◽  
Neural Activity ◽  
Closed Loop ◽  
Marked Point ◽  
Marked Point Process ◽  
Important Advance ◽  
New Information ◽  
Adaptive Encoding ◽  
And Behavior

AbstractReal-time, closed-loop experiments can uncover causal relationships between specific neural activity and behavior. An important advance in realizing this is the marked point process filtering framework which utilizes the “mark” or the waveform features of unsorted spikes, to construct a relationship between these features and behavior, which we call the encoding model. This relationship is not fixed, because learning changes coding properties of individual neurons, and electrodes can physically move during the experiment, changing waveform characteristics. We introduce a sequential, Bayesian encoding model which allows incorporation of new information on the fly to adapt the model in real time. A possible application of this framework is to the decoding of the contents of hippocampal ripples in rats exploring a maze. During physical exploration, we observe the marks and positions at which they occur, to update the encoding model, which is employed to decode contents of ripples when rats stop moving, and switch back to updating the model once the rat starts moving again.

scholarly journals Design of a Research Laboratory Drive System for a Synchronous Reluctance Motor for Vector Control and Performance Analysis

Inventions ◽  
2021 ◽  
Vol 6 (4) ◽  
pp. 64
Author(s):  
Hamidreza Heidari ◽  
Anton Rassõlkin ◽  
Ants Kallaste ◽  
Toomas Vaimann ◽  
Ekaterina Andriushchenko ◽  
...  
Keyword(s):  
Performance Analysis ◽  
Vector Control ◽  
Real Time ◽  
Research Laboratory ◽  
Closed Loop ◽  
Open Loop ◽  
Drive System ◽  
Control Algorithms ◽  
Synchronous Reluctance Motor ◽  
Reluctance Motor

Motor-drive systems have the most significant share in industrial energy consumption, which requires a deep study in every aspect of the field. This paper presents a synchronous reluctance motor (SynRM) drive system based on Plecs RT box 1. The system’s design provides the opportunity for the open-loop and closed-loop control of the motor and a characteristic performance analysis of the motor. This paper focuses on the hardware implementation of a research laboratory setup and the precise vector control of the SynRM in real-time. The application of the digital controller and inverter to drive SynRM is examined. The voltage, current, and speed transducers were employed for monitoring the protective measures and to control the motor in the closed-loop. The design of the signal conditioning and the intermediary cards for isolation and data acquisition are described in detail. An algorithm is proposed to measure the whole system parameters, including motor, inverter, and cables. Thanks to the RT box 1, the principle of real-time simulation of control algorithms is investigated, and the rapid control prototyping of field-oriented control (FOC) of SynRM was implemented. The simulation of the system was carried out in the Plecs platform, and the results are presented. The experimental results of the implemented control algorithms validate the setup’s performance and the control algorithm. Finally, as a study of the motor’s performance, the efficiency map of the motor is drawn in different speed and torque ranges.

Sign in / Sign up

Export Citation Format

Share Document