scholarly journals Stopping criteria for ending autonomous, single detector radiological source searches

PLoS ONE ◽  
2021 ◽  
Vol 16 (6) ◽  
pp. e0253211
Author(s):  
Gregory R. Romanchek ◽  
Shiva Abbaszadeh
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Source Localization ◽  
Likelihood Estimation ◽  
Mobile Systems ◽  
Localization Error ◽  
Stopping Criteria ◽  
Perfect Knowledge ◽  
Simulated Environment ◽  
Single Detector

While the localization of radiological sources has traditionally been handled with statistical algorithms, such a task can be augmented with advanced machine learning methodologies. The combination of deep and reinforcement learning has provided learning-based navigation to autonomous, single-detector, mobile systems. However, these approaches lacked the capacity to terminate a surveying/search task without outside influence of an operator or perfect knowledge of source location (defeating the purpose of such a system). Two stopping criteria are investigated in this work for a machine learning navigated system: one based upon Bayesian and maximum likelihood estimation (MLE) strategies commonly used in source localization, and a second providing the navigational machine learning network with a “stop search” action. A convolutional neural network was trained via reinforcement learning in a 10 m × 10 m simulated environment to navigate a randomly placed detector-agent to a randomly placed source of varied strength (stopping with perfect knowledge during training). The network agent could move in one of four directions (up, down, left, right) after taking a 1 s count measurement at the current location. During testing, the stopping criteria for this navigational algorithm was based upon a Bayesian likelihood estimation technique of source presence, updating this likelihood after each step, and terminating once the confidence of the source being in a single location exceeded 0.9. A second network was trained and tested with similar architecture as the previous but which contained a fifth action: for self-stopping. The accuracy and speed of localization with set detector and source initializations were compared over 50 trials of MLE-Bayesian approach and 1000 trials of the CNN with self-stopping. The statistical stopping condition yielded a median localization error of ~1.41 m and median localization speed of 12 steps. The machine learning stopping condition yielded a median localization error of 0 m and median localization speed of 17 steps. This work demonstrated two stopping criteria available to a machine learning guided, source localization system.

scholarly journals Representation Learning for Grounded Spatial Reasoning

2018 ◽  
Vol 6 ◽  
pp. 49-61 ◽  
Author(s):  
Michael Janner ◽  
Karthik Narasimhan ◽  
Regina Barzilay
Keyword(s):  
Reinforcement Learning ◽  
Spatial Reasoning ◽  
State Of The Art ◽  
Representation Learning ◽  
Localization Error ◽  
Value Iteration ◽  
Joint Inference ◽  
Simulated Environment ◽  
Proposed Model ◽  
The World

The interpretation of spatial references is highly contextual, requiring joint inference over both language and the environment. We consider the task of spatial reasoning in a simulated environment, where an agent can act and receive rewards. The proposed model learns a representation of the world steered by instruction text. This design allows for precise alignment of local neighborhoods with corresponding verbalizations, while also handling global references in the instructions. We train our model with reinforcement learning using a variant of generalized value iteration. The model outperforms state-of-the-art approaches on several metrics, yielding a 45% reduction in goal localization error.

Comparison of Machine Learning Techniques Based Brain Source Localization Using EEG Signals

2020 ◽  
Vol 16 ◽  
Author(s):  
Munsif Ali Jatoi ◽  
Fayaz Ali Dharejo ◽  
Sadam Hussain Teevino
Keyword(s):  
Machine Learning ◽  
Free Energy ◽  
Source Localization ◽  
Research Work ◽  
Bayesian Framework ◽  
Machine Learning Techniques ◽  
Localization Error ◽  
The Impact ◽  
The Brain ◽  
Brain Source Localization

Background: The Brain is the most complex organ of human body with millions of connections and activations. The electromagnetic signals are generated inside the brain due to a mental or physical task performed. These signals excite a bunch of neurons within a particular lobe depending upon nature of task performed. To localize this activity, certain machine learning (ML) techniques in conjunction with a neuroimaging technique (M/EEG, fMRI, PET) are developed. Different ML techniques are provided in literature for brain source localization. Among them, the most common are: minimum norm estimation (MNE), low resolution brain electromagnetic tomography (LORETA) and Bayesian framework based multiple sparse priors (MSP). Aims: In this research work, EEG is used as a neuroimaging technique. Methods: EEG data is synthetically generated at SNR=5dB. Afterwards, ML techniques are applied to estimate the active sources. Each dataset is run for multiple trials (>40). The performance is analyzed using free energy and localization error as performance indicators. Furthermore, MSP is applied with variant number of patches to observe the impact of patches on source localization. Results: It is observed that with increased number of patches, the sources are localized with more precision and accuracy as expressed in terms of free energy and localization error respectively. Conclusion: The patches optimization within Bayesian Framework produces improved results in terms of free energy and localization error.

Learning and control

2018 ◽  
Author(s):  
Ivan Herreros
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Brain Function ◽  
Control Strategies ◽  
Learning Problems ◽  
Animal Learning ◽  
Feed Forward Control ◽  
Machine Learning Applications ◽  
And Control

This chapter discusses basic concepts from control theory and machine learning to facilitate a formal understanding of animal learning and motor control. It first distinguishes between feedback and feed-forward control strategies, and later introduces the classification of machine learning applications into supervised, unsupervised, and reinforcement learning problems. Next, it links these concepts with their counterparts in the domain of the psychology of animal learning, highlighting the analogies between supervised learning and classical conditioning, reinforcement learning and operant conditioning, and between unsupervised and perceptual learning. Additionally, it interprets innate and acquired actions from the standpoint of feedback vs anticipatory and adaptive control. Finally, it argues how this framework of translating knowledge between formal and biological disciplines can serve us to not only structure and advance our understanding of brain function but also enrich engineering solutions at the level of robot learning and control with insights coming from biology.

scholarly journals Quantum Reinforcement Learning with Quantum Photonics

Photonics ◽  
2021 ◽  
Vol 8 (2) ◽  
pp. 33
Author(s):  
Lucas Lamata
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Quantum Computation ◽  
Exchange Information ◽  
Quantum Machine Learning ◽  
Quantum Technology ◽  
Quantum Photonics

Quantum machine learning has emerged as a promising paradigm that could accelerate machine learning calculations. Inside this field, quantum reinforcement learning aims at designing and building quantum agents that may exchange information with their environment and adapt to it, with the aim of achieving some goal. Different quantum platforms have been considered for quantum machine learning and specifically for quantum reinforcement learning. Here, we review the field of quantum reinforcement learning and its implementation with quantum photonics. This quantum technology may enhance quantum computation and communication, as well as machine learning, via the fruitful marriage between these previously unrelated fields.

scholarly journals How to train your robot with deep reinforcement learning: lessons we have learned

2021 ◽  
pp. 027836492098785
Author(s):  
Julian Ibarz ◽  
Jie Tan ◽  
Chelsea Finn ◽  
Mrinal Kalakrishnan ◽  
Peter Pastor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Case Studies ◽  
Real World ◽  
Review Article ◽  
The Real ◽  
Complex Skills ◽  
Real World Learning ◽  
Level Sensor ◽  
Embodied Agent

Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.

scholarly journals Applying a Deep Q Network for OpenAIs Car Racing Game

2020 ◽  
Author(s):  
Ali Fakhry
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Transfer Learning ◽  
State Of The Art ◽  
Learning Techniques ◽  
Car Racing ◽  
Custom Made ◽  
Learning Technique ◽  
Reward Threshold

The applications of Deep Q-Networks are seen throughout the field of reinforcement learning, a large subsect of machine learning. Using a classic environment from OpenAI, CarRacing-v0, a 2D car racing environment, alongside a custom based modification of the environment, a DQN, Deep Q-Network, was created to solve both the classic and custom environments. The environments are tested using custom made CNN architectures and applying transfer learning from Resnet18. While DQNs were state of the art years ago, using it for CarRacing-v0 appears somewhat unappealing and not as effective as other reinforcement learning techniques. Overall, while the model did train and the agent learned various parts of the environment, attempting to reach the reward threshold for the environment with this reinforcement learning technique seems problematic and difficult as other techniques would be more useful.

Sign in / Sign up

Export Citation Format

Share Document