scholarly journals Deep Reinforcement Learning For Trading—A Critical Survey

Data ◽  
2021 ◽  
Vol 6 (11) ◽  
pp. 119
Author(s):  
Adrian Millea
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Financial Markets ◽  
Risk Measures ◽  
Google Scholar ◽  
Critical Survey ◽  
Problem Space ◽  
World Model ◽  
Short Survey ◽  
Reward Shaping

Deep reinforcement learning (DRL) has achieved significant results in many machine learning (ML) benchmarks. In this short survey, we provide an overview of DRL applied to trading on financial markets with the purpose of unravelling common structures used in the trading community using DRL, as well as discovering common issues and limitations of such approaches. We include also a short corpus summarization using Google Scholar. Moreover, we discuss how one can use hierarchy for dividing the problem space, as well as using model-based RL to learn a world model of the trading environment which can be used for prediction. In addition, multiple risk measures are defined and discussed, which not only provide a way of quantifying the performance of various algorithms, but they can also act as (dense) reward-shaping mechanisms for the agent. We discuss in detail the various state representations used for financial markets, which we consider critical for the success and efficiency of such DRL agents. The market in focus for this survey is the cryptocurrency market; the results of this survey are two-fold: firstly, to find the most promising directions for further research and secondly, to show how a lack of consistency in the community can significantly impede research and the development of DRL agents for trading.

scholarly journals Deep Reinforcement Learning For Trading - A Critical Survey

2021 ◽  
Author(s):  
Adrian Millea
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Financial Markets ◽  
Risk Measures ◽  
Meta Analysis ◽  
Google Scholar ◽  
Critical Survey ◽  
Problem Space ◽  
World Model ◽  
Reward Shaping

Deep reinforcement learning (DRL) has achieved significant results in many Machine Learning (ML) benchmarks. In this short survey we provide an overview of DRL applied to trading on financial markets, including a short meta-analysis using Google Scholar, with an emphasis on using hierarchy for dividing the problem space as well as using model-based RL to learn a world model of the trading environment which can be used for prediction. In addition, multiple risk measures are defined and discussed, which not only provide a way of quantifying the performance of various algorithms, but they can also act as (dense) reward-shaping mechanisms for the agent. We discuss in detail the various state representations used for financial markets, which we consider critical for the success and efficiency of such DRL agents. The market in focus for this survey is the cryptocurrency market.

scholarly journals Dreaming machine learning: Lipschitz extensions for reinforcement learning on financial markets

2020 ◽  
Vol 398 ◽  
pp. 172-184 ◽  
Author(s):  
J.M. Calabuig ◽  
H. Falciani ◽  
E.A. Sánchez-Pérez
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Financial Markets ◽  
Lipschitz Extensions

Learning and control

2018 ◽  
Author(s):  
Ivan Herreros
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Brain Function ◽  
Control Strategies ◽  
Learning Problems ◽  
Animal Learning ◽  
Feed Forward Control ◽  
Machine Learning Applications ◽  
And Control

This chapter discusses basic concepts from control theory and machine learning to facilitate a formal understanding of animal learning and motor control. It first distinguishes between feedback and feed-forward control strategies, and later introduces the classification of machine learning applications into supervised, unsupervised, and reinforcement learning problems. Next, it links these concepts with their counterparts in the domain of the psychology of animal learning, highlighting the analogies between supervised learning and classical conditioning, reinforcement learning and operant conditioning, and between unsupervised and perceptual learning. Additionally, it interprets innate and acquired actions from the standpoint of feedback vs anticipatory and adaptive control. Finally, it argues how this framework of translating knowledge between formal and biological disciplines can serve us to not only structure and advance our understanding of brain function but also enrich engineering solutions at the level of robot learning and control with insights coming from biology.

scholarly journals Quantum Reinforcement Learning with Quantum Photonics

Photonics ◽  
2021 ◽  
Vol 8 (2) ◽  
pp. 33
Author(s):  
Lucas Lamata
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Quantum Computation ◽  
Exchange Information ◽  
Quantum Machine Learning ◽  
Quantum Technology ◽  
Quantum Photonics

Quantum machine learning has emerged as a promising paradigm that could accelerate machine learning calculations. Inside this field, quantum reinforcement learning aims at designing and building quantum agents that may exchange information with their environment and adapt to it, with the aim of achieving some goal. Different quantum platforms have been considered for quantum machine learning and specifically for quantum reinforcement learning. Here, we review the field of quantum reinforcement learning and its implementation with quantum photonics. This quantum technology may enhance quantum computation and communication, as well as machine learning, via the fruitful marriage between these previously unrelated fields.

scholarly journals How to train your robot with deep reinforcement learning: lessons we have learned

2021 ◽  
pp. 027836492098785
Author(s):  
Julian Ibarz ◽  
Jie Tan ◽  
Chelsea Finn ◽  
Mrinal Kalakrishnan ◽  
Peter Pastor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Case Studies ◽  
Real World ◽  
Review Article ◽  
The Real ◽  
Complex Skills ◽  
Real World Learning ◽  
Level Sensor ◽  
Embodied Agent

Deep reinforcement learning (RL) has emerged as a promising approach for autonomously acquiring complex behaviors from low-level sensor observations. Although a large portion of deep RL research has focused on applications in video games and simulated control, which does not connect with the constraints of learning in real environments, deep RL has also demonstrated promise in enabling physical robots to learn complex skills in the real world. At the same time, real-world robotics provides an appealing domain for evaluating such algorithms, as it connects directly to how humans learn: as an embodied agent in the real world. Learning to perceive and move in the real world presents numerous challenges, some of which are easier to address than others, and some of which are often not considered in RL research that focuses only on simulated domains. In this review article, we present a number of case studies involving robotic deep RL. Building off of these case studies, we discuss commonly perceived challenges in deep RL and how they have been addressed in these works. We also provide an overview of other outstanding challenges, many of which are unique to the real-world robotics setting and are not often the focus of mainstream RL research. Our goal is to provide a resource both for roboticists and machine learning researchers who are interested in furthering the progress of deep RL in the real world.

scholarly journals Signature-based and Machine-Learning-based Web Application Firewalls: A Short Survey

2021 ◽  
Vol 189 ◽  
pp. 359-367
Author(s):  
Simon Applebaum ◽  
Tarek Gaber ◽  
Ali Ahmed
Keyword(s):  
Machine Learning ◽  
Web Application ◽  
Short Survey

scholarly journals Applying a Deep Q Network for OpenAIs Car Racing Game

2020 ◽  
Author(s):  
Ali Fakhry
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Transfer Learning ◽  
State Of The Art ◽  
Learning Techniques ◽  
Car Racing ◽  
Custom Made ◽  
Learning Technique ◽  
Reward Threshold

The applications of Deep Q-Networks are seen throughout the field of reinforcement learning, a large subsect of machine learning. Using a classic environment from OpenAI, CarRacing-v0, a 2D car racing environment, alongside a custom based modification of the environment, a DQN, Deep Q-Network, was created to solve both the classic and custom environments. The environments are tested using custom made CNN architectures and applying transfer learning from Resnet18. While DQNs were state of the art years ago, using it for CarRacing-v0 appears somewhat unappealing and not as effective as other reinforcement learning techniques. Overall, while the model did train and the agent learned various parts of the environment, attempting to reach the reward threshold for the environment with this reinforcement learning technique seems problematic and difficult as other techniques would be more useful.

scholarly journals Temporal Uncertainty During Overshadowing

2011 ◽  
pp. 46-55
Author(s):  
Dómhnall J. Jennings ◽  
Eduardo Alonso ◽  
Esther Mondragón ◽  
Charlotte Bonardi
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Associative Learning ◽  
Learning Theories ◽  
Temporal Difference ◽  
Temporal Uncertainty ◽  
Learning Problem ◽  
Difference Model ◽  
Distribution Form ◽  
Temporal Properties

Standard associative learning theories typically fail to conceptualise the temporal properties of a stimulus, and hence cannot easily make predictions about the effects such properties might have on the magnitude of conditioning phenomena. Despite this, in intuitive terms we might expect that the temporal properties of a stimulus that is paired with some outcome to be important. In particular, there is no previous research addressing the way that fixed or variable duration stimuli can affect overshadowing. In this chapter we report results which show that the degree of overshadowing depends on the distribution form - fixed or variable - of the overshadowing stimulus, and argue that conditioning is weaker under conditions of temporal uncertainty. These results are discussed in terms of models of conditioning and timing. We conclude that the temporal difference model, which has been extensively applied to the reinforcement learning problem in machine learning, accounts for the key findings of our study.

scholarly journals Label Enhancement for Label Distribution Learning via Prior Knowledge

2020 ◽  
Author(s):  
Yongbiao Gao ◽  
Yu Zhang ◽  
Xin Geng
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Emotion Recognition ◽  
Prior Knowledge ◽  
Decision Process ◽  
Age Estimation ◽  
State Of The Art ◽  
Learning Agent ◽  
Label Distribution Learning ◽  
Label Distribution

Label distribution learning (LDL) is a novel machine learning paradigm that gives a description degree of each label to an instance. However, most of training datasets only contain simple logical labels rather than label distributions due to the difficulty of obtaining the label distributions directly. We propose to use the prior knowledge to recover the label distributions. The process of recovering the label distributions from the logical labels is called label enhancement. In this paper, we formulate the label enhancement as a dynamic decision process. Thus, the label distribution is adjusted by a series of actions conducted by a reinforcement learning agent according to sequential state representations. The target state is defined by the prior knowledge. Experimental results show that the proposed approach outperforms the state-of-the-art methods in both age estimation and image emotion recognition.

Uprising E-sports Industry: machine learning/AI improve in-game performance using deep reinforcement learning

2021 ◽  
Author(s):  
Xianzuo Du ◽  
Xiwei Fuqian ◽  
Jiaxi Hu ◽  
Zechen Wang ◽  
Dongju Yang
Keyword(s):  
Machine Learning ◽  
Reinforcement Learning ◽  
Sports Industry ◽  
Game Performance
Sign in / Sign up

Export Citation Format

Share Document