Selecting Portfolios Directly Using Recurrent Reinforcement Learning (Student Abstract)

Portfolio selection has attracted increasing attention in machine learning and AI communities recently. Existing portfolio selection using recurrent reinforcement learning (RRL) heavily relies on single asset trading system to heuristically obtain the portfolio weights. In this paper, we propose a novel method, the direct portfolio selection using recurrent reinforcement learning (DPS-RRL), to select portfolios directly. Instead of trading single asset one by one to obtain portfolio weights, our method learns to quantify the asset allocation weight directly via optimizing the Sharpe ratio of financial portfolios. We empirically demonstrate the effectiveness of our method, which is able to outperform state-of-the-art portfolio selection methods.

Download Full-text

Reinforcement Learning with Self-Attention Networks for Cryptocurrency Trading

Applied Sciences ◽

10.3390/app11167377 ◽

2021 ◽

Vol 11 (16) ◽

pp. 7377

Author(s):

Carlos Betancourt ◽

Wen-Hui Chen

Keyword(s):

Reinforcement Learning ◽

State Of The Art ◽

Experimental Results ◽

Financial Assets ◽

Trading System ◽

Attention Network ◽

Attention Networks ◽

High Frequencies ◽

Transaction Fees ◽

Using Data

This work presents an application of self-attention networks for cryptocurrency trading. Cryptocurrencies are extremely volatile and unpredictable. Thus, cryptocurrency trading is challenging and involves higher risks than trading traditional financial assets such as stocks. To overcome the aforementioned problems, we propose a deep reinforcement learning (DRL) approach for cryptocurrency trading. The proposed trading system contains a self-attention network trained using an actor-critic DRL algorithm. Cryptocurrency markets contain hundreds of assets, allowing greater investment diversification, which can be accomplished if all the assets are analyzed against one another. Self-attention networks are suitable for dealing with the problem because the attention mechanism can process long sequences of data and focus on the most relevant parts of the inputs. Transaction fees are also considered in formulating the studied problem. Systems that perform trades in high frequencies cannot overlook this issue, since, after many trades, small fees can add up to significant expenses. To validate the proposed approach, a DRL environment is built using data from an important cryptocurrency market. We test our method against a state-of-the-art baseline in two different experiments. The experimental results show the proposed approach can obtain higher daily profits and has several advantages over existing methods.

Download Full-text

Object-sensitive Deep Reinforcement Learning

10.29007/xtgm ◽

2018 ◽

Author(s):

Yuezhang Li ◽

Katia Sycara ◽

Rahul Iyer

Keyword(s):

Reinforcement Learning ◽

State Of The Art ◽

Robot Navigation ◽

Learning Models ◽

New Approach ◽

Learning Agents ◽

Saliency Maps ◽

Novel Method ◽

Reinforcement Learning Models ◽

Learning Frameworks

Deep reinforcement learning has become popular over recent years, showing superiority on different visual-input tasks such as playing Atari games and robot navigation. Although objects are important image elements, few work considers enhancing deep reinforcement learning with object characteristics. In this paper, we propose a novel method that can incorporate object recognition processing to deep reinforcement learning models. This approach can be adapted to any existing deep reinforcement learning frameworks. State-of-the-art results are shown in experiments on Atari games. We also propose a new approach called “object saliency maps” to visually explain the actions made by deep reinforcement learning agents.

Download Full-text

COVID-19 infection map generation and detection from chest X-ray images

Health Information Science and Systems ◽

10.1007/s13755-021-00146-8 ◽

2021 ◽

Vol 9 (1) ◽

Author(s):

Aysen Degerli ◽

Mete Ahishali ◽

Mehmet Yamac ◽

Serkan Kiranyaz ◽

Muhammad E. H. Chowdhury ◽

...

Keyword(s):

State Of The Art ◽

Ground Truth ◽

Clinical Use ◽

X Ray ◽

Learning Techniques ◽

Map Generation ◽

Severity Grading ◽

Chest X Ray ◽

Novel Method ◽

Aided Diagnosis

AbstractComputer-aided diagnosis has become a necessity for accurate and immediate coronavirus disease 2019 (COVID-19) detection to aid treatment and prevent the spread of the virus. Numerous studies have proposed to use Deep Learning techniques for COVID-19 diagnosis. However, they have used very limited chest X-ray (CXR) image repositories for evaluation with a small number, a few hundreds, of COVID-19 samples. Moreover, these methods can neither localize nor grade the severity of COVID-19 infection. For this purpose, recent studies proposed to explore the activation maps of deep networks. However, they remain inaccurate for localizing the actual infestation making them unreliable for clinical use. This study proposes a novel method for the joint localization, severity grading, and detection of COVID-19 from CXR images by generating the so-called infection maps. To accomplish this, we have compiled the largest dataset with 119,316 CXR images including 2951 COVID-19 samples, where the annotation of the ground-truth segmentation masks is performed on CXRs by a novel collaborative human–machine approach. Furthermore, we publicly release the first CXR dataset with the ground-truth segmentation masks of the COVID-19 infected regions. A detailed set of experiments show that state-of-the-art segmentation networks can learn to localize COVID-19 infection with an F1-score of 83.20%, which is significantly superior to the activation maps created by the previous methods. Finally, the proposed approach achieved a COVID-19 detection performance with 94.96% sensitivity and 99.88% specificity.

Download Full-text

Distinguishing Adenocarcinomas from Granulomas in the CT scan of the chest: performance degradation evaluation in the automatic segmentation framework

BMC Research Notes ◽

10.1186/s13104-021-05502-1 ◽

2021 ◽

Vol 14 (1) ◽

Author(s):

Mahsa Bank Tavakoli ◽

Mahdi Orooji ◽

Mehdi Teimouri ◽

Ramita Shahabifar

Keyword(s):

State Of The Art ◽

Automatic Segmentation ◽

Texture Features ◽

Standards Of Care ◽

Shape Features ◽

Selection Methods ◽

Segmentation Framework ◽

Vessel Tortuosity ◽

Benign Nodules ◽

Degradation Evaluation

Abstract Objective The most common histopathologic malignant and benign nodules are Adenocarcinoma and Granuloma, respectively, which have different standards of care. In this paper, we propose an automatic framework for the diagnosis of the Adenocarcinomas and the Granulomas in the CT scans of the chest from a private dataset. We use the radiomic features of the nodules and the attached vessel tortuosity for the diagnosis. The private dataset includes 22 CTs for each nodule type, i.e., adenocarcinoma and granuloma. The dataset contains the CTs of the non-smoker patients who are between 30 and 60 years old. To automatically segment the delineated nodule area and the attached vessels area, we apply a morphological-based approach. For distinguishing the malignancy of the segmented nodule, two texture features of the nodule, the curvature Mean and the number of the attached vessels are extracted. Results We compare our framework with the state-of-the-art feature selection methods for differentiating Adenocarcinomas from Granulomas. These methods employ only the shape features of the nodule, the texture features of the nodule, or the torsion features of the attached vessels along with the radiomic features of the nodule. The accuracy of our framework is improved by considering the four selected features.

Download Full-text

ART-UP: A Novel Method for Generating Scanning-Robust Aesthetic QR Codes

ACM Transactions on Multimedia Computing Communications and Applications ◽

10.1145/3418214 ◽

2021 ◽

Vol 17 (1) ◽

pp. 1-23

Author(s):

Mingliang Xu ◽

Qingfeng Li ◽

Jianwei Niu ◽

Hao Su ◽

Xiting Liu ◽

...

Keyword(s):

State Of The Art ◽

Visual Quality ◽

Qr Code ◽

Quick Response ◽

Estimation Model ◽

Qr Codes ◽

Excellent Performance ◽

Novel Method ◽

Coarse To Fine

Quick response (QR) codes are usually scanned in different environments, so they must be robust to variations in illumination, scale, coverage, and camera angles. Aesthetic QR codes improve the visual quality, but subtle changes in their appearance may cause scanning failure. In this article, a new method to generate scanning-robust aesthetic QR codes is proposed, which is based on a module-based scanning probability estimation model that can effectively balance the tradeoff between visual quality and scanning robustness. Our method locally adjusts the luminance of each module by estimating the probability of successful sampling. The approach adopts the hierarchical, coarse-to-fine strategy to enhance the visual quality of aesthetic QR codes, which sequentially generate the following three codes: a binary aesthetic QR code, a grayscale aesthetic QR code, and the final color aesthetic QR code. Our approach also can be used to create QR codes with different visual styles by adjusting some initialization parameters. User surveys and decoding experiments were adopted for evaluating our method compared with state-of-the-art algorithms, which indicates that the proposed approach has excellent performance in terms of both visual quality and scanning robustness.

Download Full-text

Synthetic Experiences for Accelerating DQN Performance in Discrete Non-Deterministic Environments

Algorithms ◽

10.3390/a14080226 ◽

2021 ◽

Vol 14 (8) ◽

pp. 226

Author(s):

Wenzel Pilar von Pilchau ◽

Anthony Stein ◽

Jörg Hähner

Keyword(s):

Reinforcement Learning ◽

State Of The Art ◽

Learning Algorithms ◽

Weighted Average ◽

Up States ◽

Experience Replay

State-of-the-art Deep Reinforcement Learning Algorithms such as DQN and DDPG use the concept of a replay buffer called Experience Replay. The default usage contains only the experiences that have been gathered over the runtime. We propose a method called Interpolated Experience Replay that uses stored (real) transitions to create synthetic ones to assist the learner. In this first approach to this field, we limit ourselves to discrete and non-deterministic environments and use a simple equally weighted average of the reward in combination with observed follow-up states. We could demonstrate a significantly improved overall mean average in comparison to a DQN network with vanilla Experience Replay on the discrete and non-deterministic FrozenLake8x8-v0 environment.

Download Full-text

Drone Deep Reinforcement Learning: A Review

Electronics ◽

10.3390/electronics10090999 ◽

2021 ◽

Vol 10 (9) ◽

pp. 999

Author(s):

Ahmad Taher Azar ◽

Anis Koubaa ◽

Nada Ali Mohamed ◽

Habiba A. Ibrahim ◽

Zahra Fathy Ibrahim ◽

...

Keyword(s):

Reinforcement Learning ◽

State Of The Art ◽

Real Life ◽

Environment Monitoring ◽

Simulated Environments ◽

Infrastructure Inspection ◽

Remote Sensing Mapping ◽

And Control ◽

The Military ◽

Uav Navigation

Unmanned Aerial Vehicles (UAVs) are increasingly being used in many challenging and diversified applications. These applications belong to the civilian and the military fields. To name a few; infrastructure inspection, traffic patrolling, remote sensing, mapping, surveillance, rescuing humans and animals, environment monitoring, and Intelligence, Surveillance, Target Acquisition, and Reconnaissance (ISTAR) operations. However, the use of UAVs in these applications needs a substantial level of autonomy. In other words, UAVs should have the ability to accomplish planned missions in unexpected situations without requiring human intervention. To ensure this level of autonomy, many artificial intelligence algorithms were designed. These algorithms targeted the guidance, navigation, and control (GNC) of UAVs. In this paper, we described the state of the art of one subset of these algorithms: the deep reinforcement learning (DRL) techniques. We made a detailed description of them, and we deduced the current limitations in this area. We noted that most of these DRL methods were designed to ensure stable and smooth UAV navigation by training computer-simulated environments. We realized that further research efforts are needed to address the challenges that restrain their deployment in real-life scenarios.

Download Full-text

Design of True Random Number Circuit with Controllable Frequency

Electronics ◽

10.3390/electronics10131517 ◽

2021 ◽

Vol 10 (13) ◽

pp. 1517

Author(s):

Xinsheng Wang ◽

Xiyue Wang

Keyword(s):

Random Number ◽

Noise Source ◽

State Of The Art ◽

Building Blocks ◽

Encryption Algorithm ◽

Random Numbers ◽

Random Number Generators ◽

Random Telegraph Noise ◽

Telegraph Noise ◽

Novel Method

True random number generators (TRNGs) have been a research hotspot due to secure encryption algorithm requirements. Therefore, such circuits are necessary building blocks in state-of-the-art security controllers. In this paper, a TRNG based on random telegraph noise (RTN) with a controllable rate is proposed. A novel method of noise array circuits is presented, which consists of digital decoder circuits and RTN noise circuits. The frequency of generating random numbers is controlled by the speed of selecting different gating signals. The results of simulation show that the array circuits consist of 64 noise source circuits that can generate random numbers by a frequency from 1 kHz to 16 kHz.

Download Full-text