Work-in-Progress: Maximizing I/O throughput and Minimizing Performance Variation via Reinforcement Learning based I/O Merging for SSDs

The value we place on our time impacts what we decide to do with it. Value it too little, and we obsess over all details. Value it too much, and we rush carelessly to move on. How to strike this often context-specific balance is a challenging decision-making problem. Average-reward, putatively encoded by tonic dopamine, serves in existing reinforcement learning theory as the stationary opportunity cost of time. However, environmental context and the cost of deliberation therein often varies in time and is hard to infer and predict. Here, we define a non-stationary opportunity cost of deliberation arising from performance variation on multiple timescales. Estimated from reward history, this cost readily adapts to reward-relevant changes in context and suggests a generalization of average-reward reinforcement learning (AR-RL) to account for non-stationary contextual factors. We use this deliberation cost in a simple decision-making heuristic called Performance-Gated Deliberation, which approximates AR-RL and is consistent with empirical results in both cognitive and systems decision-making neuroscience. We propose that deliberation cost is implemented directly as urgency, a previously characterized neural signal effectively controlling the speed of the decision-making process. We use behaviour and neural recordings from non-human primates in a non-stationary random walk prediction task to support our results. We make readily testable predictions for both neural activity and behaviour and discuss how this proposal can facilitate future work in cognitive and systems neuroscience of reward-driven behaviour.

Download Full-text

Maximizing I/O Throughput and Minimizing Performance Variation via Reinforcement Learning Based I/O Merging for SSDs

IEEE Transactions on Computers ◽

10.1109/tc.2019.2938956 ◽

2020 ◽

Vol 69 (1) ◽

pp. 72-86 ◽

Cited By ~ 5

Author(s):

Chao Wu ◽

Cheng Ji ◽

Qiao Li ◽

Congming Gao ◽

Riwei Pan ◽

...

Keyword(s):

Reinforcement Learning ◽

Performance Variation

Download Full-text

Work in Progress: Role-based Deep Reinforcement Learning with Information Sharing for Intelligent Unmanned Systems

2021 IEEE 27th Real-Time and Embedded Technology and Applications Symposium (RTAS) ◽

10.1109/rtas52030.2021.00059 ◽

2021 ◽

Author(s):

Qingshuang Sun ◽

Yuan Yao ◽

Peng Yi ◽

Xingshe Zhou ◽

Gang Yang

Keyword(s):

Reinforcement Learning ◽

Information Sharing ◽

Unmanned Systems ◽

Work In Progress ◽

Role Based

Download Full-text

Meridional Circulation, Turbulence, and Diffusion Processes in Stars

International Astronomical Union Colloquium ◽

10.1017/s0252921100080477 ◽

1976 ◽

Vol 32 ◽

pp. 109-116 ◽

Cited By ~ 1

Author(s):

S. Vauclair

Keyword(s):

Convection Zone ◽

Diffusion Processes ◽

Meridional Circulation ◽

Diffusion Time ◽

Work In Progress ◽

First Results ◽

Stellar Envelopes ◽

And Diffusion ◽

Turbulence And Diffusion ◽

Hr Diagram

This paper gives the first results of a work in progress, in collaboration with G. Michaud and G. Vauclair. It is a first attempt to compute the effects of meridional circulation and turbulence on diffusion processes in stellar envelopes. Computations have been made for a 2 Mʘstar, which lies in the Am - δ Scuti region of the HR diagram.Let us recall that in Am stars diffusion cannot occur between the two outer convection zones, contrary to what was assumed by Watson (1970, 1971) and Smith (1971), since they are linked by overshooting (Latour, 1972; Toomre et al., 1975). But diffusion may occur at the bottom of the second convection zone. According to Vauclair et al. (1974), the second convection zone, due to He II ionization, disappears after a time equal to the helium diffusion time, and then diffusion may happen at the bottom of the first convection zone, so that the arguments by Watson and Smith are preserved.

Download Full-text