Robustly Learning Composable Options in Deep Reinforcement Learning

Hierarchical reinforcement learning (HRL) is only effective for long-horizon problems when high-level skills can be reliably sequentially executed. Unfortunately, learning reliably composable skills is difficult, because all the components of every skill are constantly changing during learning. We propose three methods for improving the composability of learned skills: representing skill initiation regions using a combination of pessimistic and optimistic classifiers; learning re-targetable policies that are robust to non-stationary subgoal regions; and learning robust option policies using model-based RL. We test these improvements on four sparse-reward maze navigation tasks involving a simulated quadrupedal robot. Each method successively improves the robustness of a baseline skill discovery method, substantially outperforming state-of-the-art flat and hierarchical methods.

Annualization of Renewable Investment Costs for Finite Horizon Electricity Pricing and Cost Recovery

The increasing penetration of renewable electricity generation is complicating the bidding and estimating processes of electricity prices, partly due to the shift of the overall cost sensitivity from operation (fuel) costs to investment costs. However, cost minimization models for capacity expansion are frequently based on the principle that, for a perfectly adapted system allowing non-served energy, marginal remuneration allows overall operation and investments costs recovery. In addition, these models are usually formulated as finite-horizon problems when they should be theoretically solved for infinite horizons under the assumption of companies’ infinite lifespan, but infinite horizon cannot be dealt with mathematical programming since it requires finite sets. Previous approaches have tried to overcome this drawback with finite horizon models that tend asymptotically to the original infinite ones and, in many cases, the investment costs are annualized based on the plants’ lifespan, sometimes including a cost residual value. This paper proposes a novel approach with a finite horizon that guarantees the investment costs’ recovery. It is also able to obtain the marginal electricity costs of the original infinite horizon model, without the need for residual values or non-served energy. This new approach is especially suited for long-term electricity pricing with investments in renewable assets when non-served demand is banned or when no explicit capacity remuneration mechanisms are considered.

Holographic cosmology solutions of problems with pre-inflationary cosmology

In this paper we describe in detail how to solve the problems of pre-inflationary cosmology within the holographic cosmology model of McFadden and Skenderis [1]. The solutions of the smoothness and horizon problems, the flatness problem, the entropy and perturbation problems and the baryon asymmetry problem are shown, and the mechanisms for them complement the inflationary solutions. Most of the paper is devoted to the solution of the monopole relic problem, through a detailed calculation of 2-loop correlators of currents in a toy model which we perform in d dimensions, in order to extract its leading dependence on $$ {g}_{\mathrm{eff}}^2={g}_{\mathrm{YM}}^2N/q $$ g eff 2 = g YM 2 N / q and find a dilution effect. Taken together with the fact that holographic cosmology gives as good a fit to CMBR as the standard paradigm of Λ CDM with inflation, it means holographic cosmology extends the inflationary paradigm into new corners, explorable only through a dual perturbative field theory in 3 dimensions.

Solution of the size and horizon problems from classical string geometry

In a recent paper we developed a string cosmology background from classical string geometry. Here, we show that this background yields a solution to the size and horizon problems of Standard Big Bang cosmology while remaining compatible with the Transplanckian Censorship Conjecture. We also take a first look at the evolution of cosmological perturbations in this model.

ANALYZING EXISTING APPLIED MODELS OF THE IONOSPHERE FOR CALCULATING RADIO WAVE PROPAGATION AND POSSIBILITY OF THEIR USE FOR RADAR SYSTEMS. I. CLASSIFICATION OF APPLIED MODELS AND THE MAIN REQUIREMENTS IMPOSED ON THEM FOR RADAR AIDS

We review modern HF–X band radars covering over-the-horizon problems. The ionosphere significantly affects wave propagation in all the bands. We describe available correction techniques, which use additional evidence on the ionosphere, as well as models of different degrees of complexity. The fact that the field of view cannot be covered by ground-based instruments as well as the growing requirements to the precision and stability of the radars result in the impossibility of ionospheric correction with up-to-date models, hence the latter require further elaboration. We give a virtually full classification of the models. The article summarizes the requirements to the models for the radars depending on their function.

ANALYZING EXISTING APPLIED MODELS OF THE IONOSPHERE FOR CALCULATING RADIO WAVE PROPAGATION AND POSSIBILITY OF THEIR USE FOR RADAR SYSTEMS. I. CLASSIFICATION OF APPLIED MODELS AND THE MAIN REQUIREMENTS IMPOSED ON THEM FOR RADAR AIDS

We review modern HF–X band radars covering over-the-horizon problems. The ionosphere significantly affects wave propagation in all the bands. We describe available correction techniques, which use additional evidence on the ionosphere, as well as models of different degrees of complexity. The fact that the field of view cannot be covered by ground-based instruments as well as the growing requirements to the precision and stability of the radars result in the impossibility of ionospheric correction with up-to-date models, hence the latter require further elaboration. We give a virtually full classification of the models. The article summarizes the requirements to the models for the radars depending on their function.