Temporal Consistency-Based Loss Function for Both Deep Q-Networks and Deep Deterministic Policy Gradients for Continuous Actions

Artificial intelligence (AI) techniques in power grid control and energy management in building automation require both deep Q-networks (DQNs) and deep deterministic policy gradients (DDPGs) in deep reinforcement learning (DRL) as off-policy algorithms. Most studies on improving the stability of DRL have addressed these with replay buffers and a target network using a delayed temporal difference (TD) backup, which is known for minimizing a loss function at every iteration. The loss functions were developed for DQN and DDPG, and it is well-known that there have been few studies on improving the techniques of the loss functions used in both DQN and DDPG. Therefore, we modified the loss function based on a temporal consistency (TC) loss and adapted the proposed TC loss function for the target network update in both DQN and DDPG. The proposed TC loss function showed effective results, particularly in a critic network in DDPG. In this work, we demonstrate that, in OpenAI Gym, both “cart-pole” and “pendulum”, the proposed TC loss function shows enormously improved convergence speed and performance, particularly in the critic network in DDPG.

Parallel generation of extensive vascular networks with application to an archetypal human kidney model

Given the relevance of the inextricable coupling between microcirculation and physiology, and the relation to organ function and disease progression, the construction of synthetic vascular networks for mathematical modelling and computer simulation is becoming an increasingly broad field of research. Building vascular networks that mimic in vivo morphometry is feasible through algorithms such as constrained constructive optimization (CCO) and variations. Nevertheless, these methods are limited by the maximum number of vessels to be generated due to the whole network update required at each vessel addition. In this work, we propose a CCO-based approach endowed with a domain decomposition strategy to concurrently create vascular networks. The performance of this approach is evaluated by analysing the agreement with the sequentially generated networks and studying the scalability when building vascular networks up to 200 000 vascular segments. Finally, we apply our method to vascularize a highly complex geometry corresponding to the cortex of a prototypical human kidney. The technique presented in this work enables the automatic generation of extensive vascular networks, removing the limitation from previous works. Thus, we can extend vascular networks (e.g. obtained from medical images) to pre-arteriolar level, yielding patient-specific whole-organ vascular models with an unprecedented level of detail.

Exemplifying Stratified Deforestation in Four Protected Areas in Madagascar

Protected areas (PAs) are a cornerstone for conservation biodiversity. Madagascar, as a hotspot for biodiversity, has a network of 114 terrestrial protected areas covering the main forest types occurring on the island. Deforestation continues unabated despite the network covering 11% of the island. Here we present a case study approach reporting on four PAs from the humid forests, dry western forests, and southwestern dry and spiny forests and thickets. To describe deforestation in and around the case sites, we have considered a time window of 30 years for analysis, focusing on six years with reliable data: 1990, 2000, 2010, 2015 (the year of latest PA network update), and 2017. We have considered forest versus other land covers within the PAs in “buffers” at a distance of 500 m, 2.5 km, 5 km, and 10 km from the border of the PA. These buffers were set from the border towards the center of the PA (inside the PAs) and from the border outside the PAs. The smallest PAs, Kasijy (IUCN IV), and Behara Tranomaro (no IUCN category), showed the least forest loss. Tsaratanana (IUCN I) had the highest deforestation rates within the last two years of analysis, with deforestation concentrated in the core area. Ranobe PK-32 (no IUCN category), originally with the largest forest extent, has lost most of its forest cover and showed the highest annual deforestation rate (3.5%) between 2015 and 2017. All four cases prove to be very challenging to manage. Future conservation activities require tailored interventions to account for site-specific current and potential future threats, as detailed in this contribution.

Latte

Emerging software-defined and programmable networking technologies enable more adaptive communication infrastructures. However, leveraging these flexibilities and operating networks more adaptively is challenging, as the underlying infrastructure remains a complex distributed system that is a subject to delays, and as consistency properties need to be preserved transiently, even during network reconfiguration. Motivated by these challenges, we propose Latte, an automated approach to minimize the latency of network update schedules by avoiding unnecessary waiting times and exploiting concurrency, while at the same time provably ensuring a wide range of fundamental consistency properties like waypoint enforcement. To enable automated reasoning about the performance and consistency of software-defined networks during an update, we introduce the model of timed-arc colored Petri nets: an extension of Petri nets which allows us to account for time aspects in asynchronous networks, including characteristic timing behaviors, modeled as timed and colored tokens. This novel formalism may be of independent interest. Latte relies on an efficient translation of specific network update problems into timed-arc colored Petri nets. We show that the constructed nets can be analyzed efficiently via their unfolding into existing timed-arc Petri nets. We integrate Latte into the state-of-the-art model checking tool TAPAAL, and find that in many cases, we are able to reduce the latency of network updates by 90% or more.

GRAPPA Patient Research Partner Network: Update to the GRAPPA 2020 Annual Meeting

Eleven Patient Research Partners (PRPs) attended the virtual Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA) 2020 Annual Meeting. PRPs fully participated in the panel discussion at the 2020 GRAPPA Annual Meeting on the topic of the coronavirus disease 2019 (COVID-19; caused by SARS-CoV-2). The members of the PRP group have been involved in many GRAPPA projects over the last year, including work on the GRAPPA–Outcome Measures in Rheumatology (OMERACT) Core Set, GRAPPA’s 2020 Treatment Guidelines update project, and GRAPPA’s Collaborative Research Network project.