Jointly Learning Environments and Control Policies with Projected Stochastic Gradient Ascent

We consider the joint design and control of discrete-time stochastic dynamical systems over a finite time horizon. We formulate the problem as a multi-step optimization problem under uncertainty seeking to identify a system design and a control policy that jointly maximize the expected sum of rewards collected over the time horizon considered. The transition function, the reward function and the policy are all parametrized, assumed known and differentiable with respect to their parameters. We then introduce a deep reinforcement learning algorithm combining policy gradient methods with model-based optimization techniques to solve this problem. In essence, our algorithm iteratively approximates the gradient of the expected return via Monte-Carlo sampling and automatic differentiation and takes projected gradient ascent steps in the space of environment and policy parameters. This algorithm is referred to as Direct Environment and Policy Search (DEPS). We assess the performance of our algorithm in three environments concerned with the design and control of a mass-spring-damper system, a small-scale off-grid power system and a drone, respectively. In addition, our algorithm is benchmarked against a state-of-the-art deep reinforcement learning algorithm used to tackle joint design and control problems. We show that DEPS performs at least as well or better in all three environments, consistently yielding solutions with higher returns in fewer iterations. Finally, solutions produced by our algorithm are also compared with solutions produced by an algorithm that does not jointly optimize environment and policy parameters, highlighting the fact that higher returns can be achieved when joint optimization is performed.

Bolted Joint Preload Distribution from Torque Tightening

The purpose of this paper is to develop an understanding of how bolt preloads are distributed within a joint as each bolt is tightened in turn by the use of a calibrated torque wrench. It discusses how the order that the joints nuts/bolts are tightened can affect the final bolt preload. It also investigates the effect on incrementally increasing the bolt preload through a series of applications of the controlled torque tightening sequence. Classical analysis methods are used to develop a method of analysis that can be applied to most preloaded bolted joints. It is assumed that the static friction coefficient is approximately 15% less than the dynamic friction. It is found that the bolt preload distribution across the joint can range from slightly above the target preload to significantly less than the target preload. The bolts with a preload greater than the target preload are found to be those tightened towards the end of the tightening sequence, usually located close to the outer edges of the joint’s bolt array. The bolts with a preload less than the target preload are those tightened early in the tightening sequence, located centrally within the joints bolt array. The methods presented can be used to optimise bolted joint design and assembly procedures. Optimising the design of preloaded bolted joints leads to more efficient use of the joints.

Structural Means for Fire-Safe Wooden Façade Design

This study investigates five fire stop variants used to limit the spread of fire on wooden façades. For this purpose, five fire tests using various types of wooden façade claddings and different fire stops were conducted as full-scale tests and compared to the existing findings. The influences and interactions between the material qualities of the external wall behind the façade cladding, the construction type of the wooden façade cladding, the design of the substructure, the depth of the ventilation gap, and the design of the fire stops were investigated. In evaluating the fire stops, the design of the interior corners, the joint design, and the influence of thermal expansion were examined. Finally, design proposals for the design of fire stops at wooden façades in order to limit the spread of fire were derived based on this evaluation. The outlook presents further needs that need to be investigated in the future in order to clarify undiscussed aspects or points that were ultimately not evaluated within the scope of this study. Graphical Abstract