SigmoReLU: An Improvement Activation Function by Combining Sigmoid and ReLU

Two of the most common activation functions (AF) in deep neural networks (DNN) training are Sigmoid and ReLU. Sigmoid was tend to be more popular the previous decades, but it was suffering with the common vanishing gradient problems. ReLU has resolved these problems by using zero gradient and not tiny values for negative weights and the value “1” for all positives. Although it significant resolves the vanishing of the gradients, it poses new issues with dying neurons of the zero values. Recent approaches for improvements are in a similar direction by just proposing variations of the AF, such as Leaky ReLU (LReLU), while maintaining the solution within the same unresolved gradient problems. In this paper, the combining of the Sigmoid and ReLU in one single function is proposed, as a way to take the advantages of the two. The experimental results demonstrate that by using the ReLU’s gradient solution on positive weights, and Sigmoid’s gradient solution on negatives, has a significant improvement on performance of training Neural Networks on image classification of diseases such as COVID-19, text and tabular data classification tasks on five different datasets.

Stellar scattering and the formation of exponential discs in self-gravitating systems

ABSTRACT We show, using the N-body code gadget-2, that stellar scattering by massive clumps can produce exponential discs, and the effectiveness of the process depends on the mass of scattering centres, as well as the stability of the galactic disc. Heavy, dense scattering centres in a less stable disc generate an exponential profile quickly, with a time-scale shorter than 1 Gyr. The profile evolution due to scattering can make a near-exponential disc under various initial stellar distributions. This result supports analytic theories that predict the scattering processes always favour the zero entropy gradient solution to the Jeans/Poisson equations, whose profile is a near-exponential. Profile changes are accompanied by disc thickening, and a power-law increase in stellar velocity dispersion in both vertical and radial directions is also observed through the evolution. Close encounters between stars and clumps can produce abrupt changes in stellar orbits and shift stars radially. These events can make trajectories more eccentric, but many leave eccentricities little changed. On average, orbital eccentricities of stars increase moderately with time.

Performance evaluation of the maximum complex correntropy criterion with adaptive kernel width update

The complex correntropy is a recently defined similarity measure that extends the advantages of conventional correntropy to complex-valued data. As in the real-valued case, the maximum complex correntropy criterion (MCCC) employs a free parameter called kernel width, which affects the convergence rate, robustness, and steady-state performance of the method. However, determining the optimal value for such parameter is not always a trivial task. Within this context, several works have introduced adaptive kernel width algorithms to deal with this free parameter, but such solutions must be updated to manipulate complex-valued data. This work reviews and updates the most recent adaptive kernel width algorithms so that they become capable of dealing with complex-valued data using the complex correntropy. Besides that, a novel gradient-based solution is introduced to the Gaussian kernel and its respective convergence analysis. Simulations compare the performance of adaptive kernel width algorithms with different fixed kernel sizes in an impulsive noise environment. The results show that the iterative kernel adjustment improves the performance of the gradient solution for complex-valued data.

Interaction between an edge dislocation near a circular void within the framework of the theory of strain gradient elasticity

The purpose of this paper was to consider an edge dislocation near a circular hole within the isotropic theory of gradient elasticity. The stress field is derived with the help of a stress function method. The gradient stresses possess no singularity at the dislocation line. As a result, the image force exerted on the dislocation due to the presence of the hole remains finite when the dislocation approaches the interface. The gradient solution demonstrates a non-classical size effect.

Learning to Design Games: Strategic Environments in Reinforcement Learning

In typical reinforcement learning (RL), the environment is assumed given and the goal of the learning is to identify an optimal policy for the agent taking actions through its interactions with the environment. In this paper, we extend this setting by considering the environment is not given, but controllable and learnable through its interaction with the agent at the same time. This extension is motivated by environment design scenarios in the real-world, including game design, shopping space design and traffic signal design. Theoretically, we find a dual Markov decision process (MDP) w.r.t. the environment to that w.r.t. the agent, and derive a policy gradient solution to optimizing the parametrized environment. Furthermore, discontinuous environments are addressed by a proposed general generative framework. Our experiments on a Maze game design task show the effectiveness of the proposed algorithms in generating diverse and challenging Mazes against various agent settings.

Graphics Algorithm for Deriving Atmospheric Boundary Layer Heights from CALIPSO Data

The atmospheric boundary layer is an important atmospheric feature that affects environmental health and weather forecasting. In this study, we proposed a graphics algorithm for the derivation of atmospheric boundary layer height (BLH) from the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) data. Owing to the differences in scattering intensity between molecular and aerosol particles, the total attenuated backscatter coefficient 532 and attenuated backscatter coefficient 1064 were used simultaneously for BLH detection. The proposed algorithm transformed the gradient solution into graphics distribution solution to overcome the effects of large noise and improve the horizontal resolution. This method was then tested with real signals under different horizontal smoothing numbers (1, 3, 15 and 30). The algorithm provided a reliable result when the horizontal smoothing number was greater than 5. Finally, the results of BLH obtained by CALIPSO data were compared with the results retrieved by the ground-based Lidar and radiosonde (RS) measurements. Under the horizontal smoothing number of 15, 9 and 3, the correlation coefficients between the BLH derived by the proposed algorithm and ground-based Lidar were 0.72, 0.72 and 0.14, respectively, and those between the BLH derived by the proposed algorithm and radiosonde measurements were 0.59, 0.59 and 0.07. When the horizontal smoothing number was 15 and 9, the CALIPSO BLH derived by the proposed method demonstrated a good correlation with ground-based Lidar and RS. This finding indicated that the proposed algorithm can be applied to the CALIPSO satellite data with 3 and 5 km horizontal resolution.