Self-Adaptive Priority Correction for Prioritized Experience Replay

Deep Reinforcement Learning (DRL) is a promising approach for general artificial intelligence. However, most DRL methods suffer from the problem of data inefficiency. To alleviate this problem, DeepMind proposed Prioritized Experience Replay (PER). Though PER improves data utilization, the priorities of most samples in its Experience Memory (EM) are out of date, as only the priorities of a small part of the data are updated while the Q network parameters are updated. Consequently, the difference between storage and real priority distributions gradually increases, which will introduce bias into the gradients of Deep Q-Learning (DQL) and make the DQL update toward a non-ideal direction. In this work, we propose a novel self-adaptive priority correction algorithm named Importance-PER (Imp-PER) to fix the update deviation. Specifically, we predict the sum of real Temporal-Difference error (TD-error) of all data in EM. Data are corrected by an importance weight, which is estimated by the predicted sum and the real TD-error calculated by the latest agent. To control the unbounded importance weight, we use truncated importance sampling with a self-adaptive truncation threshold. The conducted experiments on various games of Atari 2600 with Double Deep Q-Network and MuJoCo with Deep Deterministic Policy Gradient demonstrate that Imp-PER improves the data utilization and final policy quality on discrete states and continuous states tasks without increasing the computational cost.

A radio burst detection method based on the Hough transform

ABSTRACT We present a simple and fast method for incoherent dedispersion and fast radio burst (FRB) detection based on the Hough transform, which is widely used for feature extraction in image analysis. The Hough transform maps a point in the time–frequency data to a straight line in the parameter space and points on the same dispersed f−2 curve to a bundle of lines all crossing at the same point, thus the curve is transformed to a single point in the parameter space, enabling an easier way for the detection of radio burst. By choosing an appropriate truncation threshold, in a reasonably radio quiet environment, i.e. with radio frequency interferences present but not dominant, the computing speed of the method is very fast. Using simulation data of different noise levels, we studied how the detected peak varies with different truncation thresholds. We also tested the method with some real pulsar and FRB data.

What about the relevance of the diffusion parameters identified in the case of incomplete Fickian and non-Fickian kinetics?

The aim of this work is to study the relevance of the diffusion parameters identified on kinetics whose saturation levels are unknown. Two types of diffusion kinetics have been considered: a Fickian and a non-Fickian kinetics. Numerical experiments, based on the generation of noisy data corresponding to reference values of diffusion parameters, have been carried out. The two models used in this study are Fick's model and the «Dual-Fick» model. Identification procedures were then applied on these data sets, truncated at different critical times. In the specific case of Dual Fick kinetics, the classical least square method was compared to an alternative approach based on the piecewise analysis of the shape of the diffusion curves. The identified parameters may deviate significantly from the expected reference values when the truncation threshold of the data set precedes the steady state. Also, these inaccurate parameters impact the predictions of global water uptake, water concentration profiles and stresses gradients through the thickness of the sample.

A Truncation Model for Estimating Species Richness

We propose a truncation model for the abundance distribution in species richness estimation. This model is inherently semiparametric and incorporates an unknown truncation threshold between rare and abundant observations. Using the conditional likelihood, we derive a class of estimators for the parameters in this model by stepwise maximization. The species richness estimator is given by the integer maximizing the binomial likelihood, given all other parameters in the model. Under regularity conditions, we show that our estimators of the model parameters are asymptotically efficient. We recover Chaos lower bound estimator of species richness when the parametric part of the model is single-component Poisson. Thus our class of estimators strictly generalized the latter. We illustrate the performance of the proposed method in a simulation study, and compare it favorably to other widely-used estimators. We also give an application to estimating the number of distinct vocabulary words in French playwright Molière’s Tartuffe.

Channel Estimation methods with low complexity for 3GPP/LTE

International audience OFDM based pilots channel estimation methods with processing into the transform domain appear attractive owing to their capacity to highly reduce the noise component effect. However, in current OFDM systems, null subcarriers are placed at the edge of the spectrum in order to assure isolation from interfering signals in neighboring frequency bands; and the presence of these null carriers may lead, if not taken into account, to serious degradation of the estimated channel responses due to the “border effect” phenomenon. In this paper an improved algorithm based on truncated SVD is proposed in order to correctly support the case of null carriers at border spectrum. A method for optimizing the truncation threshold whatever the system parameters is also proposed. To make the truncated SVD channel estimation method applicable to any SISO or MIMO OFDM system and whatever the system parameters, a complexity reduction algorithm based on the distribution of the power in the transfer matrix (based on DFT or DCT) is proposed. Les techniques d’estimation de canal, basée sur des symboles pilotes, par passage dansun domaine de tranfert sont très attractives pour les systèmes de télécommunications utilisant l’OFDM.Cependant, elles montrent des limites pour les systèmes de télécommunications, où un ensemble desous-porteuses de garde, est inséré sur les bords du spectre dans le but d’éviter tout recouvrementspectral avec d’autres applications utilisant des bandes voisines. Ces sous-porteuses de gardeont selon leur nombre tendance à dégrader fortement les performances de ces estimateurs. Nousproposons, dans un premier temps, une optimisation qui permet d’améliorer considérablement lesperformances de ces estimateurs quels que soit le nombre de porteuses de garde. Dans un secondtemps, pour de rendre l’estimateur proposé attractif pour les constructeurs, nous avons proposé unetechnique permettant de réduire leur complexité de réalisation de manière notable.