Uncertainty-guided learning with scaled prediction errors in the basal ganglia

To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We provide a normative analysis, comparing the performance of the new model with that of conventional models in a value tracking task. We find that the new model has an advantage over conventional models when tested across various levels of observation noise. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. The scaled prediction error feedback signal is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and the update rules are found to be consistent with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.

LS-VCE Applied to Stochastic Modeling of GNSS Observation Noise and Process Noise

Stochastic models play a crucial role in global navigation satellite systems (GNSS) data processing. Many studies contribute to the stochastic modeling of GNSS observation noise, whereas few studies focus on the stochastic modeling of process noise. This paper proposes a method that is able to jointly estimate the variances of observation noise and process noise. The method is flexible since it is based on the least-squares variance component estimation (LS-VCE), enabling users to estimate the variance components that they are specifically interested in. We apply the proposed method to estimate the variances for the dual-frequency GNSS observation noise and for the process noise of the receiver code bias (RCB). We also investigate the impact of the stochastic model upon parameter estimation, ambiguity resolution, and positioning. The results show that the precision of GNSS observations differs in systems and frequencies. Among the dual-frequency GPS, Galileo, and BDS code observations, the precision of the BDS B3 observations is highest (better than 0.2 m). The precision of the BDS phase observations is better than two millimeters, which is also higher than that of the GPS and Galileo observations. For all three systems, the RCB process noise ranges from 0.5 millimeters to 1 millimeter, with a data sampling rate of 30 s. An improper stochastic model of the observation noise results in an unreliable ambiguity dilution of precision (ADOP) and position dilution of precision (PDOP), thus adversely affecting the assessment of the ambiguity resolution and positioning performance. An inappropriate stochastic model of RCB process noise disturbs the estimation of the receiver clock and the ionosphere delays and is thus harmful for timing and ionosphere retrieval applications.

Generalized Empirical Interpolation Method With H1 Regularization: Application to Nuclear Reactor Physics

The generalized empirical interpolation method (GEIM) can be used to estimate the physical field by combining observation data acquired from the physical system itself and a reduced model of the underlying physical system. In presence of observation noise, the estimation error of the GEIM is blurred even diverged. We propose to address this issue by imposing a smooth constraint, namely, to constrain the H1 semi-norm of the reconstructed field of the reduced model. The efficiency of the approach, which we will call the H1 regularization GEIM (R-GEIM), is illustrated by numerical experiments of a typical IAEA benchmark problem in nuclear reactor physics. A theoretical analysis of the proposed R-GEIM will be presented in future works.

Investigating errors when forecasting processes with uncertain dynamics and observation noise by the self-adjusting brown's zero-order model

This paper reports a study into the errors of process forecasting under the conditions of uncertainty in the dynamics and observation noise using a self-adjusting Brown's zero-order model. The dynamics test models have been built for predicted processes and observation noises, which make it possible to investigate forecasting errors for the self-adjusting and adaptive models. The test process dynamics were determined in the form of a rectangular video pulse with a fixed unit amplitude, a radio pulse of the harmonic process with an amplitude attenuated exponentially, as well as a video pulse with amplitude increasing exponentially. As a model of observation noise, an additive discrete Gaussian process with zero mean and variable value of the mean square deviation was considered. It was established that for small values of the mean square deviation of observation noise, a self-adjusting model under the conditions of dynamics uncertainty produces a smaller error in the process forecast. For the test jump-like dynamics of the process, the variance of the forecast error was less than 1 %. At the same time, for the adaptive model, with an adaptation parameter from the classical and beyond-the-limit sets, the variance of the error was about 20 % and 5 %, respectively. With significant observation noises, the variance of the error in the forecast of the test process dynamics for the self-adjusting and adaptive models with a parameter from the classical set was in the range from 1 % to 20 %. However, for the adaptive model, with a parameter from the beyond-the-limit set, the variance of the prediction error was close to 100 % for all test models. It was established that with an increase in the mean square deviation of observation noise, there is greater masking of the predicted test process dynamics, leading to an increase in the variance of the forecast error when using a self-adjusting model. This is the price for predicting processes with uncertain dynamics and observation noises.

Grid Lifespan Enlargement for Assessment in Multihop Wireless Detector Facilities

In energy limited wireless sensor networks, both local quantization andmultihop transmission are essential to save transmission energy and thus prolong the network lifetime. The goal is to maximize the network lifetime, defined as the estimation task cycles accomplished before the network becomes nonfunctional.The network lifetime optimization problem includes three components: Optimizing source coding at each sensor node, optimizing source throughput at each sensor node.Optimizing multihop routing path. Source coding optimization can be decoupled from source throughput and multihop routing path optimization and is solved by introducing a concept of equivalent 1-bit Mean Square Error (MSE) function. Based on optimal source coding, multihop routing path optimization is formulated as a linear programming problem, which suggests a new notion of character based routing. It is also seen that optimal multihop routing improves the network lifetime bound significantly compared with single-hop routing for heterogeneous networks. Furthermore, the gain is more significant when the network is denser since there are more opportunities for multihop routing. Also the gain is more significant when the observation noise variances are more diverse.

Correction Model of BDS-3 Satellite Pseudorange Multipath Delays and Its Impact on Single-Frequency Precise Point Positioning

Precise point positioning (PPP) is used in many fields. However, pseudorange multipath delay is an important error that restricts its accuracy. Pseudorange multipath delay can be considered as the combination of effective information and observation noise; it can be modeled after removing the observation noise. In this work, elastic nets (EN) regularization denoising method is proposed and compared with L2-norm regularization denoising method. Then, quadratic polynomial (QP) model plus autoregressive (AR) model (QP + AR) are used to model the denoised pseudorange multipath delays. Finally, the modeling results are corrected to the observations to verify the improvement of BDS-3 single-frequency PPP accuracy. Three single-frequency PPP schemes are designed to verify the effectiveness of denoising method and QP + AR model. The experimental results show that the accuracy improvement of B3I and B2a is more obvious than that of B1I and B1C when the modeling values are corrected to the pseudorange observations. The improvement of B3I and B2a in the east (E) and up (U) directions can reach 10.6%∼34.4% and 5.9%∼65.7%, and the improvement of the north (N) direction is mostly less than 10.0%. The accuracy of B1I and B1C in E and U directions can be improved by 0%∼30.7% and 0.4%∼28.6%, respectively, while the accuracy of N direction can be improved slightly or even decreased. Using EN regularization denoising and QP + AR model correction, single-frequency PPP performs better at B3I and B2a, while L2-norm regularization denoising and QP + AR model correction perform better at B1I and B1C. The accuracy improvement of B2a and B3I is more obvious than that of B1I and B1C. The convergence time after MP correction of each frequency is slightly shorter. Overall, the proposed pseudorange multipath delays processing strategy is beneficial in improving the single-frequency PPP of BDS-3 satellite.

Estimation of Vehicle Longitudinal Speed Based on Improved Kalman Filter

Estimation of vehicle longitudinal acceleration is very important in vehicle active safety control system. In this paper, two driving conditions of a 4WD off-road vehicle are divided by vehicle signals such as steering angle. Under different working conditions, different estimation algorithms are adopted. In the straight driving condition, the longitudinal speed was estimated by adjusting the variance weight of acceleration Kalman observation noise based on kinematics method. For steering conditions, in order to obtain the longitudinal velocity at the center of mass, by dynamic method, a lateral state estimator was designed and tire sideslip dynamics was modeled. The CarSim-Simulink co-simulation results show that the proposed algorithm has high accuracy and strong practicability.