scholarly journals 3. On negatively dependent sampling schemes, variance reduction, and probabilistic upper discrepancy bounds

2020 ◽  
pp. 43-68
Author(s):  
Cheng Zhang ◽  
Cengiz Öztireli ◽  
Stephan Mandt ◽  
Giampiero Salvi

The convergence speed of stochastic gradient descent (SGD) can be improved by actively selecting mini-batches. We explore sampling schemes where similar data points are less likely to be selected in the same mini-batch. In particular, we prove that such repulsive sampling schemes lower the variance of the gradient estimator. This generalizes recent work on using Determinantal Point Processes (DPPs) for mini-batch diversification (Zhang et al., 2017) to the broader class of repulsive point processes. We first show that the phenomenon of variance reduction by diversified sampling generalizes in particular to non-stationary point processes. We then show that other point processes may be computationally much more efficient than DPPs. In particular, we propose and investigate Poisson Disk sampling—frequently encountered in the computer graphics community—for this task. We show empirically that our approach improves over standard SGD both in terms of convergence speed as well as final model performance.


Methodology ◽  
2012 ◽  
Vol 8 (2) ◽  
pp. 71-80 ◽  
Author(s):  
Juan Botella ◽  
Manuel Suero

In Reliability Generalization (RG) meta-analyses, the importance of bearing in mind the problems of range restriction or biased sampling and their influence on reliability estimation has often been highlighted. Nevertheless, the presence of heterogeneous variances in the included studies has been diagnosed in a subjective way and has not been taken into account in later analyses. Procedures to detect the presence of a variety of sampling schemes and to manage them in the analyses are proposed. The procedures are further explained with an example, by applying them to 25 estimates of Cronbach’s alpha coefficient in the Hamilton Scale for Depression.


2018 ◽  
Vol 482 (6) ◽  
pp. 627-630
Author(s):  
D. Belomestny ◽  
◽  
L. Iosipoi ◽  
N. Zhivotovskiy ◽  
◽  
...  

2014 ◽  
Vol 22 (2) ◽  
pp. 217-224
Author(s):  
Houlong JIANG ◽  
Shuduan LIU ◽  
Anding XU ◽  
Chao YANG

2018 ◽  
Vol 2018 ◽  
pp. 1-15 ◽  
Author(s):  
Xichuan Liu ◽  
Taichang Gao ◽  
Yuntao Hu ◽  
Xiaojian Shu

In order to improve the measurement of precipitation microphysical characteristics sensor (PMCS), the sampling process of raindrops by PMCS based on a particle-by-particle Monte-Carlo model was simulated to discuss the effect of different bin sizes on DSD measurement, and the optimum sampling bin sizes for PMCS were proposed based on the simulation results. The simulation results of five sampling schemes of bin sizes in four rain-rate categories show that the raw capture DSD has a significant fluctuation variation influenced by the capture probability, whereas the appropriate sampling bin size and width can reduce the impact of variation of raindrop number on DSD shape. A field measurement of a PMCS, an OTT PARSIVEL disdrometer, and a tipping bucket rain Gauge shows that the rain-rate and rainfall accumulations have good consistencies between PMCS, OTT, and Gauge; the DSD obtained by PMCS and OTT has a good agreement; the probability of N0, μ, and Λ shows that there is a good agreement between the Gamma parameters of PMCS and OTT; the fitted μ-Λ and Z-R relationship measured by PMCS is close to that measured by OTT, which validates the performance of PMCS on rain-rate, rainfall accumulation, and DSD related parameters.


Sign in / Sign up

Export Citation Format

Share Document