On stochastic Kaczmarz type methods for solving large scale systems of ill-posed equations

Inverse Problems ◽

10.1088/1361-6420/ac3f80 ◽

2021 ◽

Author(s):

Joel Rabelo ◽

Yuri Saporito ◽

Antonio Leitao

Keyword(s):

Large Scale ◽

Preliminary Investigation ◽

Real Data ◽

Approximate Solutions ◽

Superior Performance ◽

Regression Problem ◽

Promising Alternative ◽

Large Scale Systems ◽

Stochastic Version ◽

Ill Posed

Abstract In this article we investigate a family of "stochastic gradient type methods", for solving systems of linear ill-posed equations. The method under consideration is a stochastic version of the projective Landweber-Kaczmarz (PLWK) method in [Leitão/Svaiter, Inv. Probl. 2016] (see also [Leitão/Svaiter, NFAO 2018]). In the case of exact data, mean square convergence to zero of the iteration error is proven. In the noise data case, we couple our method with an a priori stopping rule and characterize it as a regularization method for solving systems of linear ill-posed operator equations. Numerical tests are presented for two linear ill-posed problems: (i) a Hilbert matrix type system with over 10^8 equations; (ii) a Big Data linear regression problem with real data. The obtained results indicate superior performance of the proposed method when compared with other well established iterations. Our preliminary investigation indicates that the proposed iteration is a promising alternative for computing stable approximate solutions of large scale systems of linear ill-posed equations.

Download Full-text

RoCGAN: Robust Conditional GAN

International Journal of Computer Vision ◽

10.1007/s11263-020-01348-5 ◽

2020 ◽

Vol 128 (10-11) ◽

pp. 2665-2683 ◽

Cited By ~ 1

Author(s):

Grigorios G. Chrysos ◽

Jean Kossaifi ◽

Stefanos Zafeiriou

Keyword(s):

Large Scale ◽

Real Data ◽

Superior Performance ◽

Target Space ◽

Generative Adversarial Networks ◽

Natural Scenes ◽

Adversarial Networks ◽

Target Manifold ◽

The Face ◽

Intense Noise

Abstract Conditional image generation lies at the heart of computer vision and conditional generative adversarial networks (cGAN) have recently become the method of choice for this task, owing to their superior performance. The focus so far has largely been on performance improvement, with little effort in making cGANs more robust to noise. However, the regression (of the generator) might lead to arbitrarily large errors in the output, which makes cGANs unreliable for real-world applications. In this work, we introduce a novel conditional GAN model, called RoCGAN, which leverages structure in the target space of the model to address the issue. Specifically, we augment the generator with an unsupervised pathway, which promotes the outputs of the generator to span the target manifold, even in the presence of intense noise. We prove that RoCGAN share similar theoretical properties as GAN and establish with both synthetic and real data the merits of our model. We perform a thorough experimental validation on large scale datasets for natural scenes and faces and observe that our model outperforms existing cGAN architectures by a large margin. We also empirically demonstrate the performance of our approach in the face of two types of noise (adversarial and Bernoulli).

Download Full-text

Bayesian Image Restoration Using a Large-Scale Total Patch Variation Prior

Mathematical Problems in Engineering ◽

10.1155/2011/408241 ◽

2011 ◽

Vol 2011 ◽

pp. 1-15 ◽

Cited By ~ 1

Author(s):

Yang Chen ◽

Weimin Yu ◽

Yinsheng Li ◽

Zhou Yang ◽

Limin Luo ◽

...

Keyword(s):

Image Restoration ◽

Large Scale ◽

Computational Cost ◽

Real Data ◽

Inverse Image ◽

Edge Preserving ◽

Device Architecture ◽

Mixture Prior ◽

Ill Posed ◽

Improved Performance

Edge-preserving Bayesian restorations using nonquadratic priors are often inefficient in restoring continuous variations and tend to produce block artifacts around edges in ill-posed inverse image restorations. To overcome this, we have proposed a spatial adaptive (SA) prior with improved performance. However, this SA prior restoration suffers from high computational cost and the unguaranteed convergence problem. Concerning these issues, this paper proposes a Large-scale Total Patch Variation (LS-TPV) Prior model for Bayesian image restoration. In this model, the prior for each pixel is defined as a singleton conditional probability, which is in a mixture prior form of one patch similarity prior and one weight entropy prior. A joint MAP estimation is thus built to ensure the iteration monotonicity. The intensive calculation of patch distances is greatly alleviated by the parallelization of Compute Unified Device Architecture(CUDA). Experiments with both simulated and real data validate the good performance of the proposed restoration.

Download Full-text

Multidimensional recursive filter preconditioning in geophysical estimation problems

Geophysics ◽

10.1190/1.1567228 ◽

2003 ◽

Vol 68 (2) ◽

pp. 577-588 ◽

Cited By ~ 46

Author(s):

Sergey Fomel ◽

Jon F. Claerbout

Keyword(s):

Large Scale ◽

Model Space ◽

Real Data ◽

Efficiency Gain ◽

Recursive Filtering ◽

Interpolation Problems ◽

Estimation Problems ◽

Order Of Magnitude ◽

Ill Posed ◽

Regularized Optimization

Constraining ill‐posed inverse problems often requires regularized optimization. We consider two alternative approaches to regularization. The first approach involves a column operator and an extension of the data space. It requires a regularization operator which enhances the undesirable features of the model. The second approach constructs a row operator and expands the model space. It employs a preconditioning operator which enforces a desirable behavior (such as smoothness) of the model. In large‐scale problems, when iterative optimization is incomplete, the second method is preferable, because it often leads to faster convergence. We propose a method for constructing preconditioning operators by multidimensional recursive filtering. The recursive filters are constructed by imposing helical boundary conditions. Several examples with synthetic and real data demonstrate an order of magnitude efficiency gain achieved by applying the proposed technique to data interpolation problems.

Download Full-text

Learning Inverse Depth Regression for Multi-View Stereo with Correlation Cost Volume

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6939 ◽

2020 ◽

Vol 34 (07) ◽

pp. 12508-12515

Author(s):

Qingshan Xu ◽

Wenbing Tao

Keyword(s):

Large Scale ◽

Stereo Matching ◽

Classification Problem ◽

Superior Performance ◽

Regression Problem ◽

Average Group ◽

Depth Classification ◽

Effective Cost ◽

Depth Inference ◽

The Cost

Deep learning has shown to be effective for depth inference in multi-view stereo (MVS). However, the scalability and accuracy still remain an open problem in this domain. This can be attributed to the memory-consuming cost volume representation and inappropriate depth inference. Inspired by the group-wise correlation in stereo matching, we propose an average group-wise correlation similarity measure to construct a lightweight cost volume. This can not only reduce the memory consumption but also reduce the computational burden in the cost volume filtering. Based on our effective cost volume representation, we propose a cascade 3D U-Net module to regularize the cost volume to further boost the performance. Unlike the previous methods that treat multi-view depth inference as a depth regression problem or an inverse depth classification problem, we recast multi-view depth inference as an inverse depth regression task. This allows our network to achieve sub-pixel estimation and be applicable to large-scale scenes. Through extensive experiments on DTU dataset and Tanks and Temples dataset, we show that our proposed network with Correlation cost volume and Inverse DEpth Regression (CIDER1), achieves state-of-the-art results, demonstrating its superior performance on scalability and accuracy.

Download Full-text