scholarly journals Task-Agnostic Continual Learning Using Online Variational Bayes with Fixed-Point Updates

2021 ◽  
pp. 1-39
Author(s):  
Chen Zeno ◽  
Itay Golan ◽  
Elad Hoffer ◽  
Daniel Soudry
Keyword(s):  
Fixed Point ◽  
Optimization Problem ◽  
Large Body ◽  
Data Distribution ◽  
Variational Bayes ◽  
Learning Agents ◽  
Nonstationary Data ◽  
Learning Research ◽  
Do So ◽  
Continual Learning

Abstract Catastrophic forgetting is the notorious vulnerability of neural networks to the changes in the data distribution during learning. This phenomenon has long been considered a major obstacle for using learning agents in realistic continual learning settings. A large body of continual learning research assumes that task boundaries are known during training. However, only a few works consider scenarios in which task boundaries are unknown or not well defined: task-agnostic scenarios. The optimal Bayesian solution for this requires an intractable online Bayes update to the weights posterior. We aim to approximate the online Bayes update as accurately as possible. To do so, we derive novel fixed-point equations for the online variational Bayes optimization problem for multivariate gaussian parametric distributions. By iterating the posterior through these fixed-point equations, we obtain an algorithm (FOO-VB) for continual learning that can handle nonstationary data distribution using a fixed architecture and without using external memory (i.e., without access to previous data). We demonstrate that our method (FOO-VB) outperforms existing methods in task-agnostic scenarios. FOO-VB Pytorch implementation is available at https://github.com/chenzeno/FOO-VB.

A content-adaptive unstructured grid based regularized CT reconstruction method with a SART-type preconditioned fixed-point proximity algorithm

2022 ◽  
Author(s):  
Yun Chen ◽  
Yao Lu ◽  
Xiangyuan Ma ◽  
Yuesheng Xu
Keyword(s):  
Integral Equation ◽  
Fixed Point ◽  
Optimization Problem ◽  
Unstructured Grid ◽  
Reconstruction Technique ◽  
Reconstruction Method ◽  
Ct Reconstruction ◽  
Image Domain ◽  
Computational Costs ◽  
Content Adaptive

Abstract The goal of this study is to develop a new computed tomography (CT) image reconstruction method, aiming at improving the quality of the reconstructed images of existing methods while reducing computational costs. Existing CT reconstruction is modeled by pixel-based piecewise constant approximations of the integral equation that describes the CT projection data acquisition process. Using these approximations imposes a bottleneck model error and results in a discrete system of a large size. We propose to develop a content-adaptive unstructured grid (CAUG) based regularized CT reconstruction method to address these issues. Specifically, we design a CAUG of the image domain to sparsely represent the underlying image, and introduce a CAUG-based piecewise linear approximation of the integral equation by employing a collocation method. We further apply a regularization defined on the CAUG for the resulting illposed linear system, which may lead to a sparse linear representation for the underlying solution. The regularized CT reconstruction is formulated as a convex optimization problem, whose objective function consists of a weighted least square norm based fidelity term, a regularization term and a constraint term. Here, the corresponding weighted matrix is derived from the simultaneous algebraic reconstruction technique (SART). We then develop a SART-type preconditioned fixed-point proximity algorithm to solve the optimization problem. Convergence analysis is provided for the resulting iterative algorithm. Numerical experiments demonstrate the outperformance of the proposed method over several existing methods in terms of both suppressing noise and reducing computational costs. These methods include the SART without regularization and with quadratic regularization on the CAUG, the traditional total variation (TV) regularized reconstruction method and the TV superiorized conjugate gradient method on the pixel grid.

scholarly journals Spatiotemporally Constrained Action Space Attacks on Deep Reinforcement Learning Agents

2020 ◽  
Vol 34 (04) ◽  
pp. 4577-4584
Author(s):  
Xian Yeow Lee ◽  
Sambit Ghadai ◽  
Kai Liang Tan ◽  
Chinmay Hegde ◽  
Soumik Sarkar
Keyword(s):  
Reinforcement Learning ◽  
Optimization Problem ◽  
Limited Resource ◽  
Action Space ◽  
Engineering Systems ◽  
Physical Systems ◽  
Learning Agents ◽  
Look Ahead ◽  
Real World Applications ◽  
Temporal Dimensions

Robustness of Deep Reinforcement Learning (DRL) algorithms towards adversarial attacks in real world applications such as those deployed in cyber-physical systems (CPS) are of increasing concern. Numerous studies have investigated the mechanisms of attacks on the RL agent's state space. Nonetheless, attacks on the RL agent's action space (corresponding to actuators in engineering systems) are equally perverse, but such attacks are relatively less studied in the ML literature. In this work, we first frame the problem as an optimization problem of minimizing the cumulative reward of an RL agent with decoupled constraints as the budget of attack. We propose the white-box Myopic Action Space (MAS) attack algorithm that distributes the attacks across the action space dimensions. Next, we reformulate the optimization problem above with the same objective function, but with a temporally coupled constraint on the attack budget to take into account the approximated dynamics of the agent. This leads to the white-box Look-ahead Action Space (LAS) attack algorithm that distributes the attacks across the action and temporal dimensions. Our results showed that using the same amount of resources, the LAS attack deteriorates the agent's performance significantly more than the MAS attack. This reveals the possibility that with limited resource, an adversary can utilize the agent's dynamics to malevolently craft attacks that causes the agent to fail. Additionally, we leverage these attack strategies as a possible tool to gain insights on the potential vulnerabilities of DRL agents.

Information Technology Among U.S. Local Governments

2008 ◽  
pp. 132-144
Author(s):  
Donald F. Norris
Keyword(s):  
Information Technology ◽  
Local Government ◽  
Local Governments ◽  
Large Body ◽  
Electronic Government ◽  
The Subject ◽  
The 1950S ◽  
Do So

The purpose of this chapter is to provide an overview of the adoption, uses, and impacts of information technology (IT), including electronic government, among local governments in the United States1. In the 1950s, these governments began to adopt IT for a variety of purposes and functions, and they continue to do so today. Since at least the mid 1970s, a small, but prolific group of scholars has conducted a large body of research on various aspects of IT and local government.2 It is from that research and my own studies into this subject that I have based this chapter (regarding e-government, see also, Norris, 2006). Given the constraint of space, this chapter can only highlight aspects of this important topic. Readers who wish to delve more deeply into the subject of information technology and local government may wish to avail themselves of the works found in the bibliography as well as references from other, related works which can be found through those works.

scholarly journals Global-Entropy Driven Exploration with Distributed Models under Sparsity Constraints

2018 ◽  
Vol 8 (10) ◽  
pp. 1722 ◽  
Author(s):  
Christoph Manss ◽  
Dmitriy Shutin
Keyword(s):  
Optimization Problem ◽  
Data Distribution ◽  
Consensus Algorithm ◽  
Estimation Methods ◽  
Model Estimation ◽  
Individual Agent ◽  
Distributed Models ◽  
Heterogeneous Learning ◽  
Sparsity Constraints ◽  
The Individual

This paper focuses on exploration when using different data distribution schemes and ADMM as a solver for swarms. By exploration, we mean the estimation of new measurement locations that are beneficial for the model estimation. In particular, the different distribution schemes are splitting-over-features or heterogeneous learning and splitting-over-examples or homogeneous learning. Each agent contributes a solution to solve the joint optimization problem by using ADMM and the consensus algorithm. This paper shows that some information is unknown to the individual agent, and thus, the estimation of new measurement positions is not possible without further communication. Therefore, this paper shows results for how to distribute only necessary information for a global exploration. We show the benefits between the proposed global exploration scheme and benchmark exploration schemes such as random walk and systematic traversing, i.e., meandering. The proposed waypoint estimation methods are then tested against each other and with other movement methods. This paper shows that a movement method, which considers the current information within the model, is superior to the benchmark movement methods.

scholarly journals Constructing banking networks under decreasing costs of link formation

2021 ◽  
Author(s):  
Dietmar Maringer ◽  
Ben Craig ◽  
Sandra Paterlini
Keyword(s):  
Real World ◽  
Optimization Problem ◽  
Balance Sheet ◽  
Financial Networks ◽  
Fixed Costs ◽  
Balance Sheets ◽  
Search Heuristics ◽  
Banking Networks ◽  
Link Formation ◽  
Do So

AbstractThe structure of networks plays a central role in the behavior of financial systems and their response to policy. Real-world networks, however, are rarely directly observable: banks’ assets and liabilities are typically known, but not who is lending how much and to whom. This paper adds to the existing literature in two ways. First, it shows how to simulate realistic networks that are based on balance-sheet information. To do so, we introduce a model where links cause fixed-costs, independent of contract size; but the costs per link decrease the more connected a bank is (scale economies). Second, to approach the optimization problem, we develop a new algorithm inspired by the transportation planning literature and research in stochastic search heuristics. Computational experiments find that the resulting networks are not only consistent with the balance sheets, but also resemble real-world financial networks in their density (which is sparse but not minimally dense) and in their core-periphery and disassortative structure.

scholarly journals Split equality common null point problem for Bregman quasi-nonexpansive mappings

Filomat ◽  
2018 ◽  
Vol 32 (11) ◽  
pp. 3917-3932
Author(s):  
Ali Abkar ◽  
Elahe Shahrosvand
Keyword(s):  
Fixed Point ◽  
Equilibrium Problem ◽  
Banach Spaces ◽  
Optimization Problem ◽  
Nonexpansive Mappings ◽  
Infinite Family ◽  
Fixed Point Problem ◽  
Null Point ◽  
Point Problem ◽  
Reflexive Banach Spaces

In this paper, we introduce a new algorithm for solving the split equality common null point problem and the equality fixed point problem for an infinite family of Bregman quasi-nonexpansive mappings in reflexive Banach spaces. We then apply this algorithm to the equality equilibrium problem and the split equality optimization problem. In this way, we improve and generalize the results of Takahashi and Yao [22], Byrne et al [9], Dong et al [11], and Sitthithakerngkiet et al [21].

scholarly journals GENESIS – Generating e-learning systems in schools: School-university e-learning research partnerships for scaling up innovation

2021 ◽  
Author(s):  
Cathy Hill ◽  
Shirley Alexander ◽  
Karen Cuthbert ◽  
Robin Hall ◽  
Nerida McCredie ◽  
...  
Keyword(s):  
Learning Environment ◽  
Learning Environments ◽  
New Technologies ◽  
Scale Up ◽  
Scaling Up ◽  
Research Partnerships ◽  
E Learning ◽  
Learning Research ◽  
Lasting Change ◽  
Do So

Despite optimism about new technologies for learning, e-learning innovation has been slow to scale up. This research set out to investigate whether an e-learning environment designed by students themselves would scale easily in schools. To do so, the GENESIS Project, a collaborative undertaking between three schools and a University, created the opportunity for students as researchers to conceive, prototype, and test an e-learning environment in which they and other students could explore ideas of interest to them. Preliminary findings show that students are able to design e-learning environments that provide good contexts for learning. Furthermore, such an approach appears to set in motion deep and lasting change in schools in ways that align with Coburn’s (2003) four ways of thinking about scaling up. The Project demonstrates, in all its phases, a way in which students themselves can take information leadership of curriculum in a culture of change.

scholarly journals Convergence of a distributed method for minimizing sum of convex functions with fixed point constraints

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Nawarat Ekkarntrong ◽  
Tipsuda Arunrat ◽  
Nimit Nimana
Keyword(s):  
Fixed Point ◽  
Convergence Rate ◽  
Numerical Experiment ◽  
Theoretical Result ◽  
Convex Functions ◽  
Optimization Problem ◽  
Distributed Optimization ◽  
Distributed Method

AbstractIn this paper, we consider a distributed optimization problem of minimizing sum of convex functions over the intersection of fixed-point constraints. We propose a distributed method for solving the problem. We prove the convergence of the generated sequence to the solution of the problem under certain assumption. We further discuss the convergence rate with an appropriate positive stepsize. A numerical experiment is given to show the effectiveness of the obtained theoretical result.

scholarly journals An approximation algorithm for multi-objective optimization problems using a box-coverage

2021 ◽  
Author(s):  
Gabriele Eichfelder ◽  
Leo Warnow
Keyword(s):  
Approximation Algorithm ◽  
Optimization Problem ◽  
Optimization Problems ◽  
Multi Objective Optimization ◽  
Multi Objective ◽  
Nondominated Set ◽  
Common Technique ◽  
Nondominated Points ◽  
Nondominated Point ◽  
Do So

AbstractFor a continuous multi-objective optimization problem, it is usually not a practical approach to compute all its nondominated points because there are infinitely many of them. For this reason, a typical approach is to compute an approximation of the nondominated set. A common technique for this approach is to generate a polyhedron which contains the nondominated set. However, often these approximations are used for further evaluations. For those applications a polyhedron is a structure that is not easy to handle. In this paper, we introduce an approximation with a simpler structure respecting the natural ordering. In particular, we compute a box-coverage of the nondominated set. To do so, we use an approach that, in general, allows us to update not only one but several boxes whenever a new nondominated point is found. The algorithm is guaranteed to stop with a finite number of boxes, each being sufficiently thin.

Sign in / Sign up

Export Citation Format

Share Document