Paying for lifelong learning: problems and possibilities

In many real-world sequential decision making problems, the number of available actions (decisions) can vary over time. While problems like catastrophic forgetting, changing transition dynamics, changing rewards functions, etc. have been well-studied in the lifelong learning literature, the setting where the size of the action set changes remains unaddressed. In this paper, we present first steps towards developing an algorithm that autonomously adapts to an action set whose size changes over time. To tackle this open problem, we break it into two problems that can be solved iteratively: inferring the underlying, unknown, structure in the space of actions and optimizing a policy that leverages this structure. We demonstrate the efficiency of this approach on large-scale real-world lifelong learning problems.

Download Full-text

The role of assessment in preparing for lifelong learning: Problems and challenges

Balancing Dilemmas in Assessment and Learning in Contemporary Education ◽

10.4324/9780203942185-14 ◽

2007 ◽

pp. 97-110

Keyword(s):

Lifelong Learning ◽

Learning Problems

Download Full-text

Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer

Journal of Artificial Intelligence Research ◽

10.1613/jair.1.11304 ◽

2020 ◽

Vol 67 ◽

pp. 673-704

Author(s):

Mohammad Rostami ◽

David Isele ◽

Eric Eaton

Keyword(s):

Lifelong Learning ◽

Accurate Estimate ◽

Training Data ◽

Learning Problems ◽

Theoretical Justification ◽

Lifelong Learner ◽

Task Descriptions ◽

Improved Performance ◽

High Level ◽

Empirical Demonstration

Knowledge transfer between tasks can improve the performance of learned models, but requires an accurate estimate of inter-task relationships to identify the relevant knowledge to transfer. These inter-task relationships are typically estimated based on training data for each task, which is inefficient in lifelong learning settings where the goal is to learn each consecutive task rapidly from as little data as possible. To reduce this burden, we develop a lifelong learning method based on coupled dictionary learning that utilizes high-level task descriptions to model inter-task relationships. We show that using task descriptors improves the performance of the learned task policies, providing both theoretical justification for the benefit and empirical demonstration of the improvement across a variety of learning problems. Given only the descriptor for a new task, the lifelong learner is also able to accurately predict a model for the new task through zero-shot learning using the coupled dictionary, eliminating the need to gather training data before addressing the task.

Download Full-text