Developing the Capability to Scale and Reuse Learned Knowledge and Functionality in Learning Classifier Systems
<p>Learning is an important activity through which humanity has incrementally improved accomplishing tasks by adapting knowledge and methods based on the related feedback. Although learning is natural to humans, it is much more difficult to achieve in the technological world as tasks are often learned in isolation. Software is capable of learning novel techniques and algorithms in order to solve these basic, individual problems, however transferring said knowledge to other problems in the same or related domains presents challenges. Solutions often cannot be enumerated to discover the best one as many problems of interest can be intractable in terms of the resources needed to successfully complete them. However, many such problems contain key building blocks of knowledge that can be leveraged to achieve a suitable solution. These building blocks encapsulate important structural regularities of the problem. A technique that can learn these regularities without enumeration,may produce general solutions that apply to similar problems of any length. This implies reusing learned information. In order to reuse learned blocks of knowledge, it is important that a program be scalable and flexible. This requires a program capable of taking knowledge from a previous task and applying it to a more complex problem or a problem with a similar pattern. This is anticipated to enable the program to complete the new task in a practical amount of time and with reasonable amounts of resources. In machine learning, the degree of human intervention in solving problems is often important in many tasks. It is generally necessary for a human to provide input to direct and improve learning. In the field of Developmental Learning there is the idea known as the Threshold Concept (TC). A TC is transformative information which advocates learning. TCs are important because without them, the learner cannot progress. In addition, TCs need to be learned in a particular order, much like a curriculum, thus providing the student with viable progress towards learning more difficult ideas at a faster pace than otherwise. Therefore, human input to a learning algorithm can be to partition a problem into constituent subproblems. This is a principal concept of Layered Learning (LL),where a sequence of sub-problems are learned. The sub-problems are self-contained stages which have been separated by a human. This technique is necessary for tasks in which learning a direct mapping from inputs to outputs is intractable given existing learning algorithms. One of the first artificial learning systems developed is Learning Classifier Systems (LCSs). Past work has extended LCSs to provide more expressivity by using richer representations. One such representation is tree-based and is common to the Genetic Programming (GP) technique. GP is part of the Evolutionary Computation (EC) paradigm and produces solutions represented by trees. The tree nodes can contain functions, and leaf nodes problem features, giving GP a rich representation. A more recent technique is Code Fragments (CFs). CFs are GP-like sub-trees with an initial maximum height of two. Initially, CFs contained hard-coded functions at the root nodes and problem features or previously learned CFs at the leaf nodes of the sub-trees. CFs provided improved expressivity and scalability over the original ternary alphabet used by LCSs. Additionally, CF-based systems have successfully learned previously intractable problems, e.g. 135-bit multiplexer. Although CFs have provided increased scalability, they suffer from a structural weakness. As the problem scales, the chains of CFs grow to intractable lengths. This means that at some point the LCS will stop learning. In addition, CFs were originally meant to scale to more complex problems in the same domain. However, it is advantageous to compile cross-domain solutions, as the regularities of a problem might be from different domains to that expressed by the data. The proposed thesis is that a CF-based LCS can scale to complex problems by reusing learned solutions of problems as functions at the inner nodes of CFs together with compaction and Layered Learning. The overall goal is divided into the following three sub-goals: reuse learned functionality from smaller problems in the root nodes of CF sub-trees, identify a compaction technique that facilitates reduced solution size for improved evaluation time of CFs and develop a layered learning methodology for a CF system, which will be demonstrated by learning a general solution to an intractable problem, i.e. n-bit Multiplexer. In this novel work, Code Fragments are extended to include learned functionality at the root nodes of the sub-trees in a technique known as XCSCF². A new compaction technique is designed, which produces an equivalent set of ternary rules from CF rules. This technique is known as XCSCF3. The work culminates with a new technique XCSCF*, which combines Layered Learning, Code Fragments and Transfer Learning (TL) of knowledge and functionality to produce scalable and general solutions, i.e. to the n-bit multiplexer problem. The novel ideas are tested with the multiplexer and hidden multiplexer problems. These problems are chosen because they are difficult due to epistasis, sparsity and non-linearity. Therefore they provide ample opportunity for testing the new contributions. The thesis work has shown that CFs can be used in various ways to increase scalability and to discover solutions to complex problems. Specifically the following three contributions were produced: learned functionality was captured in LCS populations from smaller problems and was reused in the root nodes of CF sub-trees. An online compaction technique that facilitates reduced evaluation time of CFs was designed. A layered learning method to train a CF system in a manner leading to a general solution was developed. This was demonstrated through learning a solution to a previously intractable problem, i.e. the n-bit Multiplexer. The thesis concludes with suggestions for future work aimed at providing better scalability when using compaction techniques.</p>
