Genetic Programming for Automatically Synthesising Robust Image Descriptors with A Small Number of Instances

<p>Image classification is a core task in many applications of computer vision, including object detection and recognition. It aims at analysing the visual content and automatically categorising a set of images into different groups. Performing image classification can largely be affected by the features used to perform this task. Extracting features from images is a challenging task due to the large search space size and practical requirements such as domain knowledge and human intervention. Human intervention is usually needed to identify a good set of keypoints (regions of interest), design a set of features to be extracted from those keypoints such as lines and corners, and develop a way to extract those features. Automating these tasks has great potential to dramatically decrease the time and cost, and may potentially improve the performance of the classification task. There are two well-recognised approaches in the literature to automate the processes of identifying keypoints and extracting image features. Designing a set of domain-independent features is the first approach, where the focus is on dividing the image into a number of predefined regions and extracting features from those regions. The second approach is synthesising a function or a set of functions to form an image descriptor that aims at automatically detecting a set of keypoints such as lines and corners, and performing feature extraction. Although employing image descriptors is more effective and very popular in the literature, designing those descriptors is a difficult task that in most cases requires domain-expert intervention. The overall goal of this thesis is to develop a new domain independent Genetic Programming (GP) approach to image classification by utilising GP to evolve programs that are capable of automatically detecting diverse and informative keypoints, designing a set of features, and performing feature extraction using only a small number of training instances to facilitate image classification, and are robust to different image changes such as illumination and rotation. This thesis focuses on incorporating a variety of simple arithmetic operators and first-order statistics (mid-level features) into the evolutionary process and on representation of GP to evolve programs that are robust to image changes for image classification. This thesis proposes methods for domain-independent binary classification in images using GP to automatically identify regions within an image that have the potential to improve classification while considering the limitation of having a small training set. Experimental results show that in over 67% of cases the new methods significantly outperform the use of existing hand-crafted features and features automatically detected by other methods. This thesis proposes the first GP approach for automatically evolving an illumination-invariant dense image descriptor that detects automatically designed keypoints, and performs feature extraction using only a few instances of each class. The experimental results show improvement of 86% on average compared to two GP-based methods, and can significantly outperform domain-expert hand-crafted descriptors in more than 89% of the cases. This thesis also considers rotation variation of images and proposes a method for automatically evolving rotation-invariant image descriptors through integrating a set of first-order statistics as terminals. Compared to hand-crafted descriptors, the experimental results reveal that the proposed method has significantly better performance in more than 83% of the cases. This thesis proposes a new GP representation that allows the system to automatically choose the length of the feature vector side-by-side with evolving an image descriptor. Automatically determining the length of the feature vector helps to reduce the number of the parameters to be set. The results show that this method has evolved descriptors with a very small feature vector which yet still significantly outperform the competitive methods in more than 91% of the cases. This thesis proposes a method for transfer learning by model in GP, where an image descriptor evolved on instances of a related problem (source domain) is applied directly to solve a problem being tackled (target domain). The results show that the new method evolves image descriptors that have better generalisability compared to hand-crafted image descriptors. Those automatically evolved descriptors show positive influence on classifying the target domain datasets in more than 56% of the cases.</p>

Genetic Programming for Automatically Synthesising Robust Image Descriptors with A Small Number of Instances

<p>Image classification is a core task in many applications of computer vision, including object detection and recognition. It aims at analysing the visual content and automatically categorising a set of images into different groups. Performing image classification can largely be affected by the features used to perform this task. Extracting features from images is a challenging task due to the large search space size and practical requirements such as domain knowledge and human intervention. Human intervention is usually needed to identify a good set of keypoints (regions of interest), design a set of features to be extracted from those keypoints such as lines and corners, and develop a way to extract those features. Automating these tasks has great potential to dramatically decrease the time and cost, and may potentially improve the performance of the classification task. There are two well-recognised approaches in the literature to automate the processes of identifying keypoints and extracting image features. Designing a set of domain-independent features is the first approach, where the focus is on dividing the image into a number of predefined regions and extracting features from those regions. The second approach is synthesising a function or a set of functions to form an image descriptor that aims at automatically detecting a set of keypoints such as lines and corners, and performing feature extraction. Although employing image descriptors is more effective and very popular in the literature, designing those descriptors is a difficult task that in most cases requires domain-expert intervention. The overall goal of this thesis is to develop a new domain independent Genetic Programming (GP) approach to image classification by utilising GP to evolve programs that are capable of automatically detecting diverse and informative keypoints, designing a set of features, and performing feature extraction using only a small number of training instances to facilitate image classification, and are robust to different image changes such as illumination and rotation. This thesis focuses on incorporating a variety of simple arithmetic operators and first-order statistics (mid-level features) into the evolutionary process and on representation of GP to evolve programs that are robust to image changes for image classification. This thesis proposes methods for domain-independent binary classification in images using GP to automatically identify regions within an image that have the potential to improve classification while considering the limitation of having a small training set. Experimental results show that in over 67% of cases the new methods significantly outperform the use of existing hand-crafted features and features automatically detected by other methods. This thesis proposes the first GP approach for automatically evolving an illumination-invariant dense image descriptor that detects automatically designed keypoints, and performs feature extraction using only a few instances of each class. The experimental results show improvement of 86% on average compared to two GP-based methods, and can significantly outperform domain-expert hand-crafted descriptors in more than 89% of the cases. This thesis also considers rotation variation of images and proposes a method for automatically evolving rotation-invariant image descriptors through integrating a set of first-order statistics as terminals. Compared to hand-crafted descriptors, the experimental results reveal that the proposed method has significantly better performance in more than 83% of the cases. This thesis proposes a new GP representation that allows the system to automatically choose the length of the feature vector side-by-side with evolving an image descriptor. Automatically determining the length of the feature vector helps to reduce the number of the parameters to be set. The results show that this method has evolved descriptors with a very small feature vector which yet still significantly outperform the competitive methods in more than 91% of the cases. This thesis proposes a method for transfer learning by model in GP, where an image descriptor evolved on instances of a related problem (source domain) is applied directly to solve a problem being tackled (target domain). The results show that the new method evolves image descriptors that have better generalisability compared to hand-crafted image descriptors. Those automatically evolved descriptors show positive influence on classifying the target domain datasets in more than 56% of the cases.</p>

An Automated CAD System for Accurate Grading of Uveitis Using Optical Coherence Tomography Images

Uveitis is one of the leading causes of severe vision loss that can lead to blindness worldwide. Clinical records show that early and accurate detection of vitreous inflammation can potentially reduce the blindness rate. In this paper, a novel framework is proposed for automatic quantification of the vitreous on optical coherence tomography (OCT) with particular application for use in the grading of vitreous inflammation. The proposed pipeline consists of two stages, vitreous region segmentation followed by a neural network classifier. In the first stage, the vitreous region is automatically segmented using a U-net convolutional neural network (U-CNN). For the input of U-CNN, we utilized three novel image descriptors to account for the visual appearance similarity of the vitreous region and other tissues. Namely, we developed an adaptive appearance-based approach that utilizes a prior shape information, which consisted of a labeled dataset of the manually segmented images. This image descriptor is adaptively updated during segmentation and is integrated with the original greyscale image and a distance map image descriptor to construct an input fused image for the U-net segmentation stage. In the second stage, a fully connected neural network (FCNN) is proposed as a classifier to assess the vitreous inflammation severity. To achieve this task, a novel discriminatory feature of the segmented vitreous region is extracted. Namely, the signal intensities of the vitreous are represented by a cumulative distribution function (CDF). The constructed CDFs are then used to train and test the FCNN classifier for grading (grade from 0 to 3). The performance of the proposed pipeline is evaluated on a dataset of 200 OCT images. Our segmentation approach documented a higher performance than related methods, as evidenced by the Dice coefficient of 0.988 ± 0.01 and Hausdorff distance of 0.0003 mm ± 0.001 mm. On the other hand, the FCNN classification is evidenced by its average accuracy of 86%, which supports the benefits of the proposed pipeline as an aid for early and objective diagnosis of uvea inflammation.

Bangla language textual image description by hybrid neural network model

Automatic image captioning task in different language is a challenging task which has not been well investigated yet due to the lack of dataset and effective models. It also requires good understanding of scene and contextual embedding for robust semantic interpretation of images for natural language image descriptor. To generate image descriptor in Bangla, we created a new Bangla dataset of images paired with target language label, named as Bangla Natural Language Image to Text (BNLIT) dataset. To deal with the image understanding, we propose a hybrid encoder-decoder model based on encoder-decoder architecture and the model is evaluated on our newly created dataset. This proposed approach achieves significance performance improvement on task of semantic retrieval of images. Our hybrid model uses the Convolutional Neural<br />Network as an encoder whereas the Bidirectional Long Short Term Memory is used for the sentence representation that decreases the computational complexities without trading off the exactness of the descriptor. The model yielded benchmark accuracy in recovering Bangla natural language and we also conducted a thorough numerical analysis of the model performance on the BNLIT dataset.

Automatically Evolving Texture Image Descriptors using the Multi-tree Representation in Genetic Programming using Few Instances

The performance of image classification is highly dependent on the quality of the extracted features that are used to build a model. Designing such features usually requires prior knowledge of the domain and is often undertaken by a domain expert who, if available, is very costly to employ. Automating the process of designing such features can largely reduce the cost and efforts associated with this task. Image descriptors, such as local binary patterns, have emerged in computer vision, and aim at detecting keypoints, e.g., corners, line-segments and shapes, in an image and extracting features from those keypoints. In this paper, genetic programming (GP) is used to automatically evolve an image descriptor using only two instances per class by utilising a multi-tree program representation. The automatically evolved descriptor operates directly on the raw pixel values of an image and generates the corresponding feature vector. Seven well-known datasets were adapted to the few-shot setting and used to assess the performance of the proposed method and compared against six hand-crafted and one evolutionary computation-based image descriptor as well as three convolutional neural network (CNN) based methods. The experimental results show that the new method has significantly outperformed the competitor image descriptors and CNN-based methods. Furthermore, different patterns have been identified from analysing the evolved programs.