Towards Tangible Cultural Heritage Experiences—Enriching VR-based Object Inspection with Haptic Feedback

VR/AR technology is a key enabler for new ways of immersively experiencing cultural heritage artifacts based on their virtual counterparts obtained from a digitization process. In this article, we focus on enriching VR-based object inspection by additional haptic feedback, thereby creating tangible cultural heritage experiences. For this purpose, we present an approach for interactive and collaborative VR-based object inspection and annotation. Our system supports high-quality 3D models with accurate reflectance characteristics while additionally providing haptic feedback regarding shape features of the object based on a 3D printed replica. The digital object model in terms of a printable representation of the geometry as well as reflectance characteristics are stored in a compact and streamable representation on a central server, which streams the data to remotely connected users/clients. The latter can jointly perform an interactive inspection of the object in VR with additional haptic feedback through the 3D printed replica. Evaluations regarding system performance, visual quality of the considered models, as well as insights from a user study indicate an improved interaction, assessment, and experience of the considered objects.

Shape variability of the central sulcus in the developing brain: a longitudinal descriptive and predictive study in preterm infants.

Despite growing evidence of links between sulcation and function in the adult brain, the folding dynamics, occurring mostly before normal-term-birth, is vastly unknown. Looking into the development of cortical sulci in babies can give us keys to address fundamental questions: what is the sulcal shape variability in the developing brain? When are the shape features encoded? How are these morphological parameters related to further functional development? In this study, we aimed to investigate the shape variability of the developing central sulcus, which is the frontier between the primary somatosensory and motor cortices. We studied a cohort of 71 extremely preterm infants scanned twice using MRI - once around 30 weeks post-menstrual age (w PMA) and once at term-equivalent age, around 40w PMA -, in order to quantify the sulcus's shape variability using manifold learning, regardless of age-group or hemisphere. We then used these shape descriptors to evaluate the sulcus's variability at both ages and to assess hemispheric and age-group specificities. This led us to propose a description of ten shape features capturing the variability in the central sulcus of preterm infants. Our results suggested that most of these features (8/10) are encoded as early as 30w PMA. We unprecedentedly observed hemispheric asymmetries at both ages, and the one captured at term-equivalent age seems to correspond with the asymmetry pattern previously reported in adults. We further trained classifiers in order to explore the predictive value of these shape features on manual performance at 5 years of age (handedness and fine motor outcome). The central sulcus's shape alone showed a limited but relevant predictive capacity in both cases. The study of sulcal shape features during early neurodevelopment may participate to a better comprehension of the complex links between morphological and functional organization of the developing brain.

Time Series Classification using Shape Features based on Angle Statistics

Séries temporais possuem grande aplicabilidade nos mais diversos cenários, incluindo os domínios científicos, agrícola, econômico, entre outros. Portanto, criar representações efetivas de uma série temporal é uma tarefa desafiadora, pois possibilita análises mais precisas e, consequentemente, obtenção de resultados e conclusões mais assertivas em diversas tarefas de aprendizado de máquina. Uma das principais tarefas associadas é a classificação, que pode ser realizada a partir de diferentes representações computacionais das séries temporais. Este trabalho tem como principal objetivo melhorar a eficácia de tarefas de classificação, utilizando uma representação das séries temporais obtida pelo algoritmo Beam Angle Statistics, um extrator de características de contorno baseado em estatísticas angulares.

Genetic Programming for Supervised Figure-ground Image Segmentation

<p>Figure-ground segmentation is a process of separating regions of interest from unimportant backgrounds. It is essential to various applications in computer vi- sion and image processing, e.g. object tracking and image editing, as they are only interested in certain regions of an image and use figure-ground segmenta- tion as a pre-processing step. Traditional figure-ground segmentation methods often require heavy human workload (e.g. ground truth labeling), and/or rely heavily on human guidance (e.g. locating an initial model), accordingly cannot easily adapt to diverse image domains. Evolutionary computation (EC) is a family of algorithms for global optimi- sation, which are inspired by biological evolution. As an EC technique, genetic programming (GP) can evolve algorithms automatically for complex problems without pre-defining solution models. Compared with other EC techniques, GP is more flexible as it can utilise complex and variable-length representations (e.g. trees) of candidate solutions. It is hypothesised that this flexibility of GP makes it possible to evolve better solutions than those designed by experts. However, there have been limited attempts at applying GP to figure-ground segmentation. In this thesis, GP is enabled to successfully address figure-ground segmentation through evolving well-performing segmentors and generating effective features. The objectives are to investigate various image features as inputs of GP, develop multi-objective approaches, develop feature selection/construction methods, and conduct further evaluations of the proposed GP methods. The following new methods have been developed. Effective terminal sets of GP are investigated for figure-ground segmentation, covering three general types of image features, i.e. colour/brightness, texture and shape features. Results show that texture features are more effective than intensities and shape features as they are discriminative for different materials that foreground and background regions normally belong to (e.g. metal or wood). Two new multi-objective GP methods are proposed to evolve figure-ground segmentors, aiming at producing solutions balanced between the segmentation performance and solution complexity. Compared with a reference method that does not consider complexity and a parsimony pressure based method (a popular bloat control technique), the proposed methods can significantly reduce the solution size while achieving similar segmentation performance based on the Mann- Whitney U-Test at the significance level 5%. GP is introduced for the first time to conduct feature selection for figure- ground segmentation tasks, aiming to maximise the segmentation performance and minimise the number of selected features. The proposed methods produce feature subsets that lead to solutions achieving better segmentation performance with lower features than those of two benchmark methods (i.e. sequential forward selection and sequential backward selection) and the original full feature set. This is due to GP’s high search ability and higher likelihood of finding the global optima. GP is introduced for the first time to construct high-level features from primitive image features, which aims to improve the image segmentation performance, especially on complex images. By considering linear/non-linear interactions of the original features, the proposed methods construct fewer features that achieve better segmentation performance than the original full feature set. This investigation has shown that GP is suited for figure-ground image segmentation for the following reasons. Firstly, the proposed methods can evolve segmentors with useful class characteristic patterns to segment various types of objects. Secondly, the segmentors evolved from one type of foreground object can generalise well on similar objects. Thirdly, both the selected and constructed features of the proposed GP methods are more effective than original features, with the selected/constructed features being better for subsequent tasks. Finally, compared with other segmentation techniques, the major strengths of GP are that it does not require pre-defined problem models, and can be easily adapted to diverse image domains without major parameter tuning or human intervention.</p>

Genetic Programming for Supervised Figure-ground Image Segmentation

<p>Figure-ground segmentation is a process of separating regions of interest from unimportant backgrounds. It is essential to various applications in computer vi- sion and image processing, e.g. object tracking and image editing, as they are only interested in certain regions of an image and use figure-ground segmenta- tion as a pre-processing step. Traditional figure-ground segmentation methods often require heavy human workload (e.g. ground truth labeling), and/or rely heavily on human guidance (e.g. locating an initial model), accordingly cannot easily adapt to diverse image domains. Evolutionary computation (EC) is a family of algorithms for global optimi- sation, which are inspired by biological evolution. As an EC technique, genetic programming (GP) can evolve algorithms automatically for complex problems without pre-defining solution models. Compared with other EC techniques, GP is more flexible as it can utilise complex and variable-length representations (e.g. trees) of candidate solutions. It is hypothesised that this flexibility of GP makes it possible to evolve better solutions than those designed by experts. However, there have been limited attempts at applying GP to figure-ground segmentation. In this thesis, GP is enabled to successfully address figure-ground segmentation through evolving well-performing segmentors and generating effective features. The objectives are to investigate various image features as inputs of GP, develop multi-objective approaches, develop feature selection/construction methods, and conduct further evaluations of the proposed GP methods. The following new methods have been developed. Effective terminal sets of GP are investigated for figure-ground segmentation, covering three general types of image features, i.e. colour/brightness, texture and shape features. Results show that texture features are more effective than intensities and shape features as they are discriminative for different materials that foreground and background regions normally belong to (e.g. metal or wood). Two new multi-objective GP methods are proposed to evolve figure-ground segmentors, aiming at producing solutions balanced between the segmentation performance and solution complexity. Compared with a reference method that does not consider complexity and a parsimony pressure based method (a popular bloat control technique), the proposed methods can significantly reduce the solution size while achieving similar segmentation performance based on the Mann- Whitney U-Test at the significance level 5%. GP is introduced for the first time to conduct feature selection for figure- ground segmentation tasks, aiming to maximise the segmentation performance and minimise the number of selected features. The proposed methods produce feature subsets that lead to solutions achieving better segmentation performance with lower features than those of two benchmark methods (i.e. sequential forward selection and sequential backward selection) and the original full feature set. This is due to GP’s high search ability and higher likelihood of finding the global optima. GP is introduced for the first time to construct high-level features from primitive image features, which aims to improve the image segmentation performance, especially on complex images. By considering linear/non-linear interactions of the original features, the proposed methods construct fewer features that achieve better segmentation performance than the original full feature set. This investigation has shown that GP is suited for figure-ground image segmentation for the following reasons. Firstly, the proposed methods can evolve segmentors with useful class characteristic patterns to segment various types of objects. Secondly, the segmentors evolved from one type of foreground object can generalise well on similar objects. Thirdly, both the selected and constructed features of the proposed GP methods are more effective than original features, with the selected/constructed features being better for subsequent tasks. Finally, compared with other segmentation techniques, the major strengths of GP are that it does not require pre-defined problem models, and can be easily adapted to diverse image domains without major parameter tuning or human intervention.</p>

Evaluation of conventional and deep learning based image harmonization methods in radiomics studies

Objective: To evaluate the impact of image harmonization on outcome prediction models using radiomics. Approach: 234 patients from the Brain Tumor Image Segmentation Benchmark (BRATS) dataset with T1 MRI were enrolled in this study. Images were harmonized through a reference image using histogram matching (HHM) and a generative adversarial network (GAN)-based method (HGAN). 88 radiomics features were extracted on HHM, HGAN and original (HNONE) images. Wilcoxon paired test was used to identify features significantly impacted by the harmonization protocol used. Radiomic prediction models were built using feature selection with the Least Absolute Shrinkage and Selection Operator (LASSO) and Kaplan-Meier analysis. Main results: More than 50% of the features (49/88) were statistically modified by the harmonization with HHM and 55 with HGAN (adjusted p-value < 0.05). The contribution of histogram and texture features selected by the LASSO, in comparison to shape features that were not impacted by harmonization, was higher in harmonized datasets (47% for Hnone, 62% for HHM and 71% for HGAN). Both image-based harmonization methods allowed to split patients into two groups with significantly different survival (p<0.05). With the HGAN images, we were also able to build and validate a model using only features impacted by the harmonization (median survivals of 189 vs. 437 days, p=0.006) Significance: Data harmonization in a multi-institutional cohort allows to recover the predictive value of some radiomics features that was lost due to differences in the image properties across centers. In terms of ability to build survival prediction models in the BRATS dataset, the loss of power from impacted histogram and heterogeneity features was compensated by the selection of additional shape features. The harmonization using a GAN-based approach outperformed the histogram matching technique, supporting the interest for the development of new advanced harmonization techniques for radiomic analysis purposes.