Virtual sacristy: the interaction with the heritage

Digital technologies contribute to access, knowledge and interaction with the historical heritage. From this stimulus, the user gets to know the patrimonial asset, as well as tends to visit, appreciate and preserve it. This may help to promote tourism and strengthen local culture. This article aims to present different forms of interaction with heritage through remote and virtual communication resources. The patrimonial asset analysed is a convent named Santo Antônio do Cairu, located at the municipality of Cairu, in the state of Bahia, Brazil. This study presents a virtual visit to the Convent through photographic panoramas. The lavabo is highlighted, which has been digitally reconstituted by Dense Stereo Matching (DSM) technique. The results are available for visualization and interaction in Virtual Reality (VR) or Augmented Reality (AR). From these resources, a website has been created for the user to easily interact with the virtual elements and to access the information about the historical and architectural details of the sacristy. This content, available on the website https://conventodecairu.wixsite.com/sacristia, seeks to facilitate the access to scientific information and to contribute to the diffusion of the knowledge regarding the convent amongst the general public and the specialists in the field.

LEARNING MULTI-MODAL FEATURES FOR DENSE MATCHING-BASED CONFIDENCE ESTIMATION

In recent years, the ability to assess the uncertainty of depth estimates in the context of dense stereo matching has received increased attention due to its potential to detect erroneous estimates. Especially, the introduction of deep learning approaches greatly improved general performance, with feature extraction from multiple modalities proving to be highly advantageous due to the unique and different characteristics of each modality. However, most work in the literature focuses on using only mono- or bi- or rarely tri-modal input, not considering the potential effectiveness of modalities, going beyond tri-modality. To further advance the idea of combining different types of features for confidence estimation, in this work, a CNN-based approach is proposed, exploiting uncertainty cues from up to four modalities. For this purpose, a state-of-the-art local-global approach is used as baseline and extended accordingly. Additionally, a novel disparity-based modality named warped difference is presented to support uncertainty estimation at common failure cases of dense stereo matching. The general validity and improved performance of the proposed approach is demonstrated and compared against the bi-modal baseline in an evaluation on three datasets using two common dense stereo matching techniques.

MIXED PROBABILITY MODELS FOR ALEATORIC UNCERTAINTY ESTIMATION IN THE CONTEXT OF DENSE STEREO MATCHING

The ability to identify erroneous depth estimates is of fundamental interest. Information regarding the aleatoric uncertainty of depth estimates can be, for example, used to support the process of depth reconstruction itself. Consequently, various methods for the estimation of aleatoric uncertainty in the context of dense stereo matching have been presented in recent years, with deep learning-based approaches being particularly popular. Among these deep learning-based methods, probabilistic strategies are increasingly attracting interest, because the estimated uncertainty can be quantified in pixels or in metric units due to the consideration of real error distributions. However, existing probabilistic methods usually assume a unimodal distribution to describe the error distribution while simply neglecting cases in real-world scenarios that could violate this assumption. To overcome this limitation, we propose two novel mixed probability models consisting of Laplacian and Uniform distributions for the task of aleatoric uncertainty estimation. In this way, we explicitly address commonly challenging regions in the context of dense stereo matching and outlier measurements, respectively. To allow a fair comparison, we adapt a common neural network architecture to investigate the effects of the different uncertainty models. In an extensive evaluation using two datasets and two common dense stereo matching methods, the proposed methods demonstrate state-of-the-art accuracy.