A Novel Approach for Stereo-Matching Based on Feature Correspondence

Abstract. We present a novel approach for dense 3-D cloud reconstruction above an area of 10 × 10 km2 using two hemispheric sky imagers with fisheye lenses in a stereo setup. We examine an epipolar rectification model designed for fisheye cameras, which allows the use of efficient out-of-the-box dense matching algorithms designed for classical pinhole-type cameras to search for correspondence information at every pixel. The resulting dense point cloud allows to recover a detailed and more complete cloud morphology compared to previous approaches that employed sparse feature-based stereo or assumed geometric constraints on the cloud field. Our approach is very efficient and can be fully automated. From the obtained 3-D shapes, cloud dynamics, size, motion, type and spacing can be derived, and used for radiation closure under cloudy conditions, for example. Fisheye lenses follow a different projection function than classical pinhole-type cameras and provide a large field of view with a single image. However, the computation of dense 3-D information is more complicated and standard implementations for dense 3-D stereo reconstruction cannot be easily applied. Together with an appropriate camera calibration, which includes internal camera geometry, global position and orientation of the stereo camera pair, we use the correspondence information from the stereo matching for dense 3-D stereo reconstruction of clouds located around the cameras. We implement and evaluate the proposed approach using real world data and present two case studies. In the first case, we validate the quality and accuracy of the method by comparing the stereo reconstruction of a stratocumulus layer with reflectivity observations measured by a cloud radar and the cloud-base height estimated from a Lidar-ceilometer. The second case analyzes a rapid cumulus evolution in the presence of strong wind shear.

Download Full-text

Equal Baseline Camera Array—Calibration, Testbed and Applications

Applied Sciences ◽

10.3390/app11188464 ◽

2021 ◽

Vol 11 (18) ◽

pp. 8464

Author(s):

Adam L. Kaczmarek ◽

Bernhard Blaschitz

Keyword(s):

Stereo Matching ◽

Outdoor Environment ◽

Learning Technology ◽

Disparity Map ◽

Stereo Camera ◽

Camera Array ◽

3D Data ◽

Novel Approach ◽

Calibration Methods ◽

Wide Range

This paper presents research on 3D scanning by taking advantage of a camera array consisting of up to five adjacent cameras. Such an array makes it possible to make a disparity map with a higher precision than a stereo camera, however it preserves the advantages of a stereo camera such as a possibility to operate in wide range of distances and in highly illuminated areas. In an outdoor environment, the array is a competitive alternative to other 3D imaging equipment such as Structured-light 3D scanners or Light Detection and Ranging (LIDAR). The considered kinds of arrays are called Equal Baseline Camera Array (EBCA). This paper presents a novel approach to calibrating the array based on the use of self-calibration methods. This paper also introduces a testbed which makes it possible to develop new algorithms for obtaining 3D data from images taken by the array. The testbed was released under open-source. Moreover, this paper shows new results of using these arrays with different stereo matching algorithms including an algorithm based on a convolutional neural network and deep learning technology.

Download Full-text

Seamless 3D Image Mapping and Mosaicing of Valles Marineris on Mars Using Orbital HRSC Stereo and Panchromatic Images

Remote Sensing ◽

10.3390/rs13071385 ◽

2021 ◽

Vol 13 (7) ◽

pp. 1385

Author(s):

Yu Tao ◽

Greg Michael ◽

Jan-Peter Muller ◽

Susan J. Conway ◽

Alfiah R. D. Putri

Keyword(s):

Stereo Matching ◽

3D Models ◽

Grid Spacing ◽

Laser Altimeter ◽

Terrain Model ◽

Valles Marineris ◽

Digital Terrain ◽

3D Terrain ◽

Spline Fitting ◽

Novel Approach

A seamless mosaic has been constructed including a 3D terrain model at 50 m grid-spacing and a corresponding terrain-corrected orthoimage at 12.5 m using a novel approach applied to ESA Mars Express High Resolution Stereo Camera orbital (HRSC) images of Mars. This method consists of blending and harmonising 3D models and normalising reflectance to a global albedo map. Eleven HRSC image sets were processed to Digital Terrain Models (DTM) based on an opensource stereo photogrammetric package called CASP-GO and merged with 71 published DTMs from the HRSC team. In order to achieve high quality and complete DTM coverage, a new method was developed to combine data derived from different stereo matching approaches to achieve a uniform outcome. This new approach was developed for high-accuracy data fusion of different DTMs at dissimilar grid-spacing and provenance which employs joint 3D and image co-registration, and B-spline fitting against the global Mars Orbiter Laser Altimeter (MOLA) standard reference. Each HRSC strip is normalised against a global albedo map to ensure that the very different lighting conditions could be corrected and resulting in a tiled set of seamless mosaics. The final 3D terrain model is compared against the MOLA height reference and the results shown of this intercomparison both in altitude and planum. Visualisation and access mechanisms to the final open access products are described.

Download Full-text

Hierarchical Guided-Image-Filtering for Efficient Stereo Matching

Applied Sciences ◽

10.3390/app9153122 ◽

2019 ◽

Vol 9 (15) ◽

pp. 3122 ◽

Cited By ~ 1

Author(s):

Chengtao Zhu ◽

Yau-Zen Chang

Keyword(s):

Stereo Matching ◽

Image Filtering ◽

Data Sets ◽

Matching Accuracy ◽

Edge Preserving ◽

Novel Approach ◽

Combination Scheme ◽

Image Pairs ◽

Guided Image Filtering ◽

The Cost

Stereo matching is complicated by the uneven distribution of textures on the image pairs. We address this problem by applying the edge-preserving guided-Image-filtering (GIF) at different resolutions. In contrast to most multi-scale stereo matching algorithms, parameters of the proposed hierarchical GIF model are in an innovative weighted-combination scheme to generate an improved matching cost volume. Our method draws its strength from exploiting texture in various resolution levels and performing an effective mixture of the derived parameters. This novel approach advances our recently proposed algorithm, the pervasive guided-image-filtering scheme, by equipping it with hierarchical filtering modules, leading to disparity images with more details. The approach ensures as many different-scale patterns as possible to be involved in the cost aggregation and hence improves matching accuracy. The experimental results show that the proposed scheme achieves the best matching accuracy when compared with six well-recognized cutting-edge algorithms using version 3 of the Middlebury stereo evaluation data sets.

Download Full-text

MULTI-FEATURE MUTUAL INFORMATION IMAGE REGISTRATION

Image Analysis & Stereology ◽

10.5566/ias.v31.p43-53 ◽

2012 ◽

Vol 31 (1) ◽

pp. 43 ◽

Cited By ~ 10

Author(s):

Dejan Tomaževič ◽

Boštjan Likar ◽

Franjo Pernuš

Keyword(s):

Image Registration ◽

Mutual Information ◽

Spatial Information ◽

Similarity Measures ◽

Feature Space ◽

Efficient Estimation ◽

Mr Images ◽

Feature Correspondence ◽

Information Theoretic ◽

Novel Approach

Nowadays, information-theoretic similarity measures, especially the mutual information and its derivatives, are one of the most frequently used measures of global intensity feature correspondence in image registration. Because the traditional mutual information similarity measure ignores the dependency of intensity values of neighboring image elements, registration based on mutual information is not robust in cases of low global intensity correspondence. Robustness can be improved by adding spatial information in the form of local intensity changes to the global intensity correspondence. This paper presents a novel method, by which intensities, together with spatial information, i.e., relations between neighboring image elements in the form of intensity gradients, are included in information-theoretic similarity measures. In contrast to a number of heuristic methods that include additional features into the generic mutual information measure, the proposed method strictly follows information theory under certain assumptions on feature probability distribution. The novel approach solves the problem of efficient estimation of multifeature mutual information from sparse high-dimensional feature space. The proposed measure was tested on magnetic resonance (MR) and computed tomography (CT) images. In addition, the measure was tested on positron emission tomography (PET) and MR images from the widely used Retrospective Image Registration Evaluation project image database. The results indicate that multi-feature mutual information, which combines image intensities and intensity gradients, is more robust than the standard single-feature intensity based mutual information, especially in cases of low global intensity correspondences, such as in PET/MR images or significant intensity inhomogeneity.

Download Full-text

Copper-dependent ATP7B up-regulation drives the resistance of TMEM16A-overexpressing head-and-neck cancer models to platinum toxicity

Biochemical Journal ◽

10.1042/bcj20190591 ◽

2019 ◽

Vol 476 (24) ◽

pp. 3705-3719 ◽

Cited By ~ 2

Author(s):

Avani Vyas ◽

Umamaheswar Duvvuri ◽

Kirill Kiselyov

Keyword(s):

Oxidative Stress ◽

Head And Neck ◽

Transcriptional Activation ◽

Platinum Resistance ◽

Mrna Levels ◽

Strong Positive Correlation ◽

Platinum Compounds ◽

Copper Chelation ◽

Novel Approach ◽

Cancer Models

Platinum-containing drugs such as cisplatin and carboplatin are routinely used for the treatment of many solid tumors including squamous cell carcinoma of the head and neck (SCCHN). However, SCCHN resistance to platinum compounds is well documented. The resistance to platinum has been linked to the activity of divalent transporter ATP7B, which pumps platinum from the cytoplasm into lysosomes, decreasing its concentration in the cytoplasm. Several cancer models show increased expression of ATP7B; however, the reason for such an increase is not known. Here we show a strong positive correlation between mRNA levels of TMEM16A and ATP7B in human SCCHN tumors. TMEM16A overexpression and depletion in SCCHN cell lines caused parallel changes in the ATP7B mRNA levels. The ATP7B increase in TMEM16A-overexpressing cells was reversed by suppression of NADPH oxidase 2 (NOX2), by the antioxidant N-Acetyl-Cysteine (NAC) and by copper chelation using cuprizone and bathocuproine sulphonate (BCS). Pretreatment with either chelator significantly increased cisplatin's sensitivity, particularly in the context of TMEM16A overexpression. We propose that increased oxidative stress in TMEM16A-overexpressing cells liberates the chelated copper in the cytoplasm, leading to the transcriptional activation of ATP7B expression. This, in turn, decreases the efficacy of platinum compounds by promoting their vesicular sequestration. We think that such a new explanation of the mechanism of SCCHN tumors’ platinum resistance identifies novel approach to treating these tumors.

Download Full-text

Collecting Words: A Clinical Example of a Morphology-Focused Orthographic Intervention

Language Speech and Hearing Services in Schools ◽

10.1044/2020_lshss-19-00050 ◽

2020 ◽

Vol 51 (3) ◽

pp. 544-560 ◽

Cited By ~ 3

Author(s):

Kimberly A. Murphy ◽

Emily A. Diehm

Keyword(s):

Written Language ◽

Morphological Knowledge ◽

English Orthography ◽

Word Level ◽

Novel Approach ◽

Language Pathology ◽

The One ◽

Critical Intervention ◽

Reading And Spelling ◽

Reading And Spelling Difficulties

Purpose Morphological interventions promote gains in morphological knowledge and in other oral and written language skills (e.g., phonological awareness, vocabulary, reading, and spelling), yet we have a limited understanding of critical intervention features. In this clinical focus article, we describe a relatively novel approach to teaching morphology that considers its role as the key organizing principle of English orthography. We also present a clinical example of such an intervention delivered during a summer camp at a university speech and hearing clinic. Method Graduate speech-language pathology students provided a 6-week morphology-focused orthographic intervention to children in first through fourth grade ( n = 10) who demonstrated word-level reading and spelling difficulties. The intervention focused children's attention on morphological families, teaching how morphology is interrelated with phonology and etymology in English orthography. Results Comparing pre- and posttest scores, children demonstrated improvement in reading and/or spelling abilities, with the largest gains observed in spelling affixes within polymorphemic words. Children and their caregivers reacted positively to the intervention. Therefore, data from the camp offer preliminary support for teaching morphology within the context of written words, and the intervention appears to be a feasible approach for simultaneously increasing morphological knowledge, reading, and spelling. Conclusion Children with word-level reading and spelling difficulties may benefit from a morphology-focused orthographic intervention, such as the one described here. Research on the approach is warranted, and clinicians are encouraged to explore its possible effectiveness in their practice. Supplemental Material https://doi.org/10.23641/asha.12290687

Download Full-text