scholarly journals Unsupervised Cross-Spectral Stereo Matching by Learning to Synthesize

2019 ◽  
Vol 33 ◽  
pp. 8706-8713 ◽  
Author(s):  
Mingyang Liang ◽  
Xiaoyang Guo ◽  
Hongsheng Li ◽  
Xiaogang Wang ◽  
You Song
Keyword(s):  
Semantic Information ◽  
Stereo Matching ◽  
Spectral Image ◽  
Adversarial Learning ◽  
Additional Information ◽  
Spectral Bands ◽  
Image Translation ◽  
View Reconstruction ◽  
Image Pairs ◽  
Spectral Translation

Unsupervised cross-spectral stereo matching aims at recovering disparity given cross-spectral image pairs without any depth or disparity supervision. The estimated depth provides additional information complementary to original images, which can be helpful for other vision tasks such as tracking, recognition and detection. However, there are large appearance variations between images from different spectral bands, which is a challenge for cross-spectral stereo matching. Existing deep unsupervised stereo matching methods are sensitive to the appearance variations and do not perform well on cross-spectral data. We propose a novel unsupervised crossspectral stereo matching framework based on image-to-image translation. First, a style adaptation network transforms images across different spectral bands by cycle consistency and adversarial learning, during which appearance variations are minimized. Then, a stereo matching network is trained with image pairs from the same spectra using view reconstruction loss. At last, the estimated disparity is utilized to supervise the spectral translation network in an end-to-end way. Moreover, a novel style adaptation network F-cycleGAN is proposed to improve the robustness of spectral translation. Our method can tackle appearance variations and enhance the robustness of unsupervised cross-spectral stereo matching. Experimental results show that our method achieves good performance without using depth supervision or explicit semantic information.

Adaptive stereo matching for SPOT multi-spectral image pairs

1994 ◽  
Author(s):  
A. J. Chen ◽  
C. H. Liu ◽  
J. Y. Rau ◽  
Lin-Chi Chen
Keyword(s):  
Stereo Matching ◽  
Spectral Image ◽  
Image Pairs

scholarly journals Cross-Spectral Local Descriptors via Quadruplet Network

2017 ◽  
Author(s):  
Cristhian A. Aguilera ◽  
Angel D. Sappa ◽  
Cristhian Aguilera ◽  
Ricardo Toledo
Keyword(s):  
Spectral Band ◽  
Visible Spectrum ◽  
Input Image ◽  
Training Data ◽  
Spectral Image ◽  
Recent Success ◽  
Spectral Bands ◽  
Image Patches ◽  
Matching Pair ◽  
Image Pairs

This paper presents a novel CNN-based architecture, referred to as Q-Net, to learn local feature descriptors that are useful for matching image patches from two different spectral bands. Given correctly matched and non-matching cross-spectral image pairs, a quadruplet network is trained to map input image patches to a common Euclidean space, regardless of the input spectral band. Our approach is inspired by the recent success of triplet networks in the visible spectrum, but adapted for cross-spectral scenarios, where for each matching pair there are always two possible non-matching patches; one for each spectrum. Experimental evaluations on a public cross-spectral VIS-NIR dataset shows that the proposed approach improves the state-of-the-art. Moreover, the proposed technique can also be used in mono-spectral settings, obtaining a similar performance to triplet network descriptors, but requiring less training data.

scholarly journals Forensic analysis of beverage stains using hyperspectral imaging

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Binu Melit Devassy ◽  
Sony George
Keyword(s):  
Hyperspectral Imaging ◽  
Crime Scene ◽  
Forensic Analysis ◽  
Classification Model ◽  
Support Vector ◽  
Additional Information ◽  
Non Invasive ◽  
Spectral Bands ◽  
Data Dimensionality Reduction ◽  
Gradient Based

AbstractDocumentation and analysis of crime scene evidences are of great importance in any forensic investigation. In this paper, we present the potential of hyperspectral imaging (HSI) to detect and analyze the beverage stains on a paper towel. To detect the presence and predict the age of the commonly used drinks in a crime scene, we leveraged the additional information present in the HSI data. We used 12 different beverages and four types of paper hand towel to create the sample stains in the current study. A support vector machine (SVM) is used to achieve the classification, and a convolutional auto-encoder is used to achieve HSI data dimensionality reduction, which helps in easy perception, process, and visualization of the data. The SVM classification model was re-established for a lighter and quicker classification model on the basis of the reduced dimension. We employed volume-gradient-based band selection for the identification of relevant spectral bands in the HSI data. Spectral data recorded at different time intervals up to 72 h is analyzed to trace the spectral changes. The results show the efficacy of the HSI techniques for rapid, non-contact, and non-invasive analysis of beverage stains.

scholarly journals Efficient Stereo Matching with Decoupled Dissimilarity Measure Using Successive Weighted Summation

2014 ◽  
Vol 2014 ◽  
pp. 1-9
Author(s):  
Cheng-Tao Zhu ◽  
Yau-Zen Chang ◽  
Huai-Ming Wang ◽  
Kai He ◽  
Shih-Tseng Lee ◽  
...  
Keyword(s):  
Stereo Matching ◽  
Dissimilarity Measure ◽  
Stereo Image ◽  
Fitting Procedure ◽  
Comparable Performance ◽  
Novel Method ◽  
Disparity Maps ◽  
Image Pairs ◽  
Benchmark Evaluation ◽  
Complexity Algorithm

Developing matching algorithms from stereo image pairs to obtain correct disparity maps for 3D reconstruction has been the focus of intensive research. A constant computational complexity algorithm to calculate dissimilarity aggregation in assessing disparity based on separable successive weighted summation (SWS) among horizontal and vertical directions was proposed but still not satisfactory. This paper presents a novel method which enables decoupled dissimilarity measure in the aggregation, further improving the accuracy and robustness of stereo correspondence. The aggregated cost is also used to refine disparities based on a local curve-fitting procedure. According to our experimental results on Middlebury benchmark evaluation, the proposed approach has comparable performance when compared with the selected state-of-the-art algorithms and has the lowest mismatch rate. Besides, the refinement procedure is shown to be capable of preserving object boundaries and depth discontinuities while smoothing out disparity maps.

INDEPENDENT COMPONENT ANALYSIS FOR CLASSIFYING MULTISPECTRAL IMAGES WITH DIMENSIONALITY LIMITATION

2004 ◽  
Vol 01 (03) ◽  
pp. 201-216 ◽  
Author(s):  
QIAN DU ◽  
IVICA KOPRIVA ◽  
HAROLD SZU
Keyword(s):  
Independent Component Analysis ◽  
Spline Interpolation ◽  
Unsupervised Classification ◽  
Original Data ◽  
Component Analysis ◽  
Independent Component ◽  
Spectral Signatures ◽  
Additional Information ◽  
Spectral Bands ◽  
Linear Band

Airborne and spaceborne remote sensors can acquire invaluable information about earth surface, which have many important applications. The acquired information usually is represented as two-dimensional grids, i.e. images. One of techniques to processing such images is Independent Component Analysis (ICA), which is particularly useful for classifying objects with unknown spectral signatures in an unknown image scene, i.e. unsupervised classification. Since the weight matrix in ICA is a square matrix for the purpose of mathematical tractability, the number of objects that can be classified is equal to the data dimensionality, i.e. the number of spectral bands. When the number of sensors (or spectral channels) is very small (e.g. a 3-band CIR photograph and 6-band Landsat image with the thermal band being removed), it is impossible to classify all the different objects present in an image scene using the original data. In order to solve this problem, we present a data dimensionality expansion technique to generate artificial bands. Its basic idea is to use nonlinear functions to capture and highlight the similarity/dissimilarity between original spectral measurements, which can provide more data with additional information for detecting and classifying more objects. The results from such a nonlinear band generation approach are compared with a linear band generation method using cubic spline interpolation of pixel spectral signatures. The experiments demonstrate that nonlinear band generation approach can significantly improve unsupervised classification accuracy, while linear band generation method cannot since no new information can be provided. It is also demonstrated that ICA is more powerful than other frequently used unsupervised classification algorithms such as ISODATA.

scholarly journals SHADOW DETECTION IN HYPERSPECTRAL IMAGES ACQUIRED BY UAV

2019 ◽  
Vol XLII-2/W13 ◽  
pp. 371-377 ◽  
Author(s):  
N. N. Imai ◽  
A. M. G. Tommaselli ◽  
A. Berveglieri ◽  
E. A. S. Moriya
Keyword(s):  
Blue Light ◽  
Near Infrared ◽  
Shadow Detection ◽  
Hyperspectral Images ◽  
Agricultural Field ◽  
Additional Information ◽  
Spectral Bands ◽  
Agricultural Applications ◽  
Aerial Vehicle ◽  
Hyperspectral Camera

<p><strong>Abstract.</strong> Shadows are common in any kind of remote sensing images. Unmanned Aerial Vehicle &amp;ndash; UAV with a light camera attached can acquire images illuminated either by direct sunlight or by diffuse light under clouds. Indeed, areas with pixels shaded by clouds must be detected and labelled in order to use this additional information for image analysis. Classification of health and diseased plants in permanent culture as the orange plantation field can present some errors due to tree cast shadow. So, hyperspectral or multispectral image classification can be improved by previous shadow detection. Some FPI hyperspectral camera, designed for agricultural applications is limited in the spectral range between 500 to 900&amp;thinsp;nm. Wavelengths in the region of blue light and in the SWIR spectral region have physical properties that enable the enhancement of shaded regions in the images. In this work some combinations of different spectral bands were evaluated in order to specify those suitable to detect shadows in agricultural field images. In this sense, considering that vegetation and soil are the two main kind of coverage in an agricultural field, we hypothesized that wavelengths near blue light and the longest near infrared available in the camera range are good choices. In both spectral regions soil and vegetation targets have small spectral differences which contribute to enhance the differences between shaded and illuminated regions in the image. Hyperspectral images acquired with a FPI hyperspectral camera onboard a UAV over a plantation of oranges were used to evaluate these spectral bands. The results showed that the wavelengths of aproximatelly 510&amp;thinsp;nm and 840&amp;thinsp;nm available in the FPI camera are the best to detect any type of shadows in the agricultural fields.</p>

scholarly journals Neural User Response Generator: Fake News Detection with Collective User Intelligence

2018 ◽  
Author(s):  
Feng Qian ◽  
Chengyue Gong ◽  
Karishma Sharma ◽  
Yan Liu
Keyword(s):  
Neural Network ◽  
Social Media ◽  
Early Detection ◽  
Convolutional Neural Network ◽  
Semantic Information ◽  
Experimental Results ◽  
News Article ◽  
Fake News ◽  
Additional Information ◽  
Word Level

Fake news on social media is a major challenge and studies have shown that fake news can propagate exponentially quickly in early stages. Therefore, we focus on early detection of fake news, and consider that only news article text is available at the time of detection, since additional information such as user responses and propagation patterns can be obtained only after the news spreads. However, we find historical user responses to previous articles are available and can be treated as soft semantic labels, that enrich the binary label of an article, by providing insights into why the article must be labeled as fake. We propose a novel Two-Level Convolutional Neural Network with User Response Generator (TCNN-URG) where TCNN captures semantic information from article text by representing it at the sentence and word level, and URG learns a generative model of user response to article text from historical user responses which it can use to generate responses to new articles in order to assist fake news detection. We conduct experiments on one available dataset and a larger dataset collected by ourselves. Experimental results show that TCNN-URG outperforms the baselines based on prior approaches that detect fake news from article text alone.

Sign in / Sign up

Export Citation Format

Share Document