scholarly journals Matching Large Baseline Oblique Stereo Images Using an End-to-End Convolutional Neural Network

2021 ◽  
Vol 13 (2) ◽  
pp. 274
Author(s):  
Guobiao Yao ◽  
Alper Yilmaz ◽  
Li Zhang ◽  
Fei Meng ◽  
Haibin Ai ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Stereo Matching ◽  
Least Square ◽  
Affine Invariant ◽  
Stereo Images ◽  
Distance Ratio ◽  
Matching Algorithm ◽  
End To End

The available stereo matching algorithms produce large number of false positive matches or only produce a few true-positives across oblique stereo images with large baseline. This undesired result happens due to the complex perspective deformation and radiometric distortion across the images. To address this problem, we propose a novel affine invariant feature matching algorithm with subpixel accuracy based on an end-to-end convolutional neural network (CNN). In our method, we adopt and modify a Hessian affine network, which we refer to as IHesAffNet, to obtain affine invariant Hessian regions using deep learning framework. To improve the correlation between corresponding features, we introduce an empirical weighted loss function (EWLF) based on the negative samples using K nearest neighbors, and then generate deep learning-based descriptors with high discrimination that is realized with our multiple hard network structure (MTHardNets). Following this step, the conjugate features are produced by using the Euclidean distance ratio as the matching metric, and the accuracy of matches are optimized through the deep learning transform based least square matching (DLT-LSM). Finally, experiments on Large baseline oblique stereo images acquired by ground close-range and unmanned aerial vehicle (UAV) verify the effectiveness of the proposed approach, and comprehensive comparisons demonstrate that our matching algorithm outperforms the state-of-art methods in terms of accuracy, distribution and correct ratio. The main contributions of this article are: (i) our proposed MTHardNets can generate high quality descriptors; and (ii) the IHesAffNet can produce substantial affine invariant corresponding features with reliable transform parameters.

scholarly journals An end-to-end stereo matching algorithm based on improved convolutional neural network

2020 ◽  
Vol 17 (6) ◽  
pp. 7787-7803
Author(s):  
Yan Liu ◽  
◽  
Bingxue Lv ◽  
Yuheng Wang ◽  
Wei Huang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Stereo Matching ◽  
Matching Algorithm ◽  
End To End

scholarly journals End-to-End Deep Learning Fusion of Fingerprint and Electrocardiogram Signals for Presentation Attack Detection

Sensors ◽  
2020 ◽  
Vol 20 (7) ◽  
pp. 2085 ◽  
Author(s):  
Rami M. Jomaa ◽  
Hassan Mathkour ◽  
Yakoub Bazi ◽  
Md Saiful Islam
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Attack Detection ◽  
Ecg Signal ◽  
Ecg Signals ◽  
Biometric Systems ◽  
Fingerprint Biometrics ◽  
End To End ◽  
The Impact

Although fingerprint-based systems are the commonly used biometric systems, they suffer from a critical vulnerability to a presentation attack (PA). Therefore, several approaches based on a fingerprint biometrics have been developed to increase the robustness against a PA. We propose an alternative approach based on the combination of fingerprint and electrocardiogram (ECG) signals. An ECG signal has advantageous characteristics that prevent the replication. Combining a fingerprint with an ECG signal is a potentially interesting solution to reduce the impact of PAs in biometric systems. We also propose a novel end-to-end deep learning-based fusion neural architecture between a fingerprint and an ECG signal to improve PA detection in fingerprint biometrics. Our model uses state-of-the-art EfficientNets for generating a fingerprint feature representation. For the ECG, we investigate three different architectures based on fully-connected layers (FC), a 1D-convolutional neural network (1D-CNN), and a 2D-convolutional neural network (2D-CNN). The 2D-CNN converts the ECG signals into an image and uses inverted Mobilenet-v2 layers for feature generation. We evaluated the method on a multimodal dataset, that is, a customized fusion of the LivDet 2015 fingerprint dataset and ECG data from real subjects. Experimental results reveal that this architecture yields a better average classification accuracy compared to a single fingerprint modality.

scholarly journals A robust and interpretable, end-to-end deep learning model for cytometry data

2020 ◽  
Author(s):  
Zicheng Hu ◽  
Alice Tang ◽  
Jaiveer Singh ◽  
Sanchita Bhattacharya ◽  
Atul J. Butte
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Learning Model ◽  
Deep Convolutional Neural Network ◽  
Significant Information ◽  
End To End ◽  
Deep Learning Model

AbstractCytometry technologies are essential tools for immunology research, providing high-throughput measurements of the immune cells at the single-cell level. Traditional approaches in interpreting and using cytometry measurements include manual or automated gating to identify cell subsets from the cytometry data, providing highly intuitive results but may lead to significant information loss, in that additional details in measured or correlated cell signals might be missed. In this study, we propose and test a deep convolutional neural network for analyzing cytometry data in an end-to-end fashion, allowing a direct association between raw cytometry data and the clinical outcome of interest. Using nine large CyTOF studies from the open-access ImmPort database, we demonstrated that the deep convolutional neural network model can accurately diagnose the latent cytomegalovirus (CMV) in healthy individuals, even when using highly heterogeneous data from different studies. In addition, we developed a permutation-based method for interpreting the deep convolutional neural network model and identified a CD27-CD94+ CD8+ T cell population significantly associated with latent CMV infection. Finally, we provide a tutorial for creating, training and interpreting the tailored deep learning model for cytometry data using Keras and TensorFlow (github.com/hzc363/DeepLearningCyTOF).

Stereo Matching Algorithm Based on Three-Dimensional Convolutional Neural Network

2019 ◽  
Vol 39 (11) ◽  
pp. 1115001
Author(s):  
王玉锋 Wang Yufeng ◽  
王宏伟 Wang Hongwei ◽  
于光 Yu Guang ◽  
杨明权 Yang Mingquan ◽  
袁昱纬 Yuan Yuwei ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Stereo Matching ◽  
Three Dimensional ◽  
Matching Algorithm

scholarly journals Direct pixel to pixel principal strain mapping from tagging MRI using end to end deep convolutional neural network (DeepStrain)

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Khaled Z. Abd-Elmoniem ◽  
Inas A. Yassine ◽  
Nader S. Metwalli ◽  
Ahmed Hamimi ◽  
Ronald Ouwerkerk ◽  
...  
Keyword(s):  
Neural Network ◽  
Monte Carlo ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Ground Truth ◽  
Image Resolution ◽  
Principal Strain ◽  
Strain Mapping ◽  
End To End

AbstractRegional soft tissue mechanical strain offers crucial insights into tissue's mechanical function and vital indicators for different related disorders. Tagging magnetic resonance imaging (tMRI) has been the standard method for assessing the mechanical characteristics of organs such as the heart, the liver, and the brain. However, constructing accurate artifact-free pixelwise strain maps at the native resolution of the tagged images has for decades been a challenging unsolved task. In this work, we developed an end-to-end deep-learning framework for pixel-to-pixel mapping of the two-dimensional Eulerian principal strains $$\varvec{{\varepsilon }}_{\boldsymbol{p1}}$$ ε p 1 and $$\varvec{{\varepsilon }}_{\boldsymbol{p2}}$$ ε p 2 directly from 1-1 spatial modulation of magnetization (SPAMM) tMRI at native image resolution using convolutional neural network (CNN). Four different deep learning conditional generative adversarial network (cGAN) approaches were examined. Validations were performed using Monte Carlo computational model simulations, and in-vivo datasets, and compared to the harmonic phase (HARP) method, a conventional and validated method for tMRI analysis, with six different filter settings. Principal strain maps of Monte Carlo tMRI simulations with various anatomical, functional, and imaging parameters demonstrate artifact-free solid agreements with the corresponding ground-truth maps. Correlations with the ground-truth strain maps were R = 0.90 and 0.92 for the best-proposed cGAN approach compared to R = 0.12 and 0.73 for the best HARP method for $$\varvec{{\varepsilon }}_{\boldsymbol{p1}}$$ ε p 1 and $$\varvec{{\varepsilon }}_{\boldsymbol{p2}}$$ ε p 2 , respectively. The proposed cGAN approach's error was substantially lower than the error in the best HARP method at all strain ranges. In-vivo results are presented for both healthy subjects and patients with cardiac conditions (Pulmonary Hypertension). Strain maps, obtained directly from their corresponding tagged MR images, depict for the first time anatomical, functional, and temporal details at pixelwise native high resolution with unprecedented clarity. This work demonstrates the feasibility of using the deep learning cGAN for direct myocardial and liver Eulerian strain mapping from tMRI at native image resolution with minimal artifacts.

scholarly journals A new function of stereo matching algorithm based on hybrid convolutional neural network

2022 ◽  
Vol 25 (1) ◽  
pp. 223
Author(s):  
Mohd Saad Hamid ◽  
Nurulfajar Abd Manap ◽  
Rostam Affendi Hamzah ◽  
Ahmad Fauzan Kadmin ◽  
Shamsul Fakhar Abd Gani ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Stereo Matching ◽  
Three Dimensional ◽  
Bilateral Filter ◽  
Disparity Map ◽  
Matching Algorithm ◽  
Matching Cost ◽  
The Difference ◽  
Aggregation Step

This paper proposes a new hybrid method between the learning-based and handcrafted methods for a stereo matching algorithm. The main purpose of the stereo matching algorithm is to produce a disparity map. This map is essential for many applications, including three-dimensional (3D) reconstruction. The raw disparity map computed by a convolutional neural network (CNN) is still prone to errors in the low texture region. The algorithm is set to improve the matching cost computation stage with hybrid CNN-based combined with truncated directional intensity computation. The difference in truncated directional intensity value is employed to decrease radiometric errors. The proposed method’s raw matching cost went through the cost aggregation step using the bilateral filter (BF) to improve accuracy. The winner-take-all (WTA) optimization uses the aggregated cost volume to produce an initial disparity map. Finally, a series of refinement processes enhance the initial disparity map for a more accurate final disparity map. This paper verified the performance of the algorithm using the Middlebury online stereo benchmarking system. The proposed algorithm achieves the objective of generating a more accurate and smooth disparity map with different depths at low texture regions through better matching cost quality.

scholarly journals Classification and Visualization of Alzheimer’s Disease using Volumetric Convolutional Neural Network and Transfer Learning

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Kanghan Oh ◽  
Young-Chul Chung ◽  
Ko Woon Kim ◽  
Woo-Sung Kim ◽  
Il-Seok Oh
Keyword(s):  
Neural Network ◽  
Alzheimer’S Disease ◽  
Deep Learning ◽  
Cognitive Impairment ◽  
Mild Cognitive Impairment ◽  
Convolutional Neural Network ◽  
Transfer Learning ◽  
Classification Task ◽  
Classification Tasks ◽  
End To End

AbstractRecently, deep-learning-based approaches have been proposed for the classification of neuroimaging data related to Alzheimer’s disease (AD), and significant progress has been made. However, end-to-end learning that is capable of maximizing the impact of deep learning has yet to receive much attention due to the endemic challenge of neuroimaging caused by the scarcity of data. Thus, this study presents an approach meant to encourage the end-to-end learning of a volumetric convolutional neural network (CNN) model for four binary classification tasks (AD vs. normal control (NC), progressive mild cognitive impairment (pMCI) vs. NC, stable mild cognitive impairment (sMCI) vs. NC and pMCI vs. sMCI) based on magnetic resonance imaging (MRI) and visualizes its outcomes in terms of the decision of the CNNs without any human intervention. In the proposed approach, we use convolutional autoencoder (CAE)-based unsupervised learning for the AD vs. NC classification task, and supervised transfer learning is applied to solve the pMCI vs. sMCI classification task. To detect the most important biomarkers related to AD and pMCI, a gradient-based visualization method that approximates the spatial influence of the CNN model’s decision was applied. To validate the contributions of this study, we conducted experiments on the ADNI database, and the results demonstrated that the proposed approach achieved the accuracies of 86.60% and 73.95% for the AD and pMCI classification tasks respectively, outperforming other network models. In the visualization results, the temporal and parietal lobes were identified as key regions for classification.

scholarly journals Stereo Matching Algorithm Based on Hybrid Convolutional Neural Network and Directional Intensity Difference

2021 ◽  
Vol 11 (6) ◽  
pp. 86-96
Author(s):  
Mohd Saad Hamid ◽  
◽  
Nurulfajar Abd Manap ◽  
Rostam Affendi Hamzah ◽  
Ahmad Fauzan Kadmin
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Stereo Matching ◽  
Good Accuracy ◽  
Intensity Difference ◽  
Disparity Map ◽  
Matching Algorithm ◽  
Matching Cost ◽  
Autonomous Vehicle Navigation ◽  
Repetitive Pattern

Fundamentally, a stereo matching algorithm produces a disparity map or depth map. This map contains valuable information for many applications, such as range estimation, autonomous vehicle navigation and 3D surface reconstruction. The stereo matching process faces various challenges to get an accurate result for example low texture area, repetitive pattern and discontinuity regions. The proposed algorithm must be robust and viable with all of these challenges and is capable to deliver good accuracy. Hence, this article proposes a new stereo matching algorithm based on a hybrid Convolutional Neural Network (CNN) combined with directional intensity differences at the matching cost stage. The proposed algorithm contains a deep learning-based method and a handcrafted method. Then, the bilateral filter is used to aggregate the matching cost volume while preserving the object edges. The Winner-Take-All (WTA) is utilized at the optimization stage which the WTA normalizes the disparity values. At the last stage, a series of refinement processes will be applied to enhance the final disparity map. A standard benchmarking evaluation system from the Middlebury Stereo dataset is used to measure the algorithm performance. This dataset provides images with the characteristics of low texture area, repetitive pattern and discontinuity regions. The average error produced for all pixel regions is 8.51%, while the nonoccluded region is 5.77%. Based on the experimental results, the proposed algorithm produces good accuracy and robustness against the stereo matching challenges. It is also competitive with other published methods and can be used as a complete algorithm

scholarly journals Task-Driven Learned Hyperspectral Data Reduction Using End-to-End Supervised Deep Learning

2020 ◽  
Vol 6 (12) ◽  
pp. 132
Author(s):  
Mathé T. Zeegers ◽  
Daniël M. Pelt ◽  
Tristan van Leeuwen ◽  
Robert van Liere ◽  
Kees Joost Batenburg
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Data Reduction ◽  
Hyperspectral Data ◽  
Reduction Step ◽  
Combining Data ◽  
Wide Range ◽  
Reduction Methods ◽  
End To End

An important challenge in hyperspectral imaging tasks is to cope with the large number of spectral bins. Common spectral data reduction methods do not take prior knowledge about the task into account. Consequently, sparsely occurring features that may be essential for the imaging task may not be preserved in the data reduction step. Convolutional neural network (CNN) approaches are capable of learning the specific features relevant to the particular imaging task, but applying them directly to the spectral input data is constrained by the computational efficiency. We propose a novel supervised deep learning approach for combining data reduction and image analysis in an end-to-end architecture. In our approach, the neural network component that performs the reduction is trained such that image features most relevant for the task are preserved in the reduction step. Results for two convolutional neural network architectures and two types of generated datasets show that the proposed Data Reduction CNN (DRCNN) approach can produce more accurate results than existing popular data reduction methods, and can be used in a wide range of problem settings. The integration of knowledge about the task allows for more image compression and higher accuracies compared to standard data reduction methods.

scholarly journals A robust and interpretable end-to-end deep learning model for cytometry data

2020 ◽  
Vol 117 (35) ◽  
pp. 21373-21380
Author(s):  
Zicheng Hu ◽  
Alice Tang ◽  
Jaiveer Singh ◽  
Sanchita Bhattacharya ◽  
Atul J. Butte
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Network Model ◽  
Neural Network Model ◽  
Learning Model ◽  
Deep Convolutional Neural Network ◽  
Significant Information ◽  
End To End ◽  
Deep Learning Model

Cytometry technologies are essential tools for immunology research, providing high-throughput measurements of the immune cells at the single-cell level. Existing approaches in interpreting and using cytometry measurements include manual or automated gating to identify cell subsets from the cytometry data, providing highly intuitive results but may lead to significant information loss, in that additional details in measured or correlated cell signals might be missed. In this study, we propose and test a deep convolutional neural network for analyzing cytometry data in an end-to-end fashion, allowing a direct association between raw cytometry data and the clinical outcome of interest. Using nine large cytometry by time-of-flight mass spectrometry or mass cytometry (CyTOF) studies from the open-access ImmPort database, we demonstrated that the deep convolutional neural network model can accurately diagnose the latent cytomegalovirus (CMV) in healthy individuals, even when using highly heterogeneous data from different studies. In addition, we developed a permutation-based method for interpreting the deep convolutional neural network model. We were able to identify a CD27- CD94+ CD8+ T cell population significantly associated with latent CMV infection, confirming the findings in previous studies. Finally, we provide a tutorial for creating, training, and interpreting the tailored deep learning model for cytometry data using Keras and TensorFlow (https://github.com/hzc363/DeepLearningCyTOF).

Sign in / Sign up

Export Citation Format

Share Document