Enhanced machine perception by a scalable fusion of RGB–NIR image pairs in diverse exposure environments

2021 ◽  
Vol 32 (4) ◽  
Author(s):  
Wahengbam Kanan Kumar ◽  
Ningthoujam Johny Singh ◽  
Aheibam Dinamani Singh ◽  
Kishorjit Nongmeikapam
Keyword(s):  
Machine Perception ◽  
Image Pairs

LambdaNet: A Fully Convolutional Architecture for Directional Change Detection

2020 ◽  
Vol 2020 (8) ◽  
pp. 114-1-114-7
Author(s):  
Bryan Blakeslee ◽  
Andreas Savakis
Keyword(s):  
Change Detection ◽  
Satellite Imagery ◽  
Classification Scheme ◽  
Deep Architecture ◽  
Binary Process ◽  
Directional Change ◽  
Class Labels ◽  
Image Pairs

Change detection in image pairs has traditionally been a binary process, reporting either “Change” or “No Change.” In this paper, we present LambdaNet, a novel deep architecture for performing pixel-level directional change detection based on a four class classification scheme. LambdaNet successfully incorporates the notion of “directional change” and identifies differences between two images as “Additive Change” when a new object appears, “Subtractive Change” when an object is removed, “Exchange” when different objects are present in the same location, and “No Change.” To obtain pixel annotated change maps for training, we generated directional change class labels for the Change Detection 2014 dataset. Our tests illustrate that LambdaNet would be suitable for situations where the type of change is unstructured, such as change detection scenarios in satellite imagery.

Three-dimensional Reconstruction of Microscopic Image Based on Multi-ST Algorithm

2020 ◽  
Vol 64 (2) ◽  
pp. 20506-1-20506-7
Author(s):  
Min Zhu ◽  
Rongfu Zhang ◽  
Pei Ma ◽  
Xuedian Zhang ◽  
Qi Guo
Keyword(s):  
3D Reconstruction ◽  
Three Dimensional ◽  
Scale Model ◽  
Microscopic Image ◽  
Multi Scale ◽  
Gaussian Pyramid ◽  
Cost Aggregation ◽  
Dimensional Reconstruction ◽  
Image Pairs ◽  
Accurate Matching

Abstract Three-dimensional (3D) reconstruction is extensively used in microscopic applications. Reducing excessive error points and achieving accurate matching of weak texture regions have been the classical challenges for 3D microscopic vision. A Multi-ST algorithm was proposed to improve matching accuracy. The process is performed in two main stages: scaled microscopic images and regularized cost aggregation. First, microscopic image pairs with different scales were extracted according to the Gaussian pyramid criterion. Second, a novel cost aggregation approach based on the regularized multi-scale model was implemented into all scales to obtain the final cost. To evaluate the performances of the proposed Multi-ST algorithm and compare different algorithms, seven groups of images from the Middlebury dataset and four groups of experimental images obtained by a binocular microscopic system were analyzed. Disparity maps and reconstruction maps generated by the proposed approach contained more information and fewer outliers or artifacts. Furthermore, 3D reconstruction of the plug gauges using the Multi-ST algorithm showed that the error was less than 0.025 mm.

scholarly journals Cloud Detection Based on High Resolution Stereo Pairs of the Geostationary Meteosat Images

2020 ◽  
Vol 12 (3) ◽  
pp. 371 ◽  
Author(s):  
Sahar Dehnavi ◽  
Yasser Maghsoudi ◽  
Klemen Zakšek ◽  
Mohammad Javad Valadan Zoej ◽  
Gunther Seckmeyer ◽  
...  
Keyword(s):  
High Resolution ◽  
Matched Filter ◽  
Stereo Pair ◽  
Cloud Detection ◽  
Operating Characteristics ◽  
Detection Techniques ◽  
Infrared Imager ◽  
Detection Approach ◽  
Considerable Impact ◽  
Image Pairs

Due to the considerable impact of clouds on the energy balance in the atmosphere and on the earth surface, they are of great importance for various applications in meteorology or remote sensing. An important aspect of the cloud research studies is the detection of cloudy pixels from the processing of satellite images. In this research, we investigated a stereographic method on a new set of Meteosat images, namely the combination of the high resolution visible (HRV) channel of the Meteosat-8 Indian Ocean Data Coverage (IODC) as a stereo pair with the HRV channel of the Meteosat Second Generation (MSG) Meteosat-10 image at 0° E. In addition, an approach based on the outputs from stereo analysis was proposed to detect cloudy pixels. This approach is introduced with a 2D-scatterplot based on the parallax value and the minimum intersection distance. The mentioned scatterplot was applied to determine/detect cloudy pixels in various image subsets with different amounts of cloud cover. Apart from the general advantage of the applied stereography method, which only depends on geometric relationships, the cloud detection results are also improved because: (1) The stereo pair is the HRV bands of the Spinning Enhanced Visible and InfraRed Imager (SEVIRI) sensor, with the highest spatial resolution available from the Meteosat geostationary platform; and (2) the time difference between the image pairs is nearly 5 s, which improves the matching results and also decreases the effect of cloud movements. In order to prove this improvement, the results of this stereo-based approach were compared with three different reflectance-based target detection techniques, including the adaptive coherent estimator (ACE), constrained energy minimization (CEM), and matched filter (MF). The comparison of the receiver operating characteristics (ROC) detection curves and the area under these curves (AUC) showed better detection results with the proposed method. The AUC value was 0.79, 0.90, 0.90, and 0.93 respectively for ACE, CEM, MF, and the proposed stereo-based detection approach. The results of this research shall enable a more realistic modelling of down-welling solar irradiance in the future.

scholarly journals Joint regression and learning from pairwise rankings for personalized image aesthetic assessment

2021 ◽  
Author(s):  
Jin Zhou ◽  
Qing Zhang ◽  
Jian-Hao Fan ◽  
Wei Sun ◽  
Wei-Shi Zheng
Keyword(s):  
Large Scale ◽  
Assessment Model ◽  
Generic Model ◽  
Small Subset ◽  
Deep Convolutional Neural Networks ◽  
Personal Taste ◽  
Hinge Loss ◽  
Novel Approach ◽  
Large Scale Dataset ◽  
Image Pairs

AbstractRecent image aesthetic assessment methods have achieved remarkable progress due to the emergence of deep convolutional neural networks (CNNs). However, these methods focus primarily on predicting generally perceived preference of an image, making them usually have limited practicability, since each user may have completely different preferences for the same image. To address this problem, this paper presents a novel approach for predicting personalized image aesthetics that fit an individual user’s personal taste. We achieve this in a coarse to fine manner, by joint regression and learning from pairwise rankings. Specifically, we first collect a small subset of personal images from a user and invite him/her to rank the preference of some randomly sampled image pairs. We then search for the K-nearest neighbors of the personal images within a large-scale dataset labeled with average human aesthetic scores, and use these images as well as the associated scores to train a generic aesthetic assessment model by CNN-based regression. Next, we fine-tune the generic model to accommodate the personal preference by training over the rankings with a pairwise hinge loss. Experiments demonstrate that our method can effectively learn personalized image aesthetic preferences, clearly outperforming state-of-the-art methods. Moreover, we show that the learned personalized image aesthetic benefits a wide variety of applications.

A Contour Co-tracking Method for Image Pairs

2021 ◽  
pp. 1-1
Author(s):  
Bin Wang ◽  
Dapeng Tao ◽  
Rui Dong ◽  
Yuanyan Tang ◽  
Xinbo Gao
Keyword(s):  
Tracking Method ◽  
Image Pairs

scholarly journals RobotP: A Benchmark Dataset for 6D Object Pose Estimation

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1299
Author(s):  
Honglin Yuan ◽  
Tim Hoogenkamp ◽  
Remco C. Veltkamp
Keyword(s):  
Pose Estimation ◽  
Ground Truth ◽  
3D Models ◽  
Depth Image ◽  
Great Success ◽  
Estimation Algorithms ◽  
Depth Images ◽  
Object Pose Estimation ◽  
Image Pairs ◽  
Bounding Boxes

Deep learning has achieved great success on robotic vision tasks. However, when compared with other vision-based tasks, it is difficult to collect a representative and sufficiently large training set for six-dimensional (6D) object pose estimation, due to the inherent difficulty of data collection. In this paper, we propose the RobotP dataset consisting of commonly used objects for benchmarking in 6D object pose estimation. To create the dataset, we apply a 3D reconstruction pipeline to produce high-quality depth images, ground truth poses, and 3D models for well-selected objects. Subsequently, based on the generated data, we produce object segmentation masks and two-dimensional (2D) bounding boxes automatically. To further enrich the data, we synthesize a large number of photo-realistic color-and-depth image pairs with ground truth 6D poses. Our dataset is freely distributed to research groups by the Shape Retrieval Challenge benchmark on 6D pose estimation. Based on our benchmark, different learning-based approaches are trained and tested by the unified dataset. The evaluation results indicate that there is considerable room for improvement in 6D object pose estimation, particularly for objects with dark colors, and photo-realistic images are helpful in increasing the performance of pose estimation algorithms.

scholarly journals Activity Recognition in Residential Spaces with Internet of Things Devices and Thermal Imaging

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 988
Author(s):  
Kshirasagar Naik ◽  
Tejas Pandit ◽  
Nitin Naik ◽  
Parth Shah
Keyword(s):  
Internet Of Things ◽  
Activity Recognition ◽  
Thermal Model ◽  
Thermal Image ◽  
Heat Radiation ◽  
Household Activity ◽  
Thermal Images ◽  
Gas Leakage ◽  
3D Space ◽  
Image Pairs

In this paper, we design algorithms for indoor activity recognition and 3D thermal model generation using thermal images, RGB images, captured from external sensors, and the internet of things setup. Indoor activity recognition deals with two sub-problems: Human activity and household activity recognition. Household activity recognition includes the recognition of electrical appliances and their heat radiation with the help of thermal images. A FLIR ONE PRO camera is used to capture RGB-thermal image pairs for a scene. Duration and pattern of activities are also determined using an iterative algorithm, to explore kitchen safety situations. For more accurate monitoring of hazardous events such as stove gas leakage, a 3D reconstruction approach is proposed to determine the temperature of all points in the 3D space of a scene. The 3D thermal model is obtained using the stereo RGB and thermal images for a particular scene. Accurate results are observed for activity detection, and a significant improvement in the temperature estimation is recorded in the 3D thermal model compared to the 2D thermal image. Results from this research can find applications in home automation, heat automation in smart homes, and energy management in residential spaces.

scholarly journals Facial UV photo imaging for skin pigmentation assessment using conditional generative adversarial networks

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Kaname Kojima ◽  
Kosuke Shido ◽  
Gen Tamiya ◽  
Kenshi Yamasaki ◽  
Kengo Kinoshita ◽  
...  
Keyword(s):  
Skin Pigmentation ◽  
Generative Adversarial Networks ◽  
Pigment Spot ◽  
Detection Analysis ◽  
Adversarial Networks ◽  
Skin Cancers ◽  
Uv Photography ◽  
Highly Correlated ◽  
Image Pairs ◽  
Skin Damages

AbstractSkin pigmentation is associated with skin damages and skin cancers, and ultraviolet (UV) photography is used as a minimally invasive mean for the assessment of pigmentation. Since UV photography equipment is not usually available in general practice, technologies emphasizing pigmentation in color photo images are desired for daily care. We propose a new method using conditional generative adversarial networks, named UV-photo Net, to generate synthetic UV images from color photo images. Evaluations using color and UV photo image pairs taken by a UV photography system demonstrated that pigment spots were well reproduced in synthetic UV images by UV-photo Net, and some of the reproduced pigment spots were difficult to be recognized in color photo images. In the pigment spot detection analysis, the rate of pigment spot areas in cheek regions for synthetic UV images was highly correlated with the rate for UV photo images (Pearson’s correlation coefficient 0.92). We also demonstrated that UV-photo Net was effective for floating up pigment spots for photo images taken by a smartphone camera. UV-photo Net enables an easy assessment of pigmentation from color photo images and will promote self-care of skin damages and early signs of skin cancers for preventive medicine.

NSCT and focus measure optimization based multi-focus image fusion

2021 ◽  
pp. 1-13
Author(s):  
N. Aishwarya ◽  
C. BennilaThangammal ◽  
N.G. Praveena
Keyword(s):  
Simulation Analysis ◽  
Structural Information ◽  
Research Area ◽  
Processing Technique ◽  
The Novel ◽  
Phase Congruency ◽  
Focus Measure ◽  
Fused Image ◽  
Focus Image ◽  
Image Pairs

Getting a complete description of scene with all the relevant objects in focus is a hot research area in surveillance, medicine and machine vision applications. In this work, transform based fusion method called as NSCT-FMO, is introduced to integrate the image pairs having different focus features. The NSCT-FMO approach basically contains four steps. Initially, the NSCT is applied on the input images to acquire the approximation and detailed structural information. Then, the approximation sub band coefficients are merged by employing the novel Focus Measure Optimization (FMO) approach. Next, the detailed sub-images are combined using Phase Congruency (PC). Finally, an inverse NSCT operation is conducted on synthesized sub images to obtain the initial synthesized image. To optimize the initial fused image, an initial decision map is first constructed and morphological post-processing technique is applied to get the final map. With the help of resultant map, the final synthesized output is produced by the selection of focused pixels from input images. Simulation analysis show that the NSCT-FMO approach achieves fair results as compared to traditional MST based methods both in qualitative and quantitative assessments.

Sign in / Sign up

Export Citation Format

Share Document