A Novel Tilt Correction Technique for Irradiance Sensors and Spectrometers On-Board Unmanned Aerial Vehicles

In unstable atmospheric conditions, using on-board irradiance sensors is one of the only robust methods to convert unmanned aerial vehicle (UAV)-based optical remote sensing data to reflectance factors. Normally, such sensors experience significant errors due to tilting of the UAV, if not installed on a stabilizing gimbal. Unfortunately, such gimbals of sufficient accuracy are heavy, cumbersome, and cannot be installed on all UAV platforms. In this paper, we present the FGI Aerial Image Reference System (FGI AIRS) developed at the Finnish Geospatial Research Institute (FGI) and a novel method for optical and mathematical tilt correction of the irradiance measurements. The FGI AIRS is a sensor unit for UAVs that provides the irradiance spectrum, Real Time Kinematic (RTK)/Post Processed Kinematic (PPK) GNSS position, and orientation for the attached cameras. The FGI AIRS processes the reference data in real time for each acquired image and can send it to an on-board or on-cloud processing unit. The novel correction method is based on three RGB photodiodes that are tilted 10° in opposite directions. These photodiodes sample the irradiance readings at different sensor tilts, from which reading of a virtual horizontal irradiance sensor is calculated. The FGI AIRS was tested, and the method was shown to allow on-board measurement of irradiance at an accuracy better than ±0.8% at UAV tilts up to 10° and ±1.2% at tilts up to 15°. In addition, the accuracy of FGI AIRS to produce reflectance-factor-calibrated aerial images was compared against the traditional methods. In the unstable weather conditions of the experiment, both the FGI AIRS and the on-ground spectrometer were able to produce radiometrically accurate and visually pleasing orthomosaics, while the reflectance reference panels and the on-board irradiance sensor without stabilization or tilt correction both failed to do so. The authors recommend the implementation of the proposed tilt correction method in all future UAV irradiance sensors if they are not to be installed on a gimbal.

Download Full-text

ANALYZING SPECTRAL CHARACTERISTICS OF SHADOW AREA FROM ADS-40 HIGH RADIOMETRIC RESOLUTION AERIAL IMAGES

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xli-b7-223-2016 ◽

2016 ◽

Vol XLI-B7 ◽

pp. 223-227 ◽

Cited By ~ 2

Author(s):

Yi-Ta Hsieh ◽

Shou-Tsung Wu ◽

Chaur-Tzuhn Chen ◽

Jan-Chang Chen

Keyword(s):

Remote Sensing ◽

Land Cover ◽

Vegetation Index ◽

Spectral Characteristics ◽

Aerial Images ◽

Aerial Image ◽

Optical Remote Sensing ◽

Shadow Area ◽

Vegetation Indexes ◽

Research Findings

The shadows in optical remote sensing images are regarded as image nuisances in numerous applications. The classification and interpretation of shadow area in a remote sensing image are a challenge, because of the reduction or total loss of spectral information in those areas. In recent years, airborne multispectral aerial image devices have been developed 12-bit or higher radiometric resolution data, including Leica ADS-40, Intergraph DMC. The increased radiometric resolution of digital imagery provides more radiometric details of potential use in classification or interpretation of land cover of shadow areas. Therefore, the objectives of this study are to analyze the spectral properties of the land cover in the shadow areas by ADS-40 high radiometric resolution aerial images, and to investigate the spectral and vegetation index differences between the various shadow and non-shadow land covers. According to research findings of spectral analysis of ADS-40 image: (i) The DN values in shadow area are much lower than in nonshadow area; (ii) DN values received from shadowed areas that will also be affected by different land cover, and it shows the possibility of land cover property retrieval as in nonshadow area; (iii) The DN values received from shadowed regions decrease in the visible band from short to long wavelengths due to scattering; (iv) The shadow area NIR of vegetation category also shows a strong reflection; (v) Generally, vegetation indexes (NDVI) still have utility to classify the vegetation and non-vegetation in shadow area. The spectral data of high radiometric resolution images (ADS-40) is potential for the extract land cover information of shadow areas.

Download Full-text

Real-time and automatic map stitching through aerial images from UAV

Journal of Physics Conference Series ◽

10.1088/1742-6596/2120/1/012025 ◽

2021 ◽

Vol 2120 (1) ◽

pp. 012025

Author(s):

J N Goh ◽

S K Phang ◽

W J Chew

Keyword(s):

Real Time ◽

Matrix Multiplication ◽

Image Features ◽

Aerial Images ◽

Aerial Image ◽

Process Time ◽

Image Stitching ◽

Singular Value Decomposition Method ◽

Value Decomposition ◽

Computational Resources

Abstract Real-time aerial map stitching through aerial images had been done through many different methods. One of the popular methods was a features-based algorithm to detect features and to match the features of two and more images to produce a map. There are several feature-based methods such as ORB, SIFT, SURF, KAZE, AKAZE and BRISK. These methods detect features and compute homography matrix from matched features to stitch images. The aim for this project is to further optimize the existing image stitching algorithm such that it will be possible to run in real-time as the UAV capture images while airborne. First, we propose to use a matrix multiplication method to replace a singular value decomposition method in the RANSAC algorithm. Next, we propose to change the workflow to detect the image features to increase the map stitching rate. The proposed algorithm was implemented and tested with an online aerial image dataset which contain 100 images with the resolution of 640 × 480. We have successfully achieved the result of 1.45 Hz update rate compared to original image stitching algorithm that runs at 0.69 Hz. The improvement shown in our proposed improved algorithm are more than two folds in terms of computational resources. The method introduced in this paper was successful speed up the process time for the program to process map stitching.

Download Full-text

TERRAIN AWARE IMAGE CLIPPING FOR REAL-TIME AERIAL MAPPING

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iv-1-61-2018 ◽

2018 ◽

Vol IV-1 ◽

pp. 61-68

Author(s):

D. Hein ◽

R. Berger

Keyword(s):

Real Time ◽

Image Data ◽

Aerial Images ◽

Aerial Image ◽

Sensing Applications ◽

Data Links ◽

Map Generation ◽

Aerial Mapping ◽

Using Data ◽

Terrain Elevation

Abstract. Many remote sensing applications demand for a fast and efficient way of generating orthophoto maps from raw aerial images. One prerequisite is direct georeferencing, which allows to geolocate aerial images to their geographic position on the earth’s surface. But this is only half the story. When dealing with a large quantity of highly overlapping images, a major challenge is to select the most suitable image parts in order to generate seamless aerial maps of the captured area. This paper proposes a method that quickly determines such an optimal (rectangular) section for each single aerial image, which in turn can be used for generating seamless aerial maps. Its key approach is to clip aerial images depending on their geometric intersections with a terrain elevation model of the captured area, which is why we call it terrain aware image clipping (TAC). The method has a modest computational footprint and is therefore applicable even for rather limited embedded vision systems. It can be applied for both, real-time aerial mapping applications using data links as well as for rapid map generation right after landing without any postprocessing step. Referring to real-time applications, this method also minimizes transmission of redundant image data. The proposed method has already been demonstrated in several search-and-rescue scenarios and real-time mapping applications using a broadband data link and diffent kinds of camera and carrier systems. Moreover, a patent for this technology is pending.

Download Full-text

Methodology of mapping of historical and cultural heritage objects by GIS technologies using archival cartographic and aerial materials

Modern achievements of geodesic science and industry ◽

10.33841/1819-1339-1-41-97-103 ◽

2021 ◽

Vol 41 (I) ◽

pp. 97-103

Author(s):

B. CHETVERIKOV ◽

◽

O. KHINTSITSKY ◽

I. KALYNYCH ◽

◽

...

Keyword(s):

Cultural Heritage ◽

Remote Sensing Data ◽

Aerial Images ◽

Aerial Image ◽

Heritage Sites ◽

Mercator Projection ◽

Protection Information ◽

On Line ◽

Cultural Heritage Objects ◽

Space Image

Aim. The purpose of the work is to process archival cartographic materials and remote sensing data for the interpretation of objects of historical and cultural heritage (OHCH) of Cherkasy, including those that have not been preserved. Method. One of the possible technological schemes for research is offered. According to her, the first step was to analyze the input data of the study, among which were: a map of Cherkasy in 1895 at a scale of 1:42000; German aerial image of 1944; a fragment of a space image of Cherkasy obtained from the GeoEye-1 satellite in 2018. Geometric correction of the input materials was performed in the Mercator projection and the WGS84 coordinate system, in which the transformed image was obtained. The next step was to vectorize the objects of historical and cultural heritage of Cherkasy, according to the list obtained on the city’s website. There are two types of objects: point and polygonal. When vectorizing polygonal objects, the historical boundaries were specified with the help of archival maps and aerial images. Special symbols have been developed for each of the types of historical and cultural heritage sites, according to the proposed classification. In addition, an attributive database of these objects was created, which had the following structure: number of the passport of object, the name of the object, the address of the OHCH, the number of the decision to take under protection, information about the OHCH. Also, the obtained vector data was exported to the exchange format with the extension kmz and an online version of the thematic map was created on the basis of the free GISFile resource. Results. As a result of the conducted researches, the thematic GIS of the objects of historical and cultural heritage of Cherkasy was created, which are plotted on the space image of high spatial resolution, obtained in 2018. An on-line version of the GIS of Cherkasy historical and cultural heritage sites has been created on the basis of the free GISFile cartographic service, with the possibility of analyzing the location of these objects and building optimal tourist routes. Scientific novelty. Possible algorithms for creating offline and on-line versions of thematic GIS are proposed. Practical value. The obtained results of mapping the objects of historical and cultural heritage of Cherkasy can be used by the structures of protection of objects of historical and cultural heritage of Cherkasy at the Ministry of Culture.

Download Full-text

FUSION OF MULTI-VIEW AND MULTI-SCALE AERIAL IMAGERY FOR REAL-TIME SITUATION AWARENESS APPLICATIONS

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprsarchives-xl-1-w4-201-2015 ◽

2015 ◽

Vol XL-1/W4 ◽

pp. 201-206 ◽

Cited By ~ 1

Author(s):

X. Zhuo ◽

F. Kurz ◽

P. Reinartz

Keyword(s):

Real Time ◽

Reference Frame ◽

Situation Awareness ◽

Large Scale ◽

Bundle Adjustment ◽

Aerial Imagery ◽

Aerial Images ◽

Aerial Image ◽

Main Concern ◽

Multi Scale

Manned aircraft has long been used for capturing large-scale aerial images, yet the high costs and weather dependence restrict its availability in emergency situations. In recent years, MAV (Micro Aerial Vehicle) emerged as a novel modality for aerial image acquisition. Its maneuverability and flexibility enable a rapid awareness of the scene of interest. Since these two platforms deliver scene information from different scale and different view, it makes sense to fuse these two types of complimentary imagery to achieve a quick, accurate and detailed description of the scene, which is the main concern of real-time situation awareness. This paper proposes a method to fuse multi-view and multi-scale aerial imagery by establishing a common reference frame. In particular, common features among MAV images and geo-referenced airplane images can be extracted by a scale invariant feature detector like SIFT. From the tie point of geo-referenced images we derive the coordinate of corresponding ground points, which are then utilized as ground control points in global bundle adjustment of MAV images. In this way, the MAV block is aligned to the reference frame. Experiment results show that this method can achieve fully automatic geo-referencing of MAV images even if GPS/IMU acquisition has dropouts, and the orientation accuracy is improved compared to the GPS/IMU based georeferencing. The concept for a subsequent 3D classification method is also described in this paper.

Download Full-text

PCNet: Cloud Detection in FY-3D True-Color Imagery Using Multi-Scale Pyramid Contextual Information

Remote Sensing ◽

10.3390/rs13183670 ◽

2021 ◽

Vol 13 (18) ◽

pp. 3670

Author(s):

Wangbin Li ◽

Kaimin Sun ◽

Zhuotong Du ◽

Xiuqing Hu ◽

Wenzhuo Li ◽

...

Keyword(s):

Remote Sensing ◽

Contextual Information ◽

Remote Sensing Data ◽

Optical Remote Sensing ◽

Atmospheric Conditions ◽

Cloud Detection ◽

Multi Scale ◽

Limited Applicability ◽

Residual Block ◽

True Color

Cloud, one of the poor atmospheric conditions, significantly reduces the usability of optical remote-sensing data and hampers follow-up applications. Thus, the identification of cloud remains a priority for various remote-sensing activities, such as product retrieval, land-use/cover classification, object detection, and especially for change detection. However, the complexity of clouds themselves make it difficult to detect thin clouds and small isolated clouds. To accurately detect clouds in satellite imagery, we propose a novel neural network named the Pyramid Contextual Network (PCNet). Considering the limited applicability of a regular convolution kernel, we employed a Dilated Residual Block (DRB) to extend the receptive field of the network, which contains a dilated convolution and residual connection. To improve the detection ability for thin clouds, the proposed new model, pyramid contextual block (PCB), was used to generate global information at different scales. FengYun-3D MERSI-II remote-sensing images covering China with 14,165 × 24,659 pixels, acquired on 17 July 2019, are processed to conduct cloud-detection experiments. Experimental results show that the overall precision rates of the trained network reach 97.1% and the overall recall rates reach 93.2%, which performs better both in quantity and quality than U-Net, UNet++, UNet3+, PSPNet and DeepLabV3+.

Download Full-text

ANALYZING SPECTRAL CHARACTERISTICS OF SHADOW AREA FROM ADS-40 HIGH RADIOMETRIC RESOLUTION AERIAL IMAGES

ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-archives-xli-b7-223-2016 ◽

2016 ◽

Vol XLI-B7 ◽

pp. 223-227

Author(s):

Yi-Ta Hsieh ◽

Shou-Tsung Wu ◽

Chaur-Tzuhn Chen ◽

Jan-Chang Chen

Keyword(s):

Remote Sensing ◽

Land Cover ◽

Vegetation Index ◽

Spectral Characteristics ◽

Aerial Images ◽

Aerial Image ◽

Optical Remote Sensing ◽

Shadow Area ◽

Vegetation Indexes ◽

Research Findings

Download Full-text

Real-time Detection of Aortic Valve in Echocardiography using Convolutional Neural Networks

Current Medical Imaging Formerly Current Medical Imaging Reviews ◽

10.2174/1573405615666190114151255 ◽

2020 ◽

Vol 16 (5) ◽

pp. 584-591 ◽

Cited By ~ 1

Author(s):

Muhammad Hanif Ahmad Nizar ◽

Chow Khuen Chan ◽

Azira Khalil ◽

Ahmad Khairuddin Mohamed Yusof ◽

Khin Wee Lai

Keyword(s):

Neural Network ◽

Neural Networks ◽

Heart Disease ◽

Aortic Valve ◽

Real Time ◽

Convolutional Neural Networks ◽

Valvular Heart Disease ◽

Detection System ◽

Processing Unit ◽

Real Time Detection

Background: Valvular heart disease is a serious disease leading to mortality and increasing medical care cost. The aortic valve is the most common valve affected by this disease. Doctors rely on echocardiogram for diagnosing and evaluating valvular heart disease. However, the images from echocardiogram are poor in comparison to Computerized Tomography and Magnetic Resonance Imaging scan. This study proposes the development of Convolutional Neural Networks (CNN) that can function optimally during a live echocardiographic examination for detection of the aortic valve. An automated detection system in an echocardiogram will improve the accuracy of medical diagnosis and can provide further medical analysis from the resulting detection. Methods: Two detection architectures, Single Shot Multibox Detector (SSD) and Faster Regional based Convolutional Neural Network (R-CNN) with various feature extractors were trained on echocardiography images from 33 patients. Thereafter, the models were tested on 10 echocardiography videos. Results: Faster R-CNN Inception v2 had shown the highest accuracy (98.6%) followed closely by SSD Mobilenet v2. In terms of speed, SSD Mobilenet v2 resulted in a loss of 46.81% in framesper- second (fps) during real-time detection but managed to perform better than the other neural network models. Additionally, SSD Mobilenet v2 used the least amount of Graphic Processing Unit (GPU) but the Central Processing Unit (CPU) usage was relatively similar throughout all models. Conclusion: Our findings provide a foundation for implementing a convolutional detection system to echocardiography for medical purposes.

Download Full-text

Fig Plant Segmentation from Aerial Images Using a Deep Convolutional Encoder-Decoder Network

Remote Sensing ◽

10.3390/rs11101157 ◽

2019 ◽

Vol 11 (10) ◽

pp. 1157 ◽

Cited By ~ 8

Author(s):

Jorge Fuentes-Pacheco ◽

Juan Torres-Olivares ◽

Edgar Roman-Rangel ◽

Salvador Cervantes ◽

Porfirio Juarez-Lopez ◽

...

Keyword(s):

Precision Agriculture ◽

Image Data ◽

Ground Truth ◽

Aerial Images ◽

Aerial Image ◽

Data Set ◽

Visual Appearance ◽

Aerial Robots ◽

Lighting Conditions ◽

Convolutional Encoder

Crop segmentation is an important task in Precision Agriculture, where the use of aerial robots with an on-board camera has contributed to the development of new solution alternatives. We address the problem of fig plant segmentation in top-view RGB (Red-Green-Blue) images of a crop grown under open-field difficult circumstances of complex lighting conditions and non-ideal crop maintenance practices defined by local farmers. We present a Convolutional Neural Network (CNN) with an encoder-decoder architecture that classifies each pixel as crop or non-crop using only raw colour images as input. Our approach achieves a mean accuracy of 93.85% despite the complexity of the background and a highly variable visual appearance of the leaves. We make available our CNN code to the research community, as well as the aerial image data set and a hand-made ground truth segmentation with pixel precision to facilitate the comparison among different algorithms.

Download Full-text

An Improved Segmentation Method for Automatic Mapping of Cone Karst from Remote Sensing Data Based on DeepLab V3+ Model

Remote Sensing ◽

10.3390/rs13030441 ◽

2021 ◽

Vol 13 (3) ◽

pp. 441

Author(s):

Han Fu ◽

Bihong Fu ◽

Pilong Shi

Keyword(s):

Remote Sensing ◽

Remote Sensing Data ◽

Guizhou Province ◽

Training Dataset ◽

Optical Remote Sensing ◽

Remote Sensing Images ◽

Karst Landscape ◽

Heritage Site ◽

Training Samples ◽

Input Module

The South China Karst, a United Nations Educational, Scientific and Cultural Organization (UNESCO) natural heritage site, is one of the world’s most spectacular examples of humid tropical to subtropical karst landscapes. The Libo cone karst in the southern Guizhou Province is considered as the world reference site for these types of karst, forming a distinctive and beautiful landscape. Geomorphic information and spatial distribution of cone karst is essential for conservation and management for Libo heritage site. In this study, a deep learning (DL) method based on DeepLab V3+ network was proposed to document the cone karst landscape in Libo by multi-source data, including optical remote sensing images and digital elevation model (DEM) data. The training samples were generated by using Landsat remote sensing images and their combination with satellite derived DEM data. Each group of training dataset contains 898 samples. The input module of DeepLab V3+ network was improved to accept four-channel input data, i.e., combination of Landsat RGB images and DEM data. Our results suggest that the mean intersection over union (MIoU) using the four-channel data as training samples by a new DL-based pixel-level image segmentation approach is the highest, which can reach 95.5%. The proposed method can accomplish automatic extraction of cone karst landscape by self-learning of deep neural network, and therefore it can also provide a powerful and automatic tool for documenting other type of geological landscapes worldwide.

Download Full-text