scholarly journals AUTOMATIC LARGE-SCALE 3D BUILDING SHAPE REFINEMENT USING CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS

2018 ◽  
Vol XLII-2 ◽  
pp. 103-108 ◽  
Author(s):  
K. Bittner ◽  
P. d’Angelo ◽  
M. Körner ◽  
P. Reinartz
Keyword(s):  
Remote Sensing ◽  
Urban Areas ◽  
Large Scale ◽  
Point Clouds ◽  
Research Area ◽  
Sar Interferometry ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Building Shape ◽  
Object Shapes

<p><strong>Abstract.</strong> Three-dimensional building reconstruction from remote sensing imagery is one of the most difficult and important 3D modeling problems for complex urban environments. The main data sources provided the digital representation of the Earths surface and related natural, cultural, and man-made objects of the urban areas in remote sensing are the <i>digital surface models (DSMs)</i>. The DSMs can be obtained either by <i>light detection and ranging (LIDAR)</i>, SAR interferometry or from stereo images. Our approach relies on automatic global 3D building shape refinement from stereo DSMs using deep learning techniques. This refinement is necessary as the DSMs, which are extracted from image matching point clouds, suffer from occlusions, outliers, and noise. Though most previous works have shown promising results for building modeling, this topic remains an open research area. We present a new methodology which not only generates images with continuous values representing the elevation models but, at the same time, enhances the 3D object shapes, buildings in our case. Mainly, we train a <i>conditional generative adversarial network (cGAN)</i> to generate accurate LIDAR-like DSM height images from the noisy stereo DSM input. The obtained results demonstrate the strong potential of creating large areas remote sensing depth images where the buildings exhibit better-quality shapes and roof forms.</p>

scholarly journals Remote Sensing Image Dataset Expansion Based on Generative Adversarial Networks with Modified Shuffle Attention

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4867
Author(s):  
Lu Chen ◽  
Hongjun Wang ◽  
Xianghao Meng
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Image Processing ◽  
Remote Sensing Image ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Evaluation Indexes ◽  
Adversarial Network ◽  
Remote Sensing Image Processing ◽  
Data Expansion

With the development of science and technology, neural networks, as an effective tool in image processing, play an important role in gradual remote-sensing image-processing. However, the training of neural networks requires a large sample database. Therefore, expanding datasets with limited samples has gradually become a research hotspot. The emergence of the generative adversarial network (GAN) provides new ideas for data expansion. Traditional GANs either require a large number of input data, or lack detail in the pictures generated. In this paper, we modify a shuffle attention network and introduce it into GAN to generate higher quality pictures with limited inputs. In addition, we improved the existing resize method and proposed an equal stretch resize method to solve the problem of image distortion caused by different input sizes. In the experiment, we also embed the newly proposed coordinate attention (CA) module into the backbone network as a control test. Qualitative indexes and six quantitative evaluation indexes were used to evaluate the experimental results, which show that, compared with other GANs used for picture generation, the modified Shuffle Attention GAN proposed in this paper can generate more refined and high-quality diversified aircraft pictures with more detailed features of the object under limited datasets.

scholarly journals Quantitative Recognition and Characteristic Analysis of Production-Living-Ecological Space Evolution for Five Resource-Based Cities: Zululand, Xuzhou, Lota, Surf Coast and Ruhr

2021 ◽  
Vol 13 (8) ◽  
pp. 1563
Author(s):  
Yuanyuan Tao ◽  
Qianxin Wang
Keyword(s):  
Remote Sensing ◽  
Urban Areas ◽  
Large Scale ◽  
Remote Sensing Image ◽  
Data Availability ◽  
Function Evaluation ◽  
Characteristic Analysis ◽  
Accurate Identification ◽  
Living Space ◽  
Evolution Characteristics

The accurate identification of PLES changes and the discovery of their evolution characteristics is a key issue to improve the ability of the sustainable development for resource-based urban areas. However, the current methods are unsuitable for the long-term and large-scale PLES investigation. In this study, a modified method of PLES recognition is proposed based on the remote sensing image classification and land function evaluation technology. A multi-dimensional index system is constructed, which can provide a comprehensive evaluation for PLES evolution characteristics. For validation of the proposed methods, the remote sensing image, geographic information, and socio-economic data of five resource-based urbans (Zululand in South Africa, Xuzhou in China, Lota in Chile, Surf Coast in Australia, and Ruhr in Germany) from 1975 to 2020 are collected and tested. The results show that the data availability and calculation efficiency are significantly improved by the proposed method, and the recognition precision is better than 87% (Kappa coefficient). Furthermore, the PLES evolution characteristics show obvious differences at the different urban development stages. The expansions of production, living, and ecological space are fastest at the mining, the initial, and the middle ecological restoration stages, respectively. However, the expansion of living space is always increasing at any stage, and the disorder expansion of living space has led to the decrease of integration of production and ecological spaces. Therefore, the active polices should be formulated to guide the transformation of the living space expansion from jumping-type and spreading-type to filling-type, and the renovation of abandoned industrial and mining lands should be encouraged.

scholarly journals TWIST-GAN: Towards Wavelet Transform and Transferred GAN for Spatio-Temporal Single Image Super Resolution

2021 ◽  
Vol 12 (6) ◽  
pp. 1-20
Author(s):  
Fayaz Ali Dharejo ◽  
Farah Deeba ◽  
Yuanchun Zhou ◽  
Bhagwan Das ◽  
Munsif Ali Jatoi ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Super Resolution ◽  
Generative Adversarial Networks ◽  
Single Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Image Super Resolution ◽  
Spatio Temporal ◽  
Single Image Super Resolution

Single Image Super-resolution (SISR) produces high-resolution images with fine spatial resolutions from a remotely sensed image with low spatial resolution. Recently, deep learning and generative adversarial networks (GANs) have made breakthroughs for the challenging task of single image super-resolution (SISR) . However, the generated image still suffers from undesirable artifacts such as the absence of texture-feature representation and high-frequency information. We propose a frequency domain-based spatio-temporal remote sensing single image super-resolution technique to reconstruct the HR image combined with generative adversarial networks (GANs) on various frequency bands (TWIST-GAN). We have introduced a new method incorporating Wavelet Transform (WT) characteristics and transferred generative adversarial network. The LR image has been split into various frequency bands by using the WT, whereas the transfer generative adversarial network predicts high-frequency components via a proposed architecture. Finally, the inverse transfer of wavelets produces a reconstructed image with super-resolution. The model is first trained on an external DIV2 K dataset and validated with the UC Merced Landsat remote sensing dataset and Set14 with each image size of 256 × 256. Following that, transferred GANs are used to process spatio-temporal remote sensing images in order to minimize computation cost differences and improve texture information. The findings are compared qualitatively and qualitatively with the current state-of-art approaches. In addition, we saved about 43% of the GPU memory during training and accelerated the execution of our simplified version by eliminating batch normalization layers.

scholarly journals Learning Representations of Inorganic Materials from Generative Adversarial Networks

Symmetry ◽  
2020 ◽  
Vol 12 (11) ◽  
pp. 1889
Author(s):  
Tiantian Hu ◽  
Hui Song ◽  
Tao Jiang ◽  
Shaobo Li
Keyword(s):  
Large Scale ◽  
Inorganic Materials ◽  
Molecular Composition ◽  
Materials Characterization ◽  
Generative Adversarial Networks ◽  
Feature Engineering ◽  
Material Research ◽  
Generative Adversarial Network ◽  
Characterization Of Materials

The two most important aspects of material research using deep learning (DL) or machine learning (ML) are the characteristics of materials data and learning algorithms, where the proper characterization of materials data is essential for generating accurate models. At present, the characterization of materials based on the molecular composition includes some methods based on feature engineering, such as Magpie and One-hot. Although these characterization methods have achieved significant results in materials research, these methods based on feature engineering cannot guarantee the integrity of materials characterization. One possible approach is to learn the materials characterization via neural networks using the chemical knowledge and implicit composition rules shown in large-scale known materials. This article chooses an adversarial method to learn the composition of atoms using the Generative Adversarial Network (GAN), which makes sense for data symmetry. The total loss value of the discriminator on the test set is reduced from 4.1e13 to 0.3194, indicating that the designed GAN network can well capture the combination of atoms in real materials. We then use the trained discriminator weights for material characterization and predict bandgap, formation energy, critical temperature (Tc) of superconductors on the Open Quantum Materials Database (OQMD), Materials Project (MP), and SuperCond datasets. Experiments show that when using the same predictive model, our proposed method performs better than One-hot and Magpie. This article provides an effective method for characterizing materials based on molecular composition in addition to Magpie, One-hot, etc. In addition, the generator learned in this study generates hypothetical materials with the same distribution as known materials, and these hypotheses can be used as a source for new material discovery.

scholarly journals Small Object Detection in Remote Sensing Images Based on Super-Resolution with Auxiliary Generative Adversarial Networks

2020 ◽  
Vol 12 (19) ◽  
pp. 3152
Author(s):  
Luc Courtrai ◽  
Minh-Tan Pham ◽  
Sébastien Lefèvre
Keyword(s):  
Remote Sensing ◽  
Object Detection ◽  
Network Architecture ◽  
Super Resolution ◽  
Remote Sensing Data ◽  
Generative Adversarial Networks ◽  
Small Object ◽  
Remote Sensing Images ◽  
Generative Adversarial Network ◽  
Small Object Detection

This article tackles the problem of detecting small objects in satellite or aerial remote sensing images by relying on super-resolution to increase image spatial resolution, thus the size and details of objects to be detected. We show how to improve the super-resolution framework starting from the learning of a generative adversarial network (GAN) based on residual blocks and then its integration into a cycle model. Furthermore, by adding to the framework an auxiliary network tailored for object detection, we considerably improve the learning and the quality of our final super-resolution architecture, and more importantly increase the object detection performance. Besides the improvement dedicated to the network architecture, we also focus on the training of super-resolution on target objects, leading to an object-focused approach. Furthermore, the proposed strategies do not depend on the choice of a baseline super-resolution framework, hence could be adopted for current and future state-of-the-art models. Our experimental study on small vehicle detection in remote sensing data conducted on both aerial and satellite images (i.e., ISPRS Potsdam and xView datasets) confirms the effectiveness of the improved super-resolution methods to assist with the small object detection tasks.

scholarly journals Mapping Urban Areas Using a Combination of Remote Sensing and Geolocation Data

2019 ◽  
Vol 11 (12) ◽  
pp. 1470 ◽  
Author(s):  
Nan Xia ◽  
Liang Cheng ◽  
ManChun Li
Keyword(s):  
Remote Sensing ◽  
Land Surface ◽  
Urban Areas ◽  
Large Scale ◽  
High Spatial Resolution ◽  
Human Life ◽  
Human Movement ◽  
Night Time ◽  
Urban Mapping ◽  
The City

Urban areas are essential to daily human life; however, the urbanization process also brings about problems, especially in China. Urban mapping at large scales relies heavily on remote sensing (RS) data, which cannot capture socioeconomic features well. Geolocation datasets contain patterns of human movement, which are closely related to the extent of urbanization. However, the integration of RS and geolocation data for urban mapping is performed mostly at the city level or finer scales due to the limitations of geolocation datasets. Tencent provides a large-scale location request density (LRD) dataset with a finer temporal resolution, and makes large-scale urban mapping possible. The objective of this study is to combine multi-source features from RS and geolocation datasets to extract information on urban areas at large scales, including night-time lights, vegetation cover, land surface temperature, population density, LRD, accessibility, and road networks. The random forest (RF) classifier is introduced to deal with these high-dimension features on a 0.01 degree grid. High spatial resolution land cover (LC) products and the normalized difference built-up index from Landsat are used to label all of the samples. The RF prediction results are evaluated using validation samples and compared with LC products for four typical cities. The results show that night-time lights and LRD features contributed the most to the urban prediction results. A total of 176,266 km2 of urban areas in China were extracted using the RF classifier, with an overall accuracy of 90.79% and a kappa coefficient of 0.790. Compared with existing LC products, our results are more consistent with the manually interpreted urban boundaries in the four selected cities. Our results reveal the potential of Tencent LRD data for the extraction of large-scale urban areas, and the reliability of the RF classifier based on a combination of RS and geolocation data.

scholarly journals TUM-MLS-2016: An Annotated Mobile LiDAR Dataset of the TUM City Campus for Semantic Point Cloud Interpretation in Urban Areas

2020 ◽  
Vol 12 (11) ◽  
pp. 1875 ◽  
Author(s):  
Jingwei Zhu ◽  
Joachim Gehrung ◽  
Rong Huang ◽  
Björn Borgmann ◽  
Zhenghao Sun ◽  
...  
Keyword(s):  
Test Data ◽  
Urban Areas ◽  
Point Cloud ◽  
Large Scale ◽  
Point Clouds ◽  
Semantic Interpretation ◽  
3D Point Clouds ◽  
Semantic Labeling ◽  
Benchmark Datasets ◽  
Semantic Point

In the past decade, a vast amount of strategies, methods, and algorithms have been developed to explore the semantic interpretation of 3D point clouds for extracting desirable information. To assess the performance of the developed algorithms or methods, public standard benchmark datasets should invariably be introduced and used, which serve as an indicator and ruler in the evaluation and comparison. In this work, we introduce and present large-scale Mobile LiDAR point clouds acquired at the city campus of the Technical University of Munich, which have been manually annotated and can be used for the evaluation of related algorithms and methods for semantic point cloud interpretation. We created three datasets from a measurement campaign conducted in April 2016, including a benchmark dataset for semantic labeling, test data for instance segmentation, and test data for annotated single 360 ° laser scans. These datasets cover an urban area of approximately 1 km long roadways and include more than 40 million annotated points with eight classes of objects labeled. Moreover, experiments were carried out with results from several baseline methods compared and analyzed, revealing the quality of this dataset and its effectiveness when using it for performance evaluation.

scholarly journals COVID-GAN+: Estimating Human Mobility Responses to COVID-19 through Spatio-temporal Generative Adversarial Networks with Enhanced Features

2022 ◽  
Vol 13 (2) ◽  
pp. 1-23
Author(s):  
Han Bao ◽  
Xun Zhou ◽  
Yiqun Xie ◽  
Yingxue Zhang ◽  
Yanhua Li
Keyword(s):  
Spatial Heterogeneity ◽  
Real World ◽  
Large Scale ◽  
Census Data ◽  
Human Mobility ◽  
Training Data ◽  
Estimation Accuracy ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Spatio Temporal

Estimating human mobility responses to the large-scale spreading of the COVID-19 pandemic is crucial, since its significance guides policymakers to give Non-pharmaceutical Interventions, such as closure or reopening of businesses. It is challenging to model due to complex social contexts and limited training data. Recently, we proposed a conditional generative adversarial network (COVID-GAN) to estimate human mobility response under a set of social and policy conditions integrated from multiple data sources. Although COVID-GAN achieves a good average estimation accuracy under real-world conditions, it produces higher errors in certain regions due to the presence of spatial heterogeneity and outliers. To address these issues, in this article, we extend our prior work by introducing a new spatio-temporal deep generative model, namely, COVID-GAN+. COVID-GAN+ deals with the spatial heterogeneity issue by introducing a new spatial feature layer that utilizes the local Moran statistic to model the spatial heterogeneity strength in the data. In addition, we redesign the training objective to learn the estimated mobility changes from historical average levels to mitigate the effects of spatial outliers. We perform comprehensive evaluations using urban mobility data derived from cell phone records and census data. Results show that COVID-GAN+ can better approximate real-world human mobility responses than prior methods, including COVID-GAN.

scholarly journals A REVIEW ON SPATIAL QUALITY ASSESSMENT METHODS FOR EVALUATION OF PAN-SHARPENED SATELLITE IMAGERY

2019 ◽  
Vol XLII-4/W18 ◽  
pp. 255-261
Author(s):  
F. Dadras Javan ◽  
F. Samadzadegan ◽  
S. Mehravar ◽  
A. Toosi
Keyword(s):  
Remote Sensing ◽  
Quality Assessment ◽  
Satellite Imagery ◽  
Urban Areas ◽  
Large Scale ◽  
Spatial Distortion ◽  
Sensing Applications ◽  
Fused Image ◽  
Spatial Quality

Abstract. Nowadays, high-resolution fused satellite imagery is widely used in multiple remote sensing applications. Although the spectral quality of pan-sharpened images plays an important role in many applications, spatial quality becomes more important in numerous cases. The high spatial quality of the fused image is essential for extraction, identification and reconstruction of significant image objects, and will result in producing high-quality large scale maps especially in the urban areas. This paper introduces the most sensitive and effective methods in detecting the spatial distortion of fused images by implementing a number of spatial quality assessment indices that are utilized in the field of remote sensing and image processing. In this regard, in order to recognize the ability of quality assessment indices for detecting the spatial distortion quantity of fused images, input images of the fusion process are affected by some intentional spatial distortions based on non-registration error. The capabilities of the investigated metrics are evaluated on four different fused images derived from Ikonos and WorldView-2 initial images. Achieved results obviously explicate that two methods namely Edge Variance Distortion and the spatial component of QNR metric called Ds are more sensitive and responsive to the imported errors.

Sign in / Sign up

Export Citation Format

Share Document