Spatio-Temporal Data Fusion for Satellite Images Using Hopfield Neural Network

Spatio-temporal data fusion refers to the technique of combining high temporal resolution from coarse satellite images and high spatial resolution from fine satellite images. However, data availability remains a major limitation in algorithm development. Existing spatio-temporal data fusion algorithms require at least one known image pair between the fine and coarse resolution image. However, data which come from two different satellite platforms do not necessarily have an overlap in their overpass times, hence restricting the application of spatio-temporal data fusion. In this paper, a new algorithm named Hopfield Neural Network SPatio-tempOral daTa fusion model (HNN-SPOT) is developed by utilizing the optimization concept in the Hopfield neural network (HNN) for spatio-temporal image fusion. The algorithm derives a synthesized fine resolution image from a coarse spatial resolution satellite image (similar to downscaling), with the use of one fine resolution image taken on an arbitrary date and one coarse image taken on a predicted date. The HNN-SPOT particularly addresses the problem when the fine resolution and coarse resolution images are acquired from different satellite overpass times over the same geographic extent. Both simulated datasets and real datasets over Hong Kong and Australia have been used in the evaluation of HNN-SPOT. Results showed that HNN-SPOT was comparable with an existing fusion algorithm, the spatial and temporal adaptive reflectance fusion model (STARFM). HNN-SPOT assumes consistent spatial structure for the target area between the date of data acquisition and the prediction date. Therefore, it is more applicable to geographical areas with little or no land cover change. It is shown that HNN-SPOT can produce accurate fusion results with >90% of correlation coefficient over consistent land covers. For areas that have undergone land cover changes, HNN-SPOT can still produce a prediction about the outlines and the tone of the features, if they are large enough to be recorded in the coarse resolution image at the prediction date. HNN-SPOT provides a relatively new approach in spatio-temporal data fusion, and further improvements can be made by modifying or adding new goals and constraints in its HNN architecture. Owing to its lower demand for data prerequisites, HNN-SPOT is expected to increase the applicability of fine-scale applications in remote sensing, such as environmental modeling and monitoring.

Download Full-text

A Simple Spatio–Temporal Data Fusion Method Based on Linear Regression Coefficient Compensation

Remote Sensing ◽

10.3390/rs12233900 ◽

2020 ◽

Vol 12 (23) ◽

pp. 3900

Author(s):

Bingxin Bai ◽

Yumin Tan ◽

Gennadii Donchyts ◽

Arjen Haag ◽

Albrecht Weerts

Keyword(s):

Linear Regression ◽

Data Fusion ◽

Spatial Resolution ◽

Temporal Resolution ◽

Fusion Method ◽

Temporal Data ◽

Resolution Image ◽

Coarse Resolution ◽

Fine Spatial Resolution ◽

Spatio Temporal

High spatio–temporal resolution remote sensing images are of great significance in the dynamic monitoring of the Earth’s surface. However, due to cloud contamination and the hardware limitations of sensors, it is difficult to obtain image sequences with both high spatial and temporal resolution. Combining coarse resolution images, such as the moderate resolution imaging spectroradiometer (MODIS), with fine spatial resolution images, such as Landsat or Sentinel-2, has become a popular means to solve this problem. In this paper, we propose a simple and efficient enhanced linear regression spatio–temporal fusion method (ELRFM), which uses fine spatial resolution images acquired at two reference dates to establish a linear regression model for each pixel and each band between the image reflectance and the acquisition date. The obtained regression coefficients are used to help allocate the residual error between the real coarse resolution image and the simulated coarse resolution image upscaled by the high spatial resolution result of the linear prediction. The developed method consists of four steps: (1) linear regression (LR), (2) residual calculation, (3) distribution of the residual and (4) singular value correction. The proposed method was tested in different areas and using different sensors. The results show that, compared to the spatial and temporal adaptive reflectance fusion model (STARFM) and the flexible spatio–temporal data fusion (FSDAF) method, the ELRFM performs better in capturing small feature changes at the fine image scale and has high prediction accuracy. For example, in the red band, the proposed method has the lowest root mean square error (RMSE) (ELRFM: 0.0123 vs. STARFM: 0.0217 vs. FSDAF: 0.0224 vs. LR: 0.0221). Furthermore, the lightweight algorithm design and calculations based on the Google Earth Engine make the proposed method computationally less expensive than the STARFM and FSDAF.

Download Full-text

Spatio-temporal data fusion for fine-resolution subsidence estimation

Environmental Modelling & Software ◽

10.1016/j.envsoft.2021.104975 ◽

2021 ◽

Vol 137 ◽

pp. 104975

Author(s):

Hone-Jay Chu ◽

Muhammad Zeeshan Ali ◽

Thomas J. Burbey

Keyword(s):

Data Fusion ◽

Temporal Data ◽

Spatio Temporal ◽

Fine Resolution

Download Full-text

A Spatio-Temporal Data Fusion Model for Generating NDVI Time Series in Heterogeneous Regions

Remote Sensing ◽

10.3390/rs9111125 ◽

2017 ◽

Vol 9 (11) ◽

pp. 1125 ◽

Cited By ~ 13

Author(s):

Chunhua Liao ◽

Jinfei Wang ◽

Ian Pritchard ◽

Jiangui Liu ◽

Jiali Shang

Keyword(s):

Time Series ◽

Data Fusion ◽

Temporal Data ◽

Fusion Model ◽

Ndvi Time Series ◽

Spatio Temporal

Download Full-text

Spatiotemporal Fusion of Formosat-2 and Landsat-8 Satellite Images: A Comparison of “Super Resolution-Then-Blend” and “Blend-Then-Super Resolution” Approaches

Remote Sensing ◽

10.3390/rs13040606 ◽

2021 ◽

Vol 13 (4) ◽

pp. 606

Author(s):

Tee-Ann Teo ◽

Yu-Ju Fu

Keyword(s):

Satellite Images ◽

Traditional Approach ◽

Spectral Band ◽

Super Resolution ◽

High Temporal Resolution ◽

Landsat 8 ◽

Fusion Model ◽

Image Blending ◽

Fine Resolution

The spatiotemporal fusion technique has the advantages of generating time-series images with high-spatial and high-temporal resolution from coarse-resolution to fine-resolution images. A hybrid fusion method that integrates image blending (i.e., spatial and temporal adaptive reflectance fusion model, STARFM) and super-resolution (i.e., very deep super resolution, VDSR) techniques for the spatiotemporal fusion of 8 m Formosat-2 and 30 m Landsat-8 satellite images is proposed. Two different fusion approaches, namely Blend-then-Super-Resolution and Super-Resolution (SR)-then-Blend, were developed to improve the results of spatiotemporal fusion. The SR-then-Blend approach performs SR before image blending. The SR refines the image resampling stage on generating the same pixel-size of coarse- and fine-resolution images. The Blend-then-SR approach is aimed at refining the spatial details after image blending. Several quality indices were used to analyze the quality of the different fusion approaches. Experimental results showed that the performance of the hybrid method is slightly better than the traditional approach. Images obtained using SR-then-Blend are more similar to the real observed images compared with images acquired using Blend-then-SR. The overall mean bias of SR-then-Blend was 4% lower than Blend-then-SR, and nearly 3% improvement for overall standard deviation in SR-B. The VDSR technique reduces the systematic deviation in spectral band between Formosat-2 and Landsat-8 satellite images. The integration of STARFM and the VDSR model is useful for improving the quality of spatiotemporal fusion.

Download Full-text

Photovoltaic Power Forecasting: Assessment of the Impact of Multiple Sources of Spatio-Temporal Data on Forecast Accuracy

Energies ◽

10.3390/en14051432 ◽

2021 ◽

Vol 14 (5) ◽

pp. 1432

Author(s):

Xwégnon Ghislain Agoua ◽

Robin Girard ◽

Georges Kariniotakis

Keyword(s):

Satellite Images ◽

Forecast Accuracy ◽

Absolute Error ◽

Data Sources ◽

Test Case ◽

Temporal Data ◽

Multiple Sources ◽

Forecasting Error ◽

Spatio Temporal ◽

The Impact

The efficient integration of photovoltaic (PV) production in energy systems is conditioned by the capacity to anticipate its variability, that is, the capacity to provide accurate forecasts. From the classical forecasting methods in the state of the art dealing with a single power plant, the focus has moved in recent years to spatio-temporal approaches, where geographically dispersed data are used as input to improve forecasts of a site for the horizons up to 6 h ahead. These spatio-temporal approaches provide different performances according to the data sources available but the question of the impact of each source on the actual forecasting performance is still not evaluated. In this paper, we propose a flexible spatio-temporal model to generate PV production forecasts for horizons up to 6 h ahead and we use this model to evaluate the effect of different spatial and temporal data sources on the accuracy of the forecasts. The sources considered are measurements from neighboring PV plants, local meteorological stations, Numerical Weather Predictions, and satellite images. The evaluation of the performance is carried out using a real-world test case featuring a high number of 136 PV plants. The forecasting error has been evaluated for each data source using the Mean Absolute Error and Root Mean Square Error. The results show that neighboring PV plants help to achieve around 10% reduction in forecasting error for the first three hours, followed by satellite images which help to gain an additional 3% all over the horizons up to 6 h ahead. The NWP data show no improvement for horizons up to 6 h but is essential for greater horizons.

Download Full-text