scholarly journals SCIDB BASED FRAMEWORK FOR STORAGE AND ANALYSIS OF REMOTE SENSING BIG DATA

Author(s):  
A. Joshi ◽  
E. Pebesma ◽  
R. Henriques ◽  
M. Appel

Abstract. Earth observation data of large part of the world is available at different temporal, spectral and spatial resolution. These data can be termed as big data as they fulfil the criteria of 3 Vs of big data: Volume, Velocity and Variety. The size of image in archives are multiple petabyte size, the size is growing continuously and the data have varied resolution and usages. These big data have variety of applications including climate change study, forestry application, agricultural application and urban planning. However, these big data also possess challenge of data storage, management and high computational requirement for processing. The solution to this computational and data management requirements is database system with distributed storage and parallel computation.In this study SciDB, an array-based database is used to store, manage and process multitemporal satellite imagery. The major aim of this study is to develop SciDB based scalable solution to store and perform time series analysis on multi-temporal satellite imagery. Total 148 scene of landsat image of 10 years period between 2006 and 2016 were stored as SciDB array. The data was then retrieved, processed and visualized. This study provides solution for storage of big RS data and also provides workflow for time series analysis of remote sensing data no matter how large is the size.

2020 ◽  
Vol 12 (6) ◽  
pp. 972 ◽  
Author(s):  
Yinyi Cheng ◽  
Kefa Zhou ◽  
Jinlin Wang ◽  
Jining Yan

The arrival of the era of big data for Earth observation (EO) indicates that traditional data management models have been unable to meet the needs of remote sensing data in big data environments. With the launch of the first remote sensing satellite, the volume of remote sensing data has also been increasing, and traditional data storage methods have been unable to ensure the efficient management of large amounts of remote sensing data. Therefore, a professional remote sensing big data integration method is sorely needed. In recent years, the emergence of some new technical methods has provided effective solutions for multi-source remote sensing data integration. This paper proposes a multi-source remote sensing data integration framework based on a distributed management model. In this framework, the multi-source remote sensing data are partitioned by the proposed spatial segmentation indexing (SSI) model through spatial grid segmentation. The designed complete information description system, based on International Organization for Standardization (ISO) 19115, can explain multi-source remote sensing data in detail. Then, the distributed storage method of data based on MongoDB is used to store multi-source remote sensing data. The distributed storage method is physically based on the sharding mechanism of the MongoDB database, and it can provide advantages for the security and performance of the preservation of remote sensing data. Finally, several experiments have been designed to test the performance of this framework in integrating multi-source remote sensing data. The results show that the storage and retrieval performance of the distributed remote sensing data integration framework proposed in this paper is superior. At the same time, the grid level of the SSI model proposed in this paper also has an important impact on the storage efficiency of remote sensing data. Therefore, the remote storage data integration framework, based on distributed storage, can provide new technical support and development prospects for big EO data.


2010 ◽  
Vol 39 (3) ◽  
pp. 955-963 ◽  
Author(s):  
Nicholas E. Clinton ◽  
Christopher Potter ◽  
Bob Crabtree ◽  
Vanessa Genovese ◽  
Peggy Gross ◽  
...  

Author(s):  
Felix Rembold ◽  
Michele Meroni ◽  
Ferdinando Urbano ◽  
Antoine Royer ◽  
Clement Atzberger ◽  
...  

2015 ◽  
Vol 738-739 ◽  
pp. 354-360
Author(s):  
Jian Jun Zhang ◽  
Ye Xin Song ◽  
Yong Qu

Time series analysis is advantageous since it offers insight into the underlying dynamics and forecasts system behavior. The construction of the discriminant function is of vital importance in the time series analysis based fault diagnosis. Aiming at the problem that some of the time series analysis based fault diagnosis methods exist the weakness of higher time complexity, weaker discriminant ability and insufficient online diagnosis power, this paper proposes an approach which makes full use of the characteristics of the model and observation data to construct the discriminant function, and presents an efficient algorithm which can effectively recognize the system state by the proposed discriminant function. As compared to the related work, it has the characteristics of lower time complexity, shorter computation time and stronger distinguished ability, without the requirement of same orders of the reference model and the detected model. The fault diagnosis steps based on the proposed discriminant function and its algorithm are also suggested.


Sign in / Sign up

Export Citation Format

Share Document