scholarly journals USE OF DISTRIBUTED DBMS FOR SPATIAL DATA PROCESSING

2020 ◽  
Vol 1 (2) ◽  
pp. 82-87
Author(s):  
Aleksey A. Kolesnikov ◽  
Elena V. Komissarova ◽  
Ivan V. Zhdanov
Keyword(s):  
Big Data ◽  
Spatial Data ◽  
Geospatial Data ◽  
Visible Range ◽  
Data Sets ◽  
Vector Data ◽  
Advantages And Disadvantages ◽  
Climate Indicators ◽  
Analytical Tools ◽  
Spatial Data Processing

Currently, data volumes are growing exponentially. Geospatial data is one of the main elements of the concept of Big data. There is a very large number of tools for analyzing Big data, but not all of them take into account the features and have the ability to process geospatial data. The article discusses three popular open analytical tools Hadoop Spatial, GeoSpark, GeoFlink for working with geospatial data of very large volumes. Their architectures, advantages and disadvantages, depending on the execution time and the amount of data used are considered. Processing evaluations were also performed in terms of both streaming and packet data. The experiments were carried out on raster and vector data sets, which are satellite imagery in the visible range, NDVI and NDWI indices, climate indicators (snow cover, precipitation intensity, surface temperature), data from the Open Street Map in the Novosibirsk and Irkutsk Regions.

scholarly journals Predictive geospatial analytics using principal component regression

2020 ◽  
Vol 10 (3) ◽  
pp. 2651
Author(s):  
Kyilai Lai Khine ◽  
ThiThi Soe Nyunt
Keyword(s):  
Big Data ◽  
Spatial Data ◽  
Data Analytics ◽  
Predictive Analytics ◽  
Hybrid Approach ◽  
Principal Component Regression ◽  
Principal Component ◽  
Geospatial Data ◽  
Data Sets ◽  
Mlr Model

Nowadays, exponential growth in geospatial or spatial data all over the globe, geospatial data analytics is absolutely deserved to pay attention in manipulating voluminous amount of geodata in various forms increasing with high velocity. In addition, dimensionality reduction has been playing a key role in high-dimensional big data sets including spatial data sets which are continuously growing not only in observations but also in features or dimensions. In this paper, predictive analytics on geospatial big data using Principal Component Regression (PCR), traditional Multiple Linear Regression (MLR) model improved with Principal Component Analysis (PCA), is implemented on distributed, parallel big data processing platform. The main objective of the system is to improve the predictive power of MLR model combined with PCA which reduces insignificant and irrelevant variables or dimensions of that model. Moreover, it is contributed to present how data mining and machine learning approaches can be efficiently utilized in predictive geospatial data analytics. For experimentation, OpenStreetMap (OSM) data is applied to develop a one-way road prediction for city Yangon, Myanmar. Experimental results show that hybrid approach of PCA and MLR can be efficiently utilized not only in road prediction using OSM data but also in improvement of traditional MLR model.

scholarly journals Geospatial data sharing, online spatial analysis and processing of Indian Biodiversity data in Internet GIS domain – A case study for raster based online geo-processing

2014 ◽  
Vol XL-8 ◽  
pp. 1133-1137 ◽  
Author(s):  
H. Karnatak ◽  
K. Pandey ◽  
K. Oberai ◽  
A. Roy ◽  
D. Joshi ◽  
...  
Keyword(s):  
Spatial Analysis ◽  
Geospatial Data ◽  
Fine Tuning ◽  
Data Sets ◽  
Vector Data ◽  
Biodiversity Data ◽  
Raster Data ◽  
Area Of Interest ◽  
Field Samples ◽  
Biological Richness

National Biodiversity Characterization at Landscape Level, a project jointly sponsored by Department of Biotechnology and Department of Space, was implemented to identify and map the potential biodiversity rich areas in India. This project has generated spatial information at three levels viz. Satellite based primary information (Vegetation Type map, spatial locations of road & village, Fire occurrence); geospatially derived or modelled information (Disturbance Index, Fragmentation, Biological Richness) and geospatially referenced field samples plots. The study provides information of high disturbance and high biological richness areas suggesting future management strategies and formulating action plans. The study has generated for the first time baseline database in India which will be a valuable input towards climate change study in the Indian Subcontinent. <br><br> The spatial data generated during the study is organized as central data repository in Geo-RDBMS environment using PostgreSQL and POSTGIS. The raster and vector data is published as OGC WMS and WFS standard for development of web base geoinformation system using Service Oriented Architecture (SOA). The WMS and WFS based system allows geo-visualization, online query and map outputs generation based on user request and response. This is a typical mashup architecture based geo-information system which allows access to remote web services like ISRO Bhuvan, Openstreet map, Google map etc., with overlay on Biodiversity data for effective study on Bio-resources. <br><br> The spatial queries and analysis with vector data is achieved through SQL queries on POSTGIS and WFS-T operations. But the most important challenge is to develop a system for online raster based geo-spatial analysis and processing based on user defined Area of Interest (AOI) for large raster data sets. The map data of this study contains approximately 20 GB of size for each data layer which are five in number. An attempt has been to develop system using python, PostGIS and PHP for raster data analysis over the web for Biodiversity conservation and prioritization. The developed system takes inputs from users as WKT, Openlayer based Polygon geometry and Shape file upload as AOI to perform raster based operation using Python and GDAL/OGR. The intermediate products are stored in temporary files and tables which generate XML outputs for web representation. The raster operations like clip-zip-ship, class wise area statistics, single to multi-layer operations, diagrammatic representation and other geo-statistical analysis are performed. This is indigenous geospatial data processing engine developed using Open system architecture for spatial analysis of Biodiversity data sets in Internet GIS environment. The performance of this applications in multi-user environment like Internet domain is another challenging task which is addressed by fine tuning the source code, server hardening, spatial indexing and running the process in load balance mode. The developed system is hosted in Internet domain (<a href="http://http://bis.iirs.gov.in" target="_blank">http://bis.iirs.gov.in</a>) for user access.

Research on Generalization Technology of Spatial Line Vector Data

2014 ◽  
Vol 687-691 ◽  
pp. 1153-1156
Author(s):  
Shi Qing Dou ◽  
Xiao Yu Zhang
Keyword(s):  
Spatial Data ◽  
Three Dimensional ◽  
Vector Data ◽  
Advantages And Disadvantages ◽  
3D Environment ◽  
Development Direction ◽  
The Future ◽  
2D And 3D ◽  
Data Generalization

Data simplification is an important factor of the spatial data generalization, which is an effective way to improve rendering speed. This paper firstly introduces the algorithms classification of the spatial line vector data in two-dimensional environment, and then it emphatically summarizes and analyzes the advantages and disadvantages of the algorithms which can be used in the spatial line vector data simplification in the three dimensional environment. The three-dimensional Douglas-Peucker algorithm with a certain overall characteristics has wide application prospect. The simplified algorithms in 3D environment represent the development direction of the future. But at present, the existing data simplification algorithms in 3D environment are not mature enough, they all have certain advantages and disadvantages, this makes their use is limited by a certain extent. The application of these simplified algorithms in 2D and 3D is mostly on multi-resolution expression. Developing from 2D algorithm to the direction of 3D algorithm, it also lists many works and problems that need us to do and study in the future.

scholarly journals GEODESY, CARTOGRAPHY AND AERIAL PHOTOGRAPHY

2020 ◽  
Vol 92,2020 (92) ◽  
pp. 24-36
Author(s):  
Yurii Karpinkyi ◽  
◽  
Nadiia Lazorenko-Hevel ◽  
Keyword(s):  
Spatial Data ◽  
Knowledge Bases ◽  
International Standards ◽  
National Standards ◽  
Geospatial Data ◽  
Practical Significance ◽  
Topographic Mapping ◽  
Data Sets ◽  
Data Infrastructure ◽  
The Creation

The article proposes a new development concept of topographic mapping in Ukraine. The goal. It is based on the implementation of a new system model that responds to the geoinformation approach to topographic mapping in the development of National Spatial Data Infrastructure (NSDI) and provides the creation of geospatial data sets in the form of databases and knowledge bases based on existing standards and specifications: series of International Standards ISO 19100 “Geographic information/Geomatics”, Open Geospatial Consortium (OGS), INSPIRE, National Standards of Ukraine (DSTU), Complex of Standards Organization of Ukraine (SOU) “Topographic database”. Methods. The basis for the research is the analysis of the possibilities of applying the theory of databases and knowledge bases International Standards and specifications. Scientific novelty and practical significance. It provides a high intellectual level of Core Reference and profile geospatial data, which is capable to provide geoinformation analysis and modeling in modern GIS. In addition, the implementation the infrastructure approach to topographic production and the creation and development of a permanent topographic monitoring system will ensure the publication of geospatial data in real time, almost simultaneously with changes in the terrain, which guarantees the maintenance of the single digital topographic basis and, accordingly, Core Reference Datasets for NSDI.

scholarly journals 3D GIS FOR FLOOD MODELLING IN RIVER VALLEYS

2016 ◽  
Vol XLI-B8 ◽  
pp. 175-178 ◽  
Author(s):  
P. Tymkow ◽  
M. Karpina ◽  
A. Borkowski
Keyword(s):  
Data Acquisition ◽  
Spatial Data ◽  
Laser Scanning ◽  
Stokes Equations ◽  
Visible Range ◽  
Hydrodynamic Modelling ◽  
Vector Data ◽  
Flood Modelling ◽  
Spatial Objects ◽  
River Valleys

The objective of this study is implementation of system architecture for collecting and analysing data as well as visualizing results for hydrodynamic modelling of flood flows in river valleys using remote sensing methods, tree-dimensional geometry of spatial objects and GPU multithread processing. The proposed solution includes: spatial data acquisition segment, data processing and transformation, mathematical modelling of flow phenomena and results visualization. Data acquisition segment was based on aerial laser scanning supplemented by images in visible range. Vector data creation was based on automatic and semiautomatic algorithms of DTM and 3D spatial features modelling. Algorithms for buildings and vegetation geometry modelling were proposed or adopted from literature. The implementation of the framework was designed as modular software using open specifications and partially reusing open source projects. The database structure for gathering and sharing vector data, including flood modelling results, was created using PostgreSQL. For the internal structure of feature classes of spatial objects in a database, the CityGML standard was used. For the hydrodynamic modelling the solutions of Navier-Stokes equations in two-dimensional version was implemented. Visualization of geospatial data and flow model results was transferred to the client side application. This gave the independence from server hardware platform. A real-world case in Poland, which is a part of Widawa River valley near Wroclaw city, was selected to demonstrate the applicability of proposed system.

scholarly journals 3D GIS FOR FLOOD MODELLING IN RIVER VALLEYS

2016 ◽  
Vol XLI-B8 ◽  
pp. 175-178 ◽  
Author(s):  
P. Tymkow ◽  
M. Karpina ◽  
A. Borkowski
Keyword(s):  
Data Acquisition ◽  
Spatial Data ◽  
Laser Scanning ◽  
Stokes Equations ◽  
Visible Range ◽  
Hydrodynamic Modelling ◽  
Vector Data ◽  
Flood Modelling ◽  
Spatial Objects ◽  
River Valleys

The objective of this study is implementation of system architecture for collecting and analysing data as well as visualizing results for hydrodynamic modelling of flood flows in river valleys using remote sensing methods, tree-dimensional geometry of spatial objects and GPU multithread processing. The proposed solution includes: spatial data acquisition segment, data processing and transformation, mathematical modelling of flow phenomena and results visualization. Data acquisition segment was based on aerial laser scanning supplemented by images in visible range. Vector data creation was based on automatic and semiautomatic algorithms of DTM and 3D spatial features modelling. Algorithms for buildings and vegetation geometry modelling were proposed or adopted from literature. The implementation of the framework was designed as modular software using open specifications and partially reusing open source projects. The database structure for gathering and sharing vector data, including flood modelling results, was created using PostgreSQL. For the internal structure of feature classes of spatial objects in a database, the CityGML standard was used. For the hydrodynamic modelling the solutions of Navier-Stokes equations in two-dimensional version was implemented. Visualization of geospatial data and flow model results was transferred to the client side application. This gave the independence from server hardware platform. A real-world case in Poland, which is a part of Widawa River valley near Wroclaw city, was selected to demonstrate the applicability of proposed system.

scholarly journals Big Data breaking barriers - first steps on a long trail

2015 ◽  
Vol XL-7/W3 ◽  
pp. 691-697 ◽  
Author(s):  
S. Schade
Keyword(s):  
Big Data ◽  
Data Storage ◽  
Data Structures ◽  
Spatial Data ◽  
Earth Observation ◽  
The Other ◽  
Data Sets ◽  
Environmental Sciences ◽  
Geospatial Information ◽  
The One

Most data sets and streams have a geospatial component. Some people even claim that about 80% of all data is related to location. In the era of Big Data this number might even be underestimated, as data sets interrelate and initially non-spatial data becomes indirectly geo-referenced. The optimal treatment of Big Data thus requires advanced methods and technologies for handling the geospatial aspects in data storage, processing, pattern recognition, prediction, visualisation and exploration. On the one hand, our work exploits earth and environmental sciences for existing interoperability standards, and the foundational data structures, algorithms and software that are required to meet these geospatial information handling tasks. On the other hand, we are concerned with the arising needs to combine human analysis capacities (intelligence augmentation) with machine power (artificial intelligence). This paper provides an overview of the emerging landscape and outlines our (Digital Earth) vision for addressing the upcoming issues. We particularly request the projection and re-use of the existing environmental, earth observation and remote sensing expertise in other sectors, i.e. to break the barriers of all of these silos by investigating integrated applications.

scholarly journals Topographic mapping in the National Spatial Data Infrastructure in Ukraine

2020 ◽  
Vol 171 ◽  
pp. 02004
Author(s):  
Yurii Karpinskyi ◽  
Nadiia Lazorenko-Hevel
Keyword(s):  
Spatial Data ◽  
Knowledge Bases ◽  
Technical Conference ◽  
International Standards ◽  
Geospatial Data ◽  
Topographic Mapping ◽  
Data Sets ◽  
National Agency ◽  
Data Infrastructure ◽  
The Creation

The article proposes a new development concept of topographic mapping in Ukraine. It is based on the implementation of a new system model that responds to the geoinformation approach to topographic mapping in the development of national geospatial data infrastructure (NSDI) and provides the creation of geospatial data sets in the form of databases and knowledge bases based on existing standards and specifications: series of international standards ISO 19100 “Geographic information/Geomatics”, Open Geospatial Consortium (OGS), INSPIRE, State Standards of Ukraine (DSTU), Complex of Standards Organization of Ukraine (SOU) “Topographic database”. It provides a high intellectual level of core reference and profile geospatial data, which is capable to provide geoinformation analysis and modeling in modern GIS. In addition, the implementation the infrastructure approach to topographic production and the creation and development of a permanent topographic monitoring system will ensure the publication of geospatial data in real time, almost simultaneously with changes in the terrain, which guarantees the maintenance of a single topographic basis and, accordingly, core reference datasets for NSDI. Publication is funded by the Polish National Agency for Academic Exchange under the International Academic Partnerships Programme from the project „Organization of the 9th International Scientific and Technical Conference entitled Environmental Engineering, Photogrammetry, Geoinformatics – Modern Technologies and Development Perspectives”.

scholarly journals Hadoop Framework for Big Data Analytics to Analyze Attacks on Business Data

2019 ◽  
Vol 9 (2) ◽  
pp. 3995-4000
Keyword(s):  
Big Data ◽  
Data Security ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Data Sets ◽  
Business Data ◽  
Analytical Tools ◽  
Security Analytics ◽  
Day By Day ◽  
Hadoop Framework

Big Data consist large volumes of data sets with various formats i.e., structured, unstructured and semi structured. Big Data requires security because day by day attackers attack on it in different manner. Big Data Security Analytics analyses Big Data for finding various threats and complex attacks. By increasing the number of targeting attacks on data and one side rapid growing of data, it is too difficult to analyze accurately. The Security Analytics Systems are used the untrusted data. So, strong security analytical tools are required to analyze the data. The organizations and industries exchange the data through networks dynamically, so this may become more vulnerable for data misusing and theft. Attackers are more advanced in the attacking on data that the existing security mechanisms are not identified before damaging. At present, the collecting and analyzing various attacks is major challenging task for Security Analytics Systems, to take suitable decision. In this research paper, we have addressed about Hadoop tool that how it analyses Big Data and how Big Data Security Analytics is applied to analyze the various threats and securing the business data more accurately.

scholarly journals A Study on Big Data Hadoop Map Reduce Job Scheduling

2018 ◽  
Vol 7 (3.31) ◽  
pp. 59
Author(s):  
N Deshai ◽  
S Venkataramana ◽  
I Hemalatha ◽  
G P. S. Varma
Keyword(s):  
Big Data ◽  
Data Storage ◽  
High Performance ◽  
Job Scheduling ◽  
Data Availability ◽  
Data Sets ◽  
Advantages And Disadvantages ◽  
Digital World ◽  
Processing Power ◽  
Key Factor

A latest tera to zeta era has been created during huge volume of data sets, which keep on collected from different social networks, machine to machine devices, google, yahoo, sensors etc. called as big data. Because day by day double the data storage size, data processing power, data availability and digital world data size in zeta bytes. Apache Hadoop is latest market weapon to handle huge volume of data sets by its most popular components like hdfs and mapreduce, to achieve an efficient storage ability and efficient processing on massive volume of data sets. To design an effective algorithm is a key factor for selecting nodes are important, to optimize and acquire high performance in Big data. An efficient and useful survey, overview, advantages and disadvantages of these scheduling algorithms provided also identified throughout this paper.  

Sign in / Sign up

Export Citation Format

Share Document