Analysis and Research on Big Data Query Technology Based on NOSQL Database

The amount of data in our industry and the world is exploding. Data is being collected and stored at unprecedented rates. The challenge is not only to store and manage the vast volume of data, which is also called big data, but also to analyze and query from it. In order to put forward the universal method to response mobile big data query, queries are separated and grouped according to kinds of query for massive mobile objects in the space. The indexing method for grouping the mobile objects with Grid (GG TPR-tree) has great efficiency to manage a massive capacity of mobile objects within a limited area, but it only could meet a part of requirements for mobile big data query if the GG TPR-tree was used solely. This thesis offers solutions to simple immediate query, simple continuous query, active window query, and continuous window query, dynamic condition query and other query requests by employing DTDI index structure. The experiments prove that with the support of DTDI index structure, query of massive mobile objects has higher precision and better query performance.

Download Full-text

Big Data Query Engines

Handbook of Big Data Technologies ◽

10.1007/978-3-319-49340-4_6 ◽

2017 ◽

pp. 179-217

Author(s):

Mohamed A. Soliman

Keyword(s):

Big Data ◽

Data Query

Download Full-text

Building OLAP Cubes From Columnar NoSQL Data Warehouses

Emerging Perspectives in Big Data Warehousing - Advances in Data Mining and Database Management ◽

10.4018/978-1-5225-5516-2.ch006 ◽

2019 ◽

pp. 129-157

Author(s):

Khaled Dehdouh

Keyword(s):

Big Data ◽

Database System ◽

Massive Data ◽

Data Warehouses ◽

Online Analysis ◽

Storage Model ◽

Data Cubes ◽

Nosql Database ◽

Oriented Approach

In the big data warehouses context, a column-oriented NoSQL database system is considered as the storage model which is highly adapted to data warehouses and online analysis. Indeed, the use of NoSQL models allows data scalability easily and the columnar store is suitable for storing and managing massive data, especially for decisional queries. However, the column-oriented NoSQL DBMS do not offer online analysis operators (OLAP). To build OLAP cubes corresponding to the analysis contexts, the most common way is to integrate other software such as HIVE or Kylin which has a CUBE operator to build data cubes. By using that, the cube is built according to the row-oriented approach and does not allow to fully obtain the benefits of a column-oriented approach. In this chapter, the main contribution is to define a cube operator called MC-CUBE (MapReduce Columnar CUBE), which allows building columnar NoSQL cubes according to the columnar approach by taking into account the non-relational and distributed aspects when data warehouses are stored.

Download Full-text

Architecture for Big Data Storage in Different Cloud Deployment Models

Research Anthology on Architectures, Frameworks, and Integration Strategies for Distributed and Cloud Computing ◽

10.4018/978-1-7998-5339-8.ch009 ◽

2021 ◽

pp. 178-208

Author(s):

Chandu Thota ◽

Gunasekaran Manogaran ◽

Daphne Lopez ◽

Revathi Sundarasekar

Keyword(s):

Cloud Computing ◽

Big Data ◽

Data Storage ◽

High Performance ◽

Data Services ◽

Big Data Applications ◽

Nosql Database ◽

Amazon Web Services ◽

Product Domains ◽

Scalable Database

Cloud Computing is a new computing model that distributes the computation on a resource pool. The need for a scalable database capable of expanding to accommodate growth has increased with the growing data in web world. More familiar Cloud Computing vendors such as Amazon Web Services, Microsoft, Google, IBM and Rackspace offer cloud based Hadoop and NoSQL database platforms to process Big Data applications. Variety of services are available that run on top of cloud platforms freeing users from the need to deploy their own systems. Nowadays, integrating Big Data and various cloud deployment models is major concern for Internet companies especially software and data services vendors that are just getting started themselves. This chapter proposes an efficient architecture for integration with comprehensive capabilities including real time and bulk data movement, bi-directional replication, metadata management, high performance transformation, data services and data quality for customer and product domains.

Download Full-text

Toward Automatic Generation of Column-Oriented NoSQL Databases in Big Data Context

International Journal of Online and Biomedical Engineering (iJOE) ◽

10.3991/ijoe.v15i09.10433 ◽

2019 ◽

Vol 15 (09) ◽

pp. 4

Author(s):

Redouane Esbai ◽

Fouad Elotmani ◽

Fatima Zahra Belkadi

Keyword(s):

Big Data ◽

Relational Databases ◽

Automatic Generation ◽

Transformation Model ◽

Model Driven ◽

Transformation Rules ◽

New Concepts ◽

Nosql Database ◽

Meta Object Facility ◽

View Transformation

<span>The growth of application architectures in all areas (e.g. Astrology, Meteorology, E-commerce, social network, etc.) has resulted in an exponential increase in data volumes, now measured in Petabytes. Managing these volumes of data has become a problem that relational databases are no longer able to handle because of the acidity properties. In response to this scaling up, new concepts have emerged such as NoSQL. In this paper, we show how to design and apply transformation rules to migrate from an SQL relational database to a Big Data solution within NoSQL. For this, we use the Model Driven Architecture (MDA) and the transformation languages like as MOF 2.0 QVT (Meta-Object Facility 2.0 Query-View-Transformation) and Acceleo which define the meta-models for the development of transformation model. The transformation rules defined in this work can generate, from the class diagram, a CQL code for creation column-oriented NoSQL database.</span>

Download Full-text

Exploiting soft and hard correlations in big data query optimization

Proceedings of the VLDB Endowment ◽

10.14778/2994509.2994519 ◽

2016 ◽

Vol 9 (12) ◽

pp. 1005-1016 ◽

Cited By ~ 7

Author(s):

Hai Liu ◽

Dongqing Xiao ◽

Pankaj Didwania ◽

Mohamed Y. Eltabakh

Keyword(s):

Big Data ◽

Query Optimization ◽

Data Query

Download Full-text

Using model Driven Engineering to transform Big Data query languages to MapReduce jobs

International Journal of Computing and Digital Systems ◽

10.12785/ijcds/100160 ◽

2021 ◽

Vol 10 (1) ◽

pp. 619-628

Author(s):

Allae Erraissi

Keyword(s):

Big Data ◽

Query Languages ◽

Model Driven Engineering ◽

Model Driven ◽

Data Query

Download Full-text

A Paged Prefetching Model for Join Operations of Cross-Databases

Xibei Gongye Daxue Xuebao/Journal of Northwestern Polytechnical University ◽

10.1051/jnwpu/20203861225 ◽

2020 ◽

Vol 38 (6) ◽

pp. 1225-1234

Author(s):

Changqing Li ◽

Jianhua Gu

Keyword(s):

Big Data ◽

Query Language ◽

Research Result ◽

Management Systems ◽

Prototype System ◽

Large Delay ◽

Basic Composition ◽

Operation Mechanism ◽

Nosql Database ◽

Research Hotspots

As the applications of big data are increasing, the NoSQL (Not only Structured Query Language) database management systems have been developed rapidly. How to integrate NoSQL database and relational database effectively has become one of the research hotspots. In the existing research results, the paged query method used for join operations of these heterogeneous multi-databases can produce a large delay. In view of this deficiency, this paper presents a paged prefetching model, and focuses on its basic composition, prefetching mode and operation mechanism. A prototype system is designed and developed. The effect of this model is verified, and the expected targets are achieved. Compared with the non-prefetched paging query method, the outstanding contribution of this research result is that it can reduce the delay of paging query and thus improve the efficiency of the join operations of cross-databases.

Download Full-text

Business-oriented customized big data query system and its SQL parser design and implementation

MATEC Web of Conferences ◽

10.1051/matecconf/201823201004 ◽

2018 ◽

Vol 232 ◽

pp. 01004

Author(s):

Wenshuai Ge ◽

Gang He ◽

Xinwen Liu

Keyword(s):

Big Data ◽

User Behavior ◽

The Internet ◽

Query Interface ◽

User Behavior Analysis ◽

Design And Implementation ◽

Data Query ◽

Query System ◽

Internet User Behavior ◽

Analysis Platform

This paper proposes a big data query system for customized queries based on specific business needs. This paper introduces the components and structure of the query system. ANTLR tools are used as language recognizer to design and implement a customized SQL dialect. The system builds a simpler and easier query interface on Spark SQL, which satisfies the query requirements of the Internet user behavior analysis platform.

Download Full-text