scholarly journals Novel IT Technologies on the Digital Battlefield: The Application of Big Data and Data Mining Technologies

Hadmérnök ◽  
2020 ◽  
Vol 15 (4) ◽  
pp. 141-158
Author(s):  
Eszter Katalin Bognár
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Data Processing ◽  
Relevant Information ◽  
Military Operations ◽  
Textual Data ◽  
Modern Warfare ◽  
Tools And Techniques

In modern warfare, the most important innovation to date has been the utilisation of information as a  weapon. The basis of successful military operations is  the ability to correctly assess a situation based on  credible collected information. In today’s military, the primary challenge is not the actual collection of data.  It has become more important to extract relevant  information from that data. This requirement cannot  be successfully completed without necessary  improvements in tools and techniques to support the acquisition and analysis of data. This study defines  Big Data and its concept as applied to military  reconnaissance, focusing on the processing of  imagery and textual data, bringing to light modern  data processing and analytics methods that enable  effective processing.

scholarly journals Vertical Data Processing for Mining Big Data: A Predicate Tree Approach

10.29007/db8n ◽  
2019 ◽  
Author(s):  
Mohammad Hossain ◽  
Maninder Singh ◽  
Sameer Abufardeh
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Data Processing ◽  
Processing Time ◽  
Traditional Approach ◽  
Critical Factor ◽  
Boolean Operations ◽  
Data Mining Algorithms ◽  
Vertical Data ◽  
Big Data Application

Time is a critical factor in processing a very large volume of data a.k.a ‘Big Data’. Many existing data mining algorithms (supervised and unsupervised) become futile because of the ubiquitous use of horizontal processing i.e. row-by-row processing of stored data. Processing time for big data is further exacerbated by its high dimensionality (# of features) and high cardinality (# of records). To address this processing-time issue, we proposed a vertical approach with predicate trees (pTree). Our approach structures data into columns of bit slices, which range from few to hundreds and are processed vertically i.e. column by column. We tested and compared our vertical approach to traditional (horizontal) approach using three basic Boolean operations namely addition, subtraction and multiplication with 10 data sizes. The length of data size ranged from half a billion bits to 5 billion bits. The results are analyzed w.r.t processing speed time and speed gain for both the approaches. The result shows that our vertical approach outperformed the traditional approach for all Boolean operations (add, subtract and multiply) across all data sizes and results in speed-gain between 24% to 96%. We concluded from our results that our approach being in data-mining ready format is best suited to apply to operations involving complex computations in big data application to achieve significant speed gain.

scholarly journals Map Reduce clustering in Incremental Big Data processing

2019 ◽  
Vol 9 (2) ◽  
pp. 4205-4211
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Social Network ◽  
Data Processing ◽  
Online Shopping ◽  
Processing Technique ◽  
Map Reduce ◽  
Data Set ◽  
Computation Procedure ◽  
Incremental Processing

An advanced Incremental processing technique is planned for data examination in knowledge to have the clustering results inform. Data is continuously arriving by different data generating factors like social network, online shopping, sensors, e-commerce etc. [1]. On account of this Big Data the consequences of data mining applications getting stale and neglected after some time. Cloud knowledge applications regularly perform iterative calculations (e.g., PageRank) on continuously converting datasets. Though going before trainings grow Map-Reduce aimed at productive iterative calculations, it's miles also pricey to carry out a whole new big-ruler Map-Reduce iterative task near well-timed quarter new adjustments to fundamental records sets. Our usage of MapReduce keeps running [4] scheduled a big cluster of product technologies and is incredibly walkable: an ordinary Map-Reduce computation procedure several terabytes of records arranged heaps of technologies. Processor operator locates the machine clean to apply: masses of MapReduce applications, we look at that during many instances, The differences result separate a totally little part of the data set, and the recently iteratively merged nation is very near the recently met state. I2MapReduce clustering adventures this commentary to keep re-calculated by way of beginning after the before affected national [2], and by using acting incremental up-dates on the converging information. The approach facilitates in enhancing the process successively period and decreases the jogging period of stimulating the consequences of big data.

scholarly journals Applications of Machine Learning in Public Security Information and Resource Management

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Zhihui Wang ◽  
Jinyu Wang
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Data Warehouse ◽  
Large Scale ◽  
Relevant Information ◽  
Practical Significance ◽  
Public Security ◽  
Data Mining Algorithms ◽  
New Findings ◽  
Security Information

The data mining and big data technologies could be of utmost importance to investigate outbound and case datasets in the police records. New findings and useful information may potentially be obtained through data preprocessing and multidimensional modeling. Public security data is a kind of “big data,” having characteristics like large volume, rapid growth, various structures, large-scale storage, low density, and time sensitiveness. In this paper, a police data warehouse is constructed and a public security information analysis system is proposed. The proposed system comprises two modules: (i) case management and (ii) public security information mining. The former is responsible for the collection and processing of case information. The latter preprocesses the data of major cases that have occurred in the past ten years to create a data warehouse. Then, we use the model to create a data warehouse based on needs. By dividing the measurement values and dimensions, the analysis and prediction of criminals’ characteristics and the case environment realize relationships between them. In the process of mining and processing crime data, data mining algorithms can quickly find out the relevant information in the data. Furthermore, the system can find out relevant trends and laws to detect criminal cases faster than other methods. This can reduce the emergence of new crimes and provide a basis for decision-making in the public security department that has practical significance.

scholarly journals The problem of analysis of big web data and the use of data mining technology for processing and searching patterns in big web data on a practical example

2019 ◽  
Vol 23 (2) ◽  
pp. 42-49 ◽  
Author(s):  
K. V. Mulyukova ◽  
V. M. Kureichik
Keyword(s):  
Data Mining ◽  
Neural Networks ◽  
Big Data ◽  
Data Processing ◽  
Practical Problem ◽  
The Internet ◽  
Web Data ◽  
Mining Technology ◽  
C Programming Language ◽  
C Programming

The purpose of the work is to study the current problems and prospects of the solution for processing big data received or stored in the Internet (web data), as well as the possibility of practical realization of Data Mining technology for big web data on practical example. Materials and methods. The study included a review of bibliographic sources on big data analysis problems.Data Mining technology was used to analyze large web data, as well as computer modeling of a practical problem using the C # programming language and creating a DDL database structure for accumulating web data.Results. In the course of the work, the specifics of big data were described, the main characteristics of big data were highlighted, and modern approaches to processing big data were analyzed. A brief description of the horizontal-scalable architecture and the BI-solution architecture for big data processing is given. The problems of processing large web data are formulated: limiting the speed of access to data, providing access via network protocols through general-purpose networks.An example showing the approach to processing large web data was also implemented. Based on the idea of big data, the described complexities of web data processing and the methods of Data Mining, techniques were proposed for effectively solving the practical problem of processing and searching patterns in a large data array.The following classes have been developed in the C # programming language:Class of receiving web data via the Internet; Data conversion class;Intelligent data processing class;Created DDL script that creates a structure for the accumulation of web data.A single UML class diagram has been developed.The constructed system of data and classes allows to solve the main part of the problems of processing large web data and perform intelligent processing using Data Mining technology in order to solve the problem posed of identifying certain records in a large array. The combination of object-oriented approach, neural networks and BI-analysis to filter data will speed up the process of data processing and obtaining the result of the studyConclusion. According to the results of the study, it can be argued that the current state of technology for analyzing large web data allows you to efficiently process data objects, identify patterns, get hidden data and get full-fledged statistical data.The obtained results can be used both for the purpose of the initial study of big data processing technologies, and as a basis for developing an already real application for analyzing web data. The use of neural networks and the created universal classes-handlers makes the created architecture flexible and self-learning, and the class declarations and the base DDL structure will greatly simplify the development of program code.

Awareness of Big Data Usage and Applications Among Librarians in Zimbabwe

2021 ◽  
pp. 22-34
Author(s):  
Josiline Phiri Chigwada ◽  
Justice Kasiroori
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Qualitative Study ◽  
Capacity Building ◽  
Research Institutions ◽  
Online Questionnaire ◽  
Data Usage ◽  
Current Trends ◽  
Big Data Applications ◽  
Tools And Techniques

The chapter showcases the awareness of big data usage among librarians in Zimbabwe. The concept of big data is new, and librarians are building capacity to move with the current trends in librarianship. This chapter assists in pointing areas where big data can be applied in libraries. It also documents the challenges that are faced when using big data applications and proffer solutions that can be applied to deal with those challenges. It answers the question of whether it is practical to utilise big data in any type of library. A qualitative study was done where an online questionnaire was administered to twenty librarians in research institutions in Zimbabwe. The findings revealed that librarians are aware of the big data concept but are not utilising the tools and techniques in data mining and analysis. The authors recommend that capacity building should be done to equip librarians with the requisite skills.

Big Data Mining

2018 ◽  
pp. 36-42
Author(s):  
V. Sucharita ◽  
P. Venkateswara Rao ◽  
A. Satya Kalyan ◽  
P. Rajarajeswari
Keyword(s):  
Data Mining ◽  
Big Data ◽  
The Self ◽  
Information Mining ◽  
Error Handling ◽  
Big Data Mining ◽  
Accessible Information ◽  
Tools And Techniques ◽  
Information Streams ◽  
Mining Tools

At present in Big Data era mining of Big Data can help us find learning which nobody has possessed the capacity to find some time recently. There is a developing interest for tools and techniques which can prepare and investigate Big Data effectively and proficiently. In this chapter, the accessible information mining tools and techniques which can deal with Big Data have been abridged. This paper additionally concentrates on tools and techniques for mining of data and information streams. Through better analysis of the vast volumes of information that are getting to be accessible, there is the potential for making speedier progresses in numerous scientific areas what's more, making strides the productivity what's more, victory of numerous organizations. The challenges incorporate not just the self-evident issues of scale, be that as it may too heterogeneity, need of structure, error handling, protection, opportunities at all stages of the analysis from acquisition of data to obtaining to result.

Mining Big Data in Manufacturing: Requirement Analysis, Tools and Techniques

2014 ◽  
Author(s):  
Utpal Roy ◽  
Bicheng Zhu ◽  
Yunpeng Li ◽  
Heng Zhang ◽  
Omer Yaman
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Product Design ◽  
Literature Study ◽  
Data Mining Techniques ◽  
Using Data ◽  
Tools And Techniques ◽  
Audio Video ◽  
Manufacturing Problems ◽  
Selection Of

Data Mining has tremendous potential and usefulness in improving the effectiveness of decision-making in manufacturing. Tools and techniques of data mining can be intelligently applied from product design analysis to the product repair and maintenance. Vast amount of data in the form of documents (text), graphical formats (CAD-file), audio/video, numbers, figures and/or hypertext are available in any typical manufacturing system. Our ultimate goal is to develop data-driven methodologies to solve manufacturing problems using data mining techniques. As a precursor, based on a literature study, this paper investigates selective manufacturing areas to identify the requirements for applying data mining techniques in solving potential manufacturing problems. The reviewed manufacturing areas are: (i) the “Design Intent” retrieval process for the product design and manufacturing, (ii) selection of materials, (iii) performance evaluations of manufacturing process design and operation management, and (iv) product inspection, and after-sales services (repair and maintenance). Industrial efforts towards addressing “Big Data” issues have also been briefly narrated in this paper. Lastly, the paper discusses two important data–related issues that may affect any applications of the data mining tools and techniques — (i) uncertainty involved in data collection, and (ii) interoperability of data collected at different levels of an enterprise.

Sign in / Sign up

Export Citation Format

Share Document