Heuristic and statistical methods of complex data processing in minerals seismic and electrical exploration

2014 ◽  
Author(s):  
Yu.V. Morozov
Keyword(s):  
Data Processing ◽  
Statistical Methods ◽  
Complex Data

scholarly journals Using Hadoop Technology to Overcome Big Data Problems by Choosing Proposed Cost-efficient Scheduler Algorithm for Heterogeneous Hadoop System (BD3)

2020 ◽  
pp. 58-84
Author(s):  
Abou_el_ela Abdou Hussein
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Storage ◽  
Database Management System ◽  
Data Sets ◽  
Complex Data ◽  
Daily Data ◽  
Complex Data Sets ◽  
Cost Efficient ◽  
Hadoop System

Day by day advanced web technologies have led to tremendous growth amount of daily data generated volumes. This mountain of huge and spread data sets leads to phenomenon that called big data which is a collection of massive, heterogeneous, unstructured, enormous and complex data sets. Big Data life cycle could be represented as, Collecting (capture), storing, distribute, manipulating, interpreting, analyzing, investigate and visualizing big data. Traditional techniques as Relational Database Management System (RDBMS) couldn’t handle big data because it has its own limitations, so Advancement in computing architecture is required to handle both the data storage requisites and the weighty processing needed to analyze huge volumes and variety of data economically. There are many technologies manipulating a big data, one of them is hadoop. Hadoop could be understand as an open source spread data processing that is one of the prominent and well known solutions to overcome handling big data problem. Apache Hadoop was based on Google File System and Map Reduce programming paradigm. Through this paper we dived to search for all big data characteristics starting from first three V's that have been extended during time through researches to be more than fifty six V's and making comparisons between researchers to reach to best representation and the precise clarification of all big data V’s characteristics. We highlight the challenges that face big data processing and how to overcome these challenges using Hadoop and its use in processing big data sets as a solution for resolving various problems in a distributed cloud based environment. This paper mainly focuses on different components of hadoop like Hive, Pig, and Hbase, etc. Also we institutes absolute description of Hadoop Pros and cons and improvements to face hadoop problems by choosing proposed Cost-efficient Scheduler Algorithm for heterogeneous Hadoop system.

scholarly journals In Situ Mass Spectrometry Diagnostics of Impaired Glucose Tolerance Using Label-Free Metabolomic Signature

Diagnostics ◽  
2020 ◽  
Vol 10 (12) ◽  
pp. 1052
Author(s):  
Petr G. Lokhov ◽  
Oxana P. Trifonova ◽  
Dmitry L. Maslov ◽  
Elena E. Balashova
Keyword(s):  
Mass Spectrometry ◽  
Glucose Tolerance ◽  
Data Processing ◽  
Impaired Glucose Tolerance ◽  
Characteristic Curve ◽  
Complex Data ◽  
Label Free ◽  
Mass Peak ◽  
Mass Spectrometers

In metabolomics, mass spectrometry is used to detect a large number of low-molecular substances in a single analysis. Such a capacity could have direct application in disease diagnostics. However, it is challenging because of the analysis complexity, and the search for a way to simplify it while maintaining the diagnostic capability is an urgent task. It has been proposed to use the metabolomic signature without complex data processing (mass peak detection, alignment, normalization, and identification of substances, as well as any complex statistical analysis) to make the analysis more simple and rapid. Methods: A label-free approach was implemented in the metabolomic signature, which makes the measurement of the actual or conditional concentrations unnecessary, uses only mass peak relations, and minimizes mass spectra processing. The approach was tested on the diagnosis of impaired glucose tolerance (IGT). Results: The label-free metabolic signature demonstrated a diagnostic accuracy for IGT equal to 88% (specificity 85%, sensitivity 90%, and area under receiver operating characteristic curve (AUC) of 0.91), which is considered to be a good quality for diagnostics. Conclusions: It is possible to compile label-free signatures for diseases that allow for diagnosing the disease in situ, i.e., right at the mass spectrometer without complex data processing. This achievement makes all mass spectrometers potentially versatile diagnostic devices and accelerates the introduction of metabolomics into medicine.

scholarly journals A Novel Finger-Controlled Passive RFID Tag Design for Human–Machine Interaction

Sensors ◽  
2019 ◽  
Vol 19 (23) ◽  
pp. 5125 ◽  
Author(s):  
Liu ◽  
Li ◽  
Yu ◽  
Zhao ◽  
Zhang
Keyword(s):  
Data Processing ◽  
Radio Frequency Identification ◽  
Dipole Antenna ◽  
The State ◽  
Complex Data ◽  
Human Machine Interaction ◽  
Rfid Tag ◽  
Text Input ◽  
Passive Rfid ◽  
Machine Interaction

Radio frequency identification (RFID) has shown its potential in human–machine interaction thanks to its inherent function of identification and relevant physical information of signals, but complex data processing and undesirable input accuracy restrict its application and promotion in practical use. This paper proposes a novel finger-controlled passive RFID tag design for human–machine interaction. The tag antenna is based on a dipole antenna with a separated T-match structure, which is able to adjust the state of the tag by the press of a finger. The state of the proposed tag can be recognized directly by the code received by the RFID reader, and no complex data processing is needed. Since the code is hardly affected by surroundings, the proposed tag is suitable to be used as a wireless switch or control button in multiple scenarios. Moreover, arrays of the proposed tag with rational tag arrangements could contribute to a series of manual control devices, such as a wireless keyboard, a remote controller, and a wireless gamepad, without batteries. A 3 × 4 array of the finger-controlled tag is presented to constitute a simple passive RFID keyboard as an example of the applications of the proposed tag array and it refers to the arrangement of a keypad and can achieve precise, convenient, quick, and practical commands and text input into machines by pressing the tags with fingers. Simulations and measurements of the proposed tag and tag array have been carried out to validate their performances in human–machine interaction.

Information and statistical methods of solving econometric, sociological and psychometric problems

2018 ◽  
Author(s):  
Сергей Юдин ◽  
Sergey Yudin ◽  
Александр Юдин ◽  
Aleksandr Yudin
Keyword(s):  
Information Theory ◽  
Graduate Students ◽  
Data Processing ◽  
Statistical Methods

The monograph contains a number of new author's theoretical and methodological developments based on the methods of mathematical information theory. It can be useful for students, graduate students and researchers in the analysis of data and the construction of models of economic, sociological and psychometric processes. It can be used as a textbook in the study of the relevant sections of the course "Mathematics", "modeling And data processing", "Econometrics" and others.

scholarly journals Processing two-dimensional X-ray diffraction and small-angle scattering data inDAWN 2

2017 ◽  
Vol 50 (3) ◽  
pp. 959-966 ◽  
Author(s):  
J. Filik ◽  
A. W. Ashton ◽  
P. C. Y. Chang ◽  
P. A. Chater ◽  
S. J. Day ◽  
...  
Keyword(s):  
Data Processing ◽  
Small Angle ◽  
Scattering Data ◽  
Data Provenance ◽  
Complex Data ◽  
Two Dimensional ◽  
X Ray Diffraction ◽  
Full Data ◽  
X Ray ◽  
Data Files

A software package for the calibration and processing of powder X-ray diffraction and small-angle X-ray scattering data is presented. It provides a multitude of data processing and visualization tools as well as a command-line scripting interface for on-the-fly processing and the incorporation of complex data treatment tasks. Customizable processing chains permit the execution of many data processing steps to convert a single image or a batch of raw two-dimensional data into meaningful data and one-dimensional diffractograms. The processed data files contain the full data provenance of each process applied to the data. The calibration routines can run automatically even for high energies and also for large detector tilt angles. Some of the functionalities are highlighted by specific use cases.

Data Mining Technology Applications in Tobacco Commercial Enterprise

2012 ◽  
Vol 461 ◽  
pp. 418-420
Author(s):  
Yi Min Mo ◽  
Xin Shun Tong ◽  
Li Hua Yang
Keyword(s):  
Data Mining ◽  
Data Processing ◽  
Demand Forecasting ◽  
Customer Relationship ◽  
Complex Data ◽  
Market Demand ◽  
Mining Technology ◽  
Technology Applications ◽  
Commercial Enterprise ◽  
Key Issues

The wide application of information technology has greatly improve the work efficiency but also caused a large and complex data accumulation. How to get the valuable information from vast amounts of data are the key issues in data processing. This paper studied the application of data mining technology in tobacco commercial enterprise from three aspects: market demand forecasting, customer relationship management and historical data processing. Analysis of how to use data mining technology to make full use of large amounts of data to provide a basis for tobacco commercial enterprise’s decision-making.

scholarly journals Metabolite AutoPlotter - an application to automatically process and visualise metabolite data in the web-browser

2020 ◽  
Author(s):  
Matthias Pietzke ◽  
Alexei Vazquez
Keyword(s):  
Stable Isotope ◽  
Data Processing ◽  
Ease Of Use ◽  
Complex Data ◽  
Isotope Tracing ◽  
Wide Audience ◽  
Web Browser ◽  
Seamless Integration ◽  
Metabolomics Data ◽  
Copy And Paste

Abstract Background Metabolomics is gaining popularity as a standard tool for the investigation of biological systems. Yet, parsing metabolomics data in the absence of in-house computational scientists can be overwhelming and time consuming. As a consequence of manual data processing the results are often not processed in full depth, so potential novel findings might get lost. Methods To tackle this problem we developed Metabolite AutoPlotter, a tool to process and visualise metabolite data. It reads as input pre-processed compound-intensity tables and accepts different experimental designs, with respect to number of compounds, conditions and replicates. The code was written in R and wrapped into a shiny-application that can be run online in a web-browser on https://mpietzke.shinyapps.io/autoplotter. Results We demonstrate the main features and the ease of use with two different metabolite datasets, for quantitative experiments and for stable isotope tracing experiments. We show how the plots generated by the tool can be interactively modified with respect to plot type, colours, text labels and the shown statistics. We also demonstrate the application towards 13-C-tracing experiments and the seamless integration of natural abundance correction, which facilitates the better interpretation of stable isotope tracing experiments. The output of the tool is a zip-file containing one single plot for each compound as well as sorted and restructured tables that can be used for further analysis. Conclusion With the help of Metabolite AutoPlotter it is now possible to automate data processing and visualisation for a wide audience. High quality plots from complex data can be generated in a short time with pressing a few buttons. This offers dramatic improvements over manual processing. It is significantly faster and allows researchers to spend more time interpreting the results or to perform follow-up experiments. Further this eliminates potential copy-and paste errors or tedious repetitions when things need to be changed. We are sure that this tool will help to improve and speed up scientific discoveries.

Sign in / Sign up

Export Citation Format

Share Document