Simultaneous Learning the Dimension and Parameter of a Statistical Model with Big Data

Application of mathematical probabilistic statistical model of base – FFCA financial data processing

Applied Mathematics and Nonlinear Sciences ◽

10.2478/amns.2021.1.00053 ◽

2021 ◽

Vol 0 (0) ◽

Author(s):

Zhengqing Li ◽

Jiliang Mu ◽

Mohammed Basheri ◽

Hafnida Hasan

Keyword(s):

Big Data ◽

Statistical Model ◽

Probability Density ◽

Mathematical Reasoning ◽

Statistical Design ◽

Financial Data ◽

Data Filtering ◽

Analysis Model ◽

Design Data ◽

Mathematical Probability

Abstract In order to improve the detection and filtering ability for financial data, a data-filtering method based on mathematical probability statistical model, a descriptive statistical analysis model of big data filtering, probability density characteristic statistical design data filtering analysis combined with fuzzy mathematical reasoning, regression analysis according to probability density of financial data distribution, and threshold test and threshold judgment are conducted to realize data filtering. The test results show that the big data filtering and the reliability and convergence of the mathematical model are optimal.

Download Full-text

The MapReduce based statistical model on maritime location big data

IOP Conference Series Earth and Environmental Science ◽

10.1088/1755-1315/310/2/022044 ◽

2019 ◽

Vol 310 ◽

pp. 022044

Author(s):

Bao wLei

Keyword(s):

Big Data ◽

Statistical Model

Download Full-text

Statistical model development for high-dimensional big data analytics

Journal of the Korean Data and Information Science Society ◽

10.7465/jkdi.2020.31.6.1009 ◽

2020 ◽

Vol 31 (6) ◽

pp. 1009-1020

Author(s):

Dae Heung Jang ◽

Il Do Ha ◽

Dong Jun Park ◽

In Ho Park ◽

Seung Jae Lee

Keyword(s):

Big Data ◽

Statistical Model ◽

Data Analytics ◽

Model Development ◽

Big Data Analytics ◽

High Dimensional ◽

Statistical Model Development

Download Full-text

Real Time Influenza Monitoring Using Hospital Big Data in Combination with Machine Learning Methods: Comparison Study (Preprint)

10.2196/preprints.11361 ◽

2018 ◽

Author(s):

Canelle Poirier ◽

Audrey Lavenu ◽

Valérie Bertaud ◽

Boris Campillo-Gimenez ◽

Emmanuel Chazard ◽

...

Keyword(s):

Big Data ◽

Statistical Model ◽

Real Time ◽

Statistical Models ◽

Incidence Rates ◽

Elastic Net ◽

University Hospital ◽

Support Vector ◽

Hospital Data ◽

Svm Model

BACKGROUND Traditional surveillance systems produce estimates of influenza-like illness (ILI) incidence rates, but with 1- to 3-week delay. Accurate real-time monitoring systems for influenza outbreaks could be useful for making public health decisions. Several studies have investigated the possibility of using internet users’ activity data and different statistical models to predict influenza epidemics in near real time. However, very few studies have investigated hospital big data. OBJECTIVE Here, we compared internet and electronic health records (EHRs) data and different statistical models to identify the best approach (data type and statistical model) for ILI estimates in real time. METHODS We used Google data for internet data and the clinical data warehouse eHOP, which included all EHRs from Rennes University Hospital (France), for hospital data. We compared 3 statistical models—random forest, elastic net, and support vector machine (SVM). RESULTS For national ILI incidence rate, the best correlation was 0.98 and the mean squared error (MSE) was 866 obtained with hospital data and the SVM model. For the Brittany region, the best correlation was 0.923 and MSE was 2364 obtained with hospital data and the SVM model. CONCLUSIONS We found that EHR data together with historical epidemiological information (French Sentinelles network) allowed for accurately predicting ILI incidence rates for the entire France as well as for the Brittany region and outperformed the internet data whatever was the statistical model used. Moreover, the performance of the two statistical models, elastic net and SVM, was comparable.

Download Full-text