Investigation of parameters recorded by the monitoring system of an engineering structure using big data analysis methods

2021 ◽  
pp. 53-60
Author(s):  
В.А. Коршунов ◽  
В.Н. Кузнецова ◽  
Р.С. Мудрик ◽  
Д.А. Пономарев ◽  
А.А. Родионов
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Analysis ◽  
Monitoring System ◽  
Strain State ◽  
Big Data Analysis ◽  
Stress Strain ◽  
Engineering Structure ◽  
Stress Strain State ◽  
Marine Engineering

В работе представлены результаты анализа и обработки с помощью математического аппарата «науки о больших данных» информации, получаемой в системе мониторинга напряженно-деформированного состояния морского инженерного сооружения. Для анализа данных использована процедура машинного обучения с учителем, рассмотрены различные модели машинного обучения. Приведен пример анализа результатов работы системы мониторинга напряженно-деформированного состояния морской нефтедобывающей платформы. Предложены варианты оптимизации системы путем уменьшения числа рассматриваемых датчиков и замены их математической моделью. Созданы математические модели для восстановления значений специально исключенных датчиков, а также для построения краткосрочного и долгосрочного прогнозов. С помощью анализа больших данных получена новая информация пригодная для практических и научных целей. The paper presents the results of analysis and processing with the help of the mathematical apparatus of the "science of big data" of information received in the monitoring system of the stress-strain state of a marine engineering structure. For data analysis, the machine learning procedure with a teacher is used, various machine learning models are considered. An example of the analysis of the results of the monitoring system of the stress-strain state of an offshore oil production platform is given. Options for optimizing the system are proposed - reducing the number of sensors under consideration. Mathematical models have been created to restore the values of specially excluded sensors, as well as to build short-and long-term forecasts. With the help of big data analysis, new information has been obtained that is suitable for practical and scientific purposes.

The Rain Prediction in Australia Based Big Data Analysis and Machine Learning Technology

2021 ◽  
Author(s):  
Bohdan Polishchuk ◽  
Andrii Berko ◽  
Lyubomyr Chyrun ◽  
Myroslava Bublyk ◽  
Vadim Schuchmann
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Analysis ◽  
Big Data Analysis ◽  
Learning Technology

scholarly journals An Automated Analytics Engine for College Program Selection using Machine Learning and Big Data Analysis

2021 ◽  
Author(s):  
Jinhui Yu ◽  
Xinyu Luan ◽  
Yu Sun
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Analysis ◽  
Big Data Analysis ◽  
Web Pages ◽  
Python Language ◽  
Program Selection ◽  
Front End

Because of the differences in the structure and content of each website, it is often difficult for international applicants to obtain the application information of each school in time. They need to spend a lot of time manually collecting and sorting information. Especially when the information of the school may be constantly updated, the information may become very inaccurate for international applicants. we designed a tool including three main steps to solve the problem: crawling links, processing web pages, and building my pages. In compiling languages, we mainly use Python and store the crawled data in JSON format [4]. In the process of crawling links, we mainly used beautiful soup to parse HTML and designed crawler. In this paper, we use Python language to design a system. First, we use the crawler method to fetch all the links related to the admission information on the school's official website. Then we traverse these links, and use the noise_remove [5] method to process their corresponding page contents, so as to further narrow the scope of effective information and save these processed contents in the JSON files. Finally, we use the Flask framework to integrate these contents into my front-end page conveniently and efficiently, so that it has the complete function of integrating and displaying information.

Techniques and Methods That Help to Make Big Data the Simplest Recipe for Success

2019 ◽  
pp. 161-194
Keyword(s):  
Machine Learning ◽  
Big Data ◽  
Data Analysis ◽  
Statistical Analysis ◽  
Data Analytics ◽  
Big Data Analysis ◽  
Customer Segmentation ◽  
Learning Context ◽  
Feature Vectors

Data analytics has grown in a machine learning context. Whatever the reason data is used or exploited, customer segmentation or marketing targeting, it must be processed first and represented on feature vectors. Many algorithms, such as clustering, regression, classification, and others, need to be represented and clarified in order to facilitate processing and statistical analysis. If we have seen, through the previous chapters, the importance of big data analysis (the Why?), as with every major innovation, the biggest confusion lies in the exact scope (What?) and its implementation (How?). In this chapter, we will take a look at the different algorithms and techniques analytics that we can use in order to exploit the large amounts of data.

Sign in / Sign up

Export Citation Format

Share Document