Boosting Intelligent Data Analysis in Smart Sensors by Integrating Knowledge and Machine Learning

The presented paper proposes a hybrid neural architecture that enables intelligent data analysis efficacy to be boosted in smart sensor devices, which are typically resource-constrained and application-specific. The postulated concept integrates prior knowledge with learning from examples, thus allowing sensor devices to be used for the successful execution of machine learning even when the volume of training data is highly limited, using compact underlying hardware. The proposed architecture comprises two interacting functional modules arranged in a homogeneous, multiple-layer architecture. The first module, referred to as the knowledge sub-network, implements knowledge in the Conjunctive Normal Form through a three-layer structure composed of novel types of learnable units, called L-neurons. In contrast, the second module is a fully-connected conventional three-layer, feed-forward neural network, and it is referred to as a conventional neural sub-network. We show that the proposed hybrid structure successfully combines knowledge and learning, providing high recognition performance even for very limited training datasets, while also benefiting from an abundance of data, as it occurs for purely neural structures. In addition, since the proposed L-neurons can learn (through classical backpropagation), we show that the architecture is also capable of repairing its knowledge.

Hadoop solution for large data protection

Investigated one of large data problems of - providing protection in the process of accumulation and processing. The case of application of Hadoop technology and its latest modification Apache Hadoop 3.3.0 is considered. A solution is proposed with strengthening the protection of processed data, connecting the Apache Knox Gateway, Apache Ranger and Apache Atlas tools. The possibil-ity of using data obtained as a result of the work of local databases, electronic archives, database management systems and individual users is provided. The solution also features the use of a pri-vate cloud and cryptographic algorithms. An example of the implementation of a secure solution to the problem of Intelligent Data Analysis is given on the example of a parallel version of the problem of finding association rules when working with unstructured data of large volumes.

ENSEMBLE MODELS FOR INTELLIGENT ANALYSIS OF SOCIOLOGICAL DATA

The paper investigates an approach to sociological information processing based on the use of intelligent data analysis methods applied to the task of processing the results of a questionnaire survey. The advantages of intelligent analysis of sociological data in comparison with traditional statistical processing are discussed, as well as the implementation features and applicability limits of various intelligent data analysis methods in solving problems of association, clustering and classification. Structure and features of representation of respondents’ survey data are considered, the appropriateness is substantiated and the advantages of their processing based on the combination of various methods of intelligent analysis within an ensemble of models are discussed. A structure of an ensemble of models is proposed based on the combination and joint use of association rules, clustering algorithms and decision trees, which makes it possible to jointly process numerical and categorical data contained in the respondent’s answers to the questionnaire and also to interpret the results of data clustering. The paper describes the results of using the constructed ensemble of models for processing and analyzing the data of a sociological survey conducted as part of the annual project for monitoring the drug abuse situation in the Bryansk region in 2013 – 2018. The use of an ensemble of intelligent data analysis models for processing the results of a sociological survey not only makes it possible to detect patterns in them that cannot be otherwise detected by traditional methods of statistical processing, but also contributes to an increase in the reliability, completeness and coherence of the analysis results, due to which the analyst creates a holistic systemic picture of the studied social phenomenon or process.

La intensificación "Análisis Inteligente de Datos": una experiencia de Aprendizaje Basado en Proyectos

The Universitat Politècnica de València and its Faculty of Business Administration and Management have created a new intensification, named, "Intelligent Data Analysis", that provides the student with sufficient knowledge to integrate data analysis in the sometimes routine tasks of a company.The statistical, computer and ICT-related skills obtained through the Business Administration and Management degree are enhanced with more advanced statiscal models for multivariate data analysis and with R language programming, which is very suitable for such data analysis. All these skills are acquired under the Project-Based Learning methodology.This project's main achievement has been the coordination between the different subjects of the intensification to use the same software, which has resulted in a continuity for the way in which students work with RStudio, R, and Rmakdown. This has provided them a high level of management and integration of data analysis in the students’ work routines which will later aid them to become more qualified professionals.