Abstract
With the rapid advancement of big data technology and statistical data analysis solutions, the computing of big data and its services has become the subject of research and popular applications. There are many problems related to data quality that lead to making wrong decisions in institutions and companies. Current research rarely discusses how to validate data effectively to ensure its quality Integrity is data validity. It is a task that is not an easy task that is usually undertaken in national statistical organizations and institutions. There is an urgent need to produce a general framework to verify the integrity of big data. This methodology has been devoted to proposing a model that works on data integrity, especially big data, and how to address the validation process. The data also includes the validity of the data fields and the validity of measuring the data and assessing the compatibility with the data cycle chain. The speed of the processing process and the accuracy of the verification process for the integrity of big data are considered. Based on using the latest technologies and programming languages, the research was based on the programming language in Python and real test data.
Implementing Integrity Assurance System for Big Data