scholarly journals Secure distributed ledgers to support IoT technologies data

2019 ◽  
Author(s):  
Adrian Concepcion Leon ◽  
Markus Endler
Keyword(s):  
Data Storage ◽  
Data Structures ◽  
Large Scale ◽  
Peer To Peer ◽  
Computer Applications ◽  
Consensus Algorithms ◽  
Complex Events ◽  
Public Record ◽  
Iot Devices ◽  
High Level

Blockchain and Tangle are data structures used to create an immutable public record of data insured by a network of peer-to-peer participants who maintain a set of constantly growing data records known as ledgers. Blockchain and Tangle technologies are a decentralized solution that guarantees the exchange of large amounts of trusted messages, among billions of connected IoT devices, which are very valuable as they are valid and complete. This highly encrypted and secure peer-to-peer messaging mechanism is adopted in this project to manage the processing of IoT transactions and the coordination between the devices that interact with the process. To maintain private transactions, secure and trustless, the distributed consensus algorithms are responsible for validating and choosing transactions and recording them in the global ledger. The results showed that the speed of the consensus algorithms can affect the creation in real time of reliable stories that track the events of the IoT networks. After incorporating Complex Event Processing that allows selecting only those high level events, it is possible to get an improvement in many situations. The result is a Middleware system that provides a framework for the construction of large-scale computer applications that use Complex Events Processing and different decentralized ledgers such as the blockchain of Ethereum or IOTA Tangle, for secure data storage.

scholarly journals Extreme Compression for Large Scale Data Store

2020 ◽  
Vol 245 ◽  
pp. 06024
Author(s):  
Jérôme Lauret ◽  
Juan Gonzalez ◽  
Gene Van Buren ◽  
Rafael Nuñez ◽  
Philippe Canal ◽  
...  
Keyword(s):  
Data Storage ◽  
Nuclear Physics ◽  
Large Scale ◽  
High Energy ◽  
Basic Solution ◽  
Private Industry ◽  
Bottom Line ◽  
High Intelligence ◽  
Numerical Computing ◽  
High Level

For the last 5 years Accelogic pioneered and perfected a radically new theory of numerical computing codenamed “Compressive Computing”, which has an extremely profound impact on real-world computer science [1]. At the core of this new theory is the discovery of one of its fundamental theorems which states that, under very general conditions, the vast majority (typically between 70% and 80%) of the bits used in modern large-scale numerical computations are absolutely irrelevant for the accuracy of the end result. This theory of Compressive Computing provides mechanisms able to identify (with high intelligence and surgical accuracy) the number of bits (i.e., the precision) that can be used to represent numbers without affecting the substance of the end results, as they are computed and vary in real time. The bottom line outcome would be to provide a state-of-the-art compression algorithm that surpasses those currently available in the ROOT framework, with the purpose of enabling substantial economic and operational gains (including speedup) for High Energy and Nuclear Physics data storage/analysis. In our initial studies, a factor of nearly x4 (3.9) compression was achieved with RHIC/STAR data where ROOT compression managed only x1.4. In this contribution, we will present our concepts of “functionally lossless compression”, have a glance at examples and achievements in other communities, present the results and outcome of our current, ongoing R&D, as well as present a high-level view of our plan to move forward with a ROOT implementation that would deliver a basic solution readily integrated into HENP applications. As a collaboration of experimental scientists, private industry, and the ROOT Team, our aim is to capitalize on the substantial success delivered by the initial effort and produce a robust technology properly packaged as an open-source tool that could be used by virtually every experiment around the world as means for improving data management and accessibility.

Large-Scale Data Streaming in Fog Computing and Its Applications

2021 ◽  
pp. 50-65
Author(s):  
Oshin Sharma ◽  
Anusha S.
Keyword(s):  
Data Storage ◽  
Large Scale ◽  
Fog Computing ◽  
Data Streaming ◽  
Cloud Data ◽  
Large Scale Data ◽  
Emerging Trends ◽  
Iot Devices ◽  
Data Centres ◽  
Scale Data

The emerging trends in fog computing have increased the interests and focus in both industry and academia. Fog computing extends cloud computing facilities like the storage, networking, and computation towards the edge of networks wherein it offloads the cloud data centres and reduces the latency of providing services to the users. This paradigm is like cloud in terms of data, storage, application, and computation services, except with a fundamental difference: it is decentralized. Furthermore, these fog systems can process huge amounts of data locally and can be installed on hardware of different types. These characteristics make fog suitable for time- and location-based applications like internet of things (IoT) devices which can process large amounts of data. In this chapter, the authors present fog data streaming, its architecture, and various applications.

scholarly journals Data-driven traffic and diffusion modeling in peer-to-peer networks: A real case study

2014 ◽  
Vol 2 (3) ◽  
pp. 341-366
Author(s):  
ROMAIN HOLLANDERS ◽  
DANIEL F. BERNARDES ◽  
BIVAS MITRA ◽  
RAPHAËL M. JUNGERS ◽  
JEAN-CHARLES DELVENNE ◽  
...  
Keyword(s):  
Large Scale ◽  
Synthetic Data ◽  
Real Data ◽  
Peer To Peer ◽  
Free Riders ◽  
Peer Network ◽  
Peer To Peer Networks ◽  
Peer To Peer Systems ◽  
High Level ◽  
And Control

AbstractPeer-to-peer systems have driven a lot of attention in the past decade as they have become a major source of Internet traffic. The amount of data flowing through the peer-to-peer network is huge and hence challenging both to comprehend and to control. In this work, we take advantage of a new and rich dataset recording the peer-to-peer activity at a remarkable scale to address these difficult problems. After extracting the relevant and measurable properties of the network from the data, we develop two models that aim to make the link between the low-level properties of the network, such as the proportion of peers that do not share content (i.e., free riders) or the distribution of the files among the peers, and its high-level properties, such as the Quality of Service or the diffusion of content, which are of interest for supervision and control purposes. We observe a significant agreement between the high-level properties measured on the real data and on the synthetic data generated by our models, which is encouraging for our models to be used in practice as large-scale prediction tools. Relying on them, we demonstrate that spending efforts to reduce the amount of free riders indeed helps to improve the availability of files on the network. We observe however a saturation of this phenomenon after 60% of free riders.

scholarly journals Tool for SPARQL Querying over Compact RDF Representations

2021 ◽  
Vol 7 (1) ◽  
pp. 33
Author(s):  
Delfina Ramos-Vidal ◽  
Guillermo de Bernardo
Keyword(s):  
Data Storage ◽  
Data Structures ◽  
Low Level ◽  
Level Data ◽  
Query Engine ◽  
Compact Data Structures ◽  
High Level

We present an architecture for the efficient storing and querying of large RDF datasets. Our approach seeks to store RDF datasets in very little space while offering complete SPARQL functionality. To achieve this, our proposal was built over HDT, an RDF serialization framework, and its interaction with the Jena query engine. We propose a set of modifications to this framework in order to incorporate a range of space-efficient compact data structures for data storage and access, while using high-level capabilities to answer more complicated SPARQL queries. As a result, our approach provides a standard mechanism for using low-level data structures in complicated query situations requiring SPARQL searches, which are typically not supported by current solutions.

scholarly journals Efficient Byzantine Consensus Mechanism Based on Reputation in IoT Blockchain

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Xu Yuan ◽  
Fang Luo ◽  
Muhammad Zeeshan Haider ◽  
Zhikui Chen ◽  
Yucheng Li
Keyword(s):  
Internet Of Things ◽  
Large Scale ◽  
Consensus Algorithm ◽  
Consensus Algorithms ◽  
Heterogeneous Devices ◽  
Blockchain Technology ◽  
Critical Problems ◽  
Iot Devices ◽  
Reliability And Robustness ◽  
The Internet Of Things

Blockchain technology has advanced rapidly in recent years and is now widely used in a variety of fields. Blockchain appears to be one of the best solutions for managing massive heterogeneous devices while achieving advanced data security and data reputation, particularly in the field of large-scale IoT (Internet of Things) networks. Despite the numerous advantages, there are still challenges while deploying IoT applications on blockchain systems due to the limited storage, power, and computing capability of IoT devices, and some of these problems are caused by the consensus algorithm, which plays a significant role in blockchain systems by ensuring overall system reliability and robustness. Nonetheless, most existing consensus algorithms are prone to poor node reliability, low transaction per second (TPS) rates, and scalability issues. Aiming at some critical problems in the existing consensus algorithms, this paper proposes the Efficient Byzantine Reputation-based Consensus (EBRC) mechanism to resolve the issues raised above. In comparison to traditional algorithms, we reinvented ways to evaluate node reliability and robustness and manage active nodes. Our experiments show that the EBRC algorithm has lower consensus delay, higher throughput, improved security, and lower verification costs. It offers new reference ideas for solving the Internet of Things+blockchain+Internet court construction problem.

scholarly journals IoT Big Data provenance scheme using blockchain on Hadoop ecosystem

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Houshyar Honar Pajooh ◽  
Mohammed A. Rashid ◽  
Fakhrul Alam ◽  
Serge Demidenko
Keyword(s):  
Big Data ◽  
Data Storage ◽  
Large Scale ◽  
Data System ◽  
Third Party ◽  
Data Provenance ◽  
Distributed Data ◽  
Distributed Data Storage ◽  
Blockchain Technology ◽  
Iot Devices

AbstractThe diversity and sheer increase in the number of connected Internet of Things (IoT) devices have brought significant concerns associated with storing and protecting a large volume of IoT data. Storage volume requirements and computational costs are continuously rising in the conventional cloud-centric IoT structures. Besides, dependencies of the centralized server solution impose significant trust issues and make it vulnerable to security risks. In this paper, a layer-based distributed data storage design and implementation of a blockchain-enabled large-scale IoT system are proposed. It has been developed to mitigate the above-mentioned challenges by using the Hyperledger Fabric (HLF) platform for distributed ledger solutions. The need for a centralized server and a third-party auditor was eliminated by leveraging HLF peers performing transaction verifications and records audits in a big data system with the help of blockchain technology. The HLF blockchain facilitates storing the lightweight verification tags on the blockchain ledger. In contrast, the actual metadata are stored in the off-chain big data system to reduce the communication overheads and enhance data integrity. Additionally, a prototype has been implemented on embedded hardware showing the feasibility of deploying the proposed solution in IoT edge computing and big data ecosystems. Finally, experiments have been conducted to evaluate the performance of the proposed scheme in terms of its throughput, latency, communication, and computation costs. The obtained results have indicated the feasibility of the proposed solution to retrieve and store the provenance of large-scale IoT data within the Big Data ecosystem using the HLF blockchain. The experimental results show the throughput of about 600 transactions, 500 ms average response time, about 2–3% of the CPU consumption at the peer process and approximately 10–20% at the client node. The minimum latency remained below 1 s however, there is an increase in the maximum latency when the sending rate reached around 200 transactions per second (TPS).

scholarly journals QoS monitoring in real-time streaming overlays based on lock-free data structures

2021 ◽  
Author(s):  
Franco Tommasi ◽  
Valerio De Luca ◽  
Catiuscia Melle
Keyword(s):  
Real Time ◽  
Data Structures ◽  
Large Scale ◽  
Video Quality ◽  
Peer To Peer ◽  
Free Data ◽  
Video Quality Metrics ◽  
Qos Monitoring ◽  
Audio Video ◽  
Monitoring Algorithm

AbstractPeer-to-peer streaming is a well-known technology for the large-scale distribution of real-time audio/video contents. Delay requirements are very strict in interactive real-time scenarios (such as synchronous distance learning), where playback lag should be of the order of seconds. Playback continuity is another key aspect in these cases: in presence of peer churning and network congestion, a peer-to-peer overlay should quickly rearrange connections among receiving nodes to avoid freezing phenomena that may compromise audio/video understanding. For this reason, we designed a QoS monitoring algorithm that quickly detects broken or congested links: each receiving node is able to independently decide whether it should switch to a secondary sending node, called “fallback node”. The architecture takes advantage of a multithreaded design based on lock-free data structures, which improve the performance by avoiding synchronization among threads. We will show the good responsiveness of the proposed approach on machines with different computational capabilities: measured times prove both departures of nodes and QoS degradations are promptly detected and clients can quickly restore a stream reception. According to PSNR and SSIM, two well-known full-reference video quality metrics, QoE remains acceptable on receiving nodes of our resilient overlay also in presence of swap procedures.

Sketching and Sublinear Data Structures in Genomics

2019 ◽  
Vol 2 (1) ◽  
pp. 93-118 ◽  
Author(s):  
Guillaume Marçais ◽  
Brad Solomon ◽  
Rob Patro ◽  
Carl Kingsford
Keyword(s):  
Computational Methods ◽  
Data Structures ◽  
Full Text ◽  
Large Scale ◽  
Processing Time ◽  
Genomic Analysis ◽  
Locality Sensitive Hashing ◽  
Membership Query ◽  
Analysis Methods ◽  
High Level

Large-scale genomics demands computational methods that scale sublinearly with the growth of data. We review several data structures and sketching techniques that have been used in genomic analysis methods. Specifically, we focus on four key ideas that take different approaches to achieve sublinear space usage and processing time: compressed full-text indices, approximate membership query data structures, locality-sensitive hashing, and minimizers schemes. We describe these techniques at a high level and give several representative applications of each.

Social inequality in the evolution of human societies

2019 ◽  
pp. 98-120
Author(s):  
Georgi Derluguian
Keyword(s):  
Social Inequality ◽  
Large Scale ◽  
Human Beings ◽  
Mass Violence ◽  
Public And Private ◽  
Tangible Assets ◽  
Transition To Agriculture ◽  
New Institutions ◽  
High Level ◽  
Political Economic

The author develops ideas about the origin of social inequality during the evolution of human societies and reflects on the possibilities of its overcoming. What makes human beings different from other primates is a high level of egalitarianism and altruism, which contributed to more successful adaptability of human collectives at early stages of the development of society. The transition to agriculture, coupled with substantially increasing population density, was marked by the emergence and institutionalisation of social inequality based on the inequality of tangible assets and symbolic wealth. Then, new institutions of warfare came into existence, and they were aimed at conquering and enslaving the neighbours engaged in productive labour. While exercising control over nature, people also established and strengthened their power over other people. Chiefdom as a new type of polity came into being. Elementary forms of power (political, economic and ideological) served as a basis for the formation of early states. The societies in those states were characterised by social inequality and cruelties, including slavery, mass violence and numerous victims. Nowadays, the old elementary forms of power that are inherent in personalistic chiefdom are still functioning along with modern institutions of public and private bureaucracy. This constitutes the key contradiction of our time, which is the juxtaposition of individual despotic power and public infrastructural one. However, society is evolving towards an ever more efficient combination of social initiatives with the sustainability and viability of large-scale organisations.

scholarly journals IoT Traffic: Modeling and Measurement Experiments

IoT ◽  
2021 ◽  
Vol 2 (1) ◽  
pp. 140-162
Author(s):  
Hung Nguyen-An ◽  
Thomas Silverston ◽  
Taku Yamazaki ◽  
Takumi Miyoshi
Keyword(s):  
Smart Home ◽  
Large Scale ◽  
Traffic Modeling ◽  
The Novel ◽  
Network Behavior ◽  
Performance Accuracy ◽  
Traffic Generator ◽  
Multiple Devices ◽  
Iot Devices ◽  
The Internet Of Things

We now use the Internet of things (IoT) in our everyday lives. The novel IoT devices collect cyber–physical data and provide information on the environment. Hence, IoT traffic will count for a major part of Internet traffic; however, its impact on the network is still widely unknown. IoT devices are prone to cyberattacks because of constrained resources or misconfigurations. It is essential to characterize IoT traffic and identify each device to monitor the IoT network and discriminate among legitimate and anomalous IoT traffic. In this study, we deployed a smart-home testbed comprising several IoT devices to study IoT traffic. We performed extensive measurement experiments using a novel IoT traffic generator tool called IoTTGen. This tool can generate traffic from multiple devices, emulating large-scale scenarios with different devices under different network conditions. We analyzed the IoT traffic properties by computing the entropy value of traffic parameters and visually observing the traffic on behavior shape graphs. We propose a new method for identifying traffic entropy-based devices, computing the entropy values of traffic features. The method relies on machine learning to classify the traffic. The proposed method succeeded in identifying devices with a performance accuracy up to 94% and is robust with unpredictable network behavior with traffic anomalies spreading in the network.

Sign in / Sign up

Export Citation Format

Share Document