scholarly journals Extreme Compression for Large Scale Data Store

2020 ◽  
Vol 245 ◽  
pp. 06024
Author(s):  
Jérôme Lauret ◽  
Juan Gonzalez ◽  
Gene Van Buren ◽  
Rafael Nuñez ◽  
Philippe Canal ◽  
...  

For the last 5 years Accelogic pioneered and perfected a radically new theory of numerical computing codenamed “Compressive Computing”, which has an extremely profound impact on real-world computer science [1]. At the core of this new theory is the discovery of one of its fundamental theorems which states that, under very general conditions, the vast majority (typically between 70% and 80%) of the bits used in modern large-scale numerical computations are absolutely irrelevant for the accuracy of the end result. This theory of Compressive Computing provides mechanisms able to identify (with high intelligence and surgical accuracy) the number of bits (i.e., the precision) that can be used to represent numbers without affecting the substance of the end results, as they are computed and vary in real time. The bottom line outcome would be to provide a state-of-the-art compression algorithm that surpasses those currently available in the ROOT framework, with the purpose of enabling substantial economic and operational gains (including speedup) for High Energy and Nuclear Physics data storage/analysis. In our initial studies, a factor of nearly x4 (3.9) compression was achieved with RHIC/STAR data where ROOT compression managed only x1.4. In this contribution, we will present our concepts of “functionally lossless compression”, have a glance at examples and achievements in other communities, present the results and outcome of our current, ongoing R&D, as well as present a high-level view of our plan to move forward with a ROOT implementation that would deliver a basic solution readily integrated into HENP applications. As a collaboration of experimental scientists, private industry, and the ROOT Team, our aim is to capitalize on the substantial success delivered by the initial effort and produce a robust technology properly packaged as an open-source tool that could be used by virtually every experiment around the world as means for improving data management and accessibility.

Author(s):  
Igor Bychkov ◽  
Andrey Demichev ◽  
Julia Dubenskaya ◽  
Oleg Fedorov ◽  
Andreas Haungs ◽  
...  

Modern experimental astroparticle physics features large-scale setups measuring different messengers, namely high-energy particles generated by cosmic accelerators (e.g. supernova remnants, active galactic nuclei, etc): cosmic and gamma rays, neutrinos and recently discovered gravitational waves. Ongoing and future experiments are distributed over the Earth including ground, underground/underwater setups as well as balloon payloads and spacecrafts. The data acquired by these experiments have different formats, storage concepts and publication policies. Such differences are a crucial issue in the era of big data and of multi-messenger analysis strategies in astroparticle physics. We propose a service ASTROPARTICLE.ONLINE in the frame of which we develop an open science system which enables to publish, store, search, select and analyse astroparticle physics data. The cosmic-ray experiments KASCADE-Grande and TAIGA were chosen as pilot experiments to be included in this framework. In the first step of our initiative we will develop and test the following components of the full data life cycle concept: (i) describing, storing and reusing of astroparticle data; (ii) software for performing multi-experiment and multi-messenger analyses like deep-learning methods; (iii) outreach including example applications and tutorial for students and scientists outside the specific research field. In the present paper we describe the concepts of our initiative, and in particular the plans toward a common, federated astroparticle data storage.


2019 ◽  
Author(s):  
Adrian Concepcion Leon ◽  
Markus Endler

Blockchain and Tangle are data structures used to create an immutable public record of data insured by a network of peer-to-peer participants who maintain a set of constantly growing data records known as ledgers. Blockchain and Tangle technologies are a decentralized solution that guarantees the exchange of large amounts of trusted messages, among billions of connected IoT devices, which are very valuable as they are valid and complete. This highly encrypted and secure peer-to-peer messaging mechanism is adopted in this project to manage the processing of IoT transactions and the coordination between the devices that interact with the process. To maintain private transactions, secure and trustless, the distributed consensus algorithms are responsible for validating and choosing transactions and recording them in the global ledger. The results showed that the speed of the consensus algorithms can affect the creation in real time of reliable stories that track the events of the IoT networks. After incorporating Complex Event Processing that allows selecting only those high level events, it is possible to get an improvement in many situations. The result is a Middleware system that provides a framework for the construction of large-scale computer applications that use Complex Events Processing and different decentralized ledgers such as the blockchain of Ethereum or IOTA Tangle, for secure data storage.


2020 ◽  
Vol 245 ◽  
pp. 02030
Author(s):  
Jakob Blomer ◽  
Philippe Canal ◽  
Axel Naumann ◽  
Danilo Piparo

The ROOT TTree data format encodes hundreds of petabytes of High Energy and Nuclear Physics events. Its columnar layout drives rapid analyses, as only those parts (“branches”) that are really used in a given analysis need to be read from storage. Its unique feature is the seamless C++ integration, which allows users to directly store their event classes without explicitly defining data schemas. In this contribution, we present the status and plans of the future ROOT 7 event I/O. Along with the ROOT 7 interface modernization, we aim for robust, where possible compile-time safe C++ interfaces to read and write event data. On the performance side, we show first benchmarks using ROOT’s new experimental I/O subsystem that combines the best of TTrees with recent advances in columnar data formats. A core ingredient is a strong separation of the high-level logical data layout (C++ classes) from the low-level physical data layout (storage backed nested vectors of simple types). We show how the new, optimized physical data layout speeds up serialization and deserialization and facilitates parallel, vectorized and bulk operations. This lets ROOT I/O run optimally on the upcoming ultra-fast NVRAM storage devices, as well as file-less storage systems such as object stores.


Author(s):  
Georgi Derluguian

The author develops ideas about the origin of social inequality during the evolution of human societies and reflects on the possibilities of its overcoming. What makes human beings different from other primates is a high level of egalitarianism and altruism, which contributed to more successful adaptability of human collectives at early stages of the development of society. The transition to agriculture, coupled with substantially increasing population density, was marked by the emergence and institutionalisation of social inequality based on the inequality of tangible assets and symbolic wealth. Then, new institutions of warfare came into existence, and they were aimed at conquering and enslaving the neighbours engaged in productive labour. While exercising control over nature, people also established and strengthened their power over other people. Chiefdom as a new type of polity came into being. Elementary forms of power (political, economic and ideological) served as a basis for the formation of early states. The societies in those states were characterised by social inequality and cruelties, including slavery, mass violence and numerous victims. Nowadays, the old elementary forms of power that are inherent in personalistic chiefdom are still functioning along with modern institutions of public and private bureaucracy. This constitutes the key contradiction of our time, which is the juxtaposition of individual despotic power and public infrastructural one. However, society is evolving towards an ever more efficient combination of social initiatives with the sustainability and viability of large-scale organisations.


2015 ◽  
Vol 51 (91) ◽  
pp. 16381-16384 ◽  
Author(s):  
Yuelong Xin ◽  
Liya Qi ◽  
Yiwei Zhang ◽  
Zicheng Zuo ◽  
Henghui Zhou ◽  
...  

A novel organic solvent-assisted freeze-drying pathway, which can effectively protect and uniformly distribute active particles, is developed to fabricate a free-standing Li2MnO3·LiNi1/3Co1/3Mn1/3O2 (LR)/rGO electrode on a large scale.


Author(s):  
Zhiqiang Luo ◽  
Silin Zheng ◽  
Shuo Zhao ◽  
Xin Jiao ◽  
Zongshuai Gong ◽  
...  

Benzoquinone with high theoretical capacity is anchored on N-plasma engraved porous carbon as a desirable cathode for rechargeable aqueous Zn-ion batteries. Such batteries display tremendous potential in large-scale energy storage applications.


2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Lorenz T. Keyßer ◽  
Manfred Lenzen

Abstract1.5  °C scenarios reported by the Intergovernmental Panel on Climate Change (IPCC) rely on combinations of controversial negative emissions and unprecedented technological change, while assuming continued growth in gross domestic product (GDP). Thus far, the integrated assessment modelling community and the IPCC have neglected to consider degrowth scenarios, where economic output declines due to stringent climate mitigation. Hence, their potential to avoid reliance on negative emissions and speculative rates of technological change remains unexplored. As a first step to address this gap, this paper compares 1.5  °C degrowth scenarios with IPCC archetype scenarios, using a simplified quantitative representation of the fuel-energy-emissions nexus. Here we find that the degrowth scenarios minimize many key risks for feasibility and sustainability compared to technology-driven pathways, such as the reliance on high energy-GDP decoupling, large-scale carbon dioxide removal and large-scale and high-speed renewable energy transformation. However, substantial challenges remain regarding political feasibility. Nevertheless, degrowth pathways should be thoroughly considered.


Nanophotonics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 3965-3975 ◽  
Author(s):  
Dmitry Yu. Fedyanin ◽  
Alexey V. Krasavin ◽  
Aleksey V. Arsenin ◽  
Anatoly V. Zayats

AbstractPlasmonics offers a unique opportunity to break the diffraction limit of light and bring photonic devices to the nanoscale. As the most prominent example, an integrated nanolaser is a key to truly nanoscale photonic circuits required for optical communication, sensing applications and high-density data storage. Here, we develop a concept of an electrically driven subwavelength surface-plasmon-polariton nanolaser, which is based on a novel amplification scheme, with all linear dimensions smaller than the operational free-space wavelength λ and a mode volume of under λ3/30. The proposed pumping approach is based on a double-heterostructure tunneling Schottky barrier diode and gives the possibility to reduce the physical size of the device and ensure in-plane emission so that the nanolaser output can be naturally coupled to a plasmonic or nanophotonic waveguide circuitry. With the high energy efficiency (8% at 300 K and 37% at 150 K), the output power of up to 100 μW and the ability to operate at room temperature, the proposed surface plasmon polariton nanolaser opens up new avenues in diverse application areas, ranging from ultrawideband optical communication on a chip to low-power nonlinear photonics, coherent nanospectroscopy, and single-molecule biosensing.


Genetics ◽  
2001 ◽  
Vol 159 (4) ◽  
pp. 1765-1778
Author(s):  
Gregory J Budziszewski ◽  
Sharon Potter Lewis ◽  
Lyn Wegrich Glover ◽  
Jennifer Reineke ◽  
Gary Jones ◽  
...  

Abstract We have undertaken a large-scale genetic screen to identify genes with a seedling-lethal mutant phenotype. From screening ~38,000 insertional mutant lines, we identified >500 seedling-lethal mutants, completed cosegregation analysis of the insertion and the lethal phenotype for >200 mutants, molecularly characterized 54 mutants, and provided a detailed description for 22 of them. Most of the seedling-lethal mutants seem to affect chloroplast function because they display altered pigmentation and affect genes encoding proteins predicted to have chloroplast localization. Although a high level of functional redundancy in Arabidopsis might be expected because 65% of genes are members of gene families, we found that 41% of the essential genes found in this study are members of Arabidopsis gene families. In addition, we isolated several interesting classes of mutants and genes. We found three mutants in the recently discovered nonmevalonate isoprenoid biosynthetic pathway and mutants disrupting genes similar to Tic40 and tatC, which are likely to be involved in chloroplast protein translocation. Finally, we directly compared T-DNA and Ac/Ds transposon mutagenesis methods in Arabidopsis on a genome scale. In each population, we found only about one-third of the insertion mutations cosegregated with a mutant phenotype.


Sign in / Sign up

Export Citation Format

Share Document