Distributed Top-K Join Queries Optimizing for RDF Datasets

2021 ◽  
pp. 1895-1913
Author(s):  
Jinguang Gu ◽  
Hao Dong ◽  
Zhao Liu ◽  
Fangfang Xu
Keyword(s):  
Distributed Storage ◽  
Storage System ◽  
Computing System ◽  
Intermediate Data ◽  
System A ◽  
Speed Up ◽  
Novel Method ◽  
Increasing Demand ◽  
The Cost ◽  
Cache Mechanism

In recent years, the scale of RDF datasets is increasing rapidly, the query research on RDF datasets in the transitional centralized environment is unable to meet the increasing demand of data query field, especially the top-k query. Based on the Spark distributed computing system and the HBase distributed storage system, a novel method is proposed for top-k query. A top–k query plan STA (Spark Threshold Algorithm) is proposed to reduce the connection operation of RDF data. Furthermore, a better algorithm SSJA (Spark Simple Join Algorithm) is presented to reduce the sorting related operations for the intermediate data. A cache mechanism is also proposed to speed up the SSJA algorithm. The experimental results show that the SSJA algorithm performs better than the STA algorithm in term of the cost and applicability, and it can significantly improve the SSJA's performance by introducing the cache mechanism.

Distributed Top-K Join Queries Optimizing for RDF Datasets

2017 ◽  
Vol 14 (3) ◽  
pp. 67-83
Author(s):  
Jinguang Gu ◽  
Hao Dong ◽  
Zhao Liu ◽  
Fangfang Xu
Keyword(s):  
Distributed Storage ◽  
Storage System ◽  
Computing System ◽  
Intermediate Data ◽  
System A ◽  
Speed Up ◽  
Novel Method ◽  
Increasing Demand ◽  
The Cost ◽  
Cache Mechanism

In recent years, the scale of RDF datasets is increasing rapidly, the query research on RDF datasets in the transitional centralized environment is unable to meet the increasing demand of data query field, especially the top-k query. Based on the Spark distributed computing system and the HBase distributed storage system, a novel method is proposed for top-k query. A top–k query plan STA (Spark Threshold Algorithm) is proposed to reduce the connection operation of RDF data. Furthermore, a better algorithm SSJA (Spark Simple Join Algorithm) is presented to reduce the sorting related operations for the intermediate data. A cache mechanism is also proposed to speed up the SSJA algorithm. The experimental results show that the SSJA algorithm performs better than the STA algorithm in term of the cost and applicability, and it can significantly improve the SSJA's performance by introducing the cache mechanism.

scholarly journals MODERN SOLUTIONS IN THE FIELD OF ROBOTIC VISION TO CREATE A UNIVERSAL CYBERNETIC PLATFORM

2019 ◽  
pp. 32-38 ◽  
Author(s):  
O. V. Danishevskiy ◽  
A. V. Parfenov ◽  
V. V. Stankevich
Keyword(s):  
Optical Sensors ◽  
Vision System ◽  
Computing System ◽  
Stereoscopic Vision ◽  
Robotic Vision ◽  
System A ◽  
Images Processing ◽  
Video Sensors ◽  
The Cost ◽  
Frequency Ultrasound

The article describes modern solutions used to create vision systems for autonomous robotic complexes, a device for controlling  video sensors, methods for obtaining stereo images, processing and further using them in the navigation system. A functional  diagram of the organization of the robotic vision system, which is part of the NIIVK M. A. Kartsev Computing System Research  and Development Institute, platform, a brief description of its functions and principles of action. The proposed solution involves  placing a pair of optical sensors in the head of the robot, simulating the binocular stereoscopic vision inherent in higher animals.  A feature of the development is the rejection of the use of laser, infrared, radio frequency, ultrasound and other active emitters,  which reduces energy costs and the cost of products without reducing their reliability.

scholarly journals A Probabilistic Approach to the Simulation of Data Processing Centers

2020 ◽  
Vol 226 ◽  
pp. 03012
Author(s):  
Vladimir Korenkov ◽  
Andrey Nechaevskiy ◽  
Gennady Ososkov ◽  
Daria Priakhina ◽  
Vladimir Trofimov
Keyword(s):  
Lower Bound ◽  
Data Processing ◽  
Data Center ◽  
Data Transfer ◽  
Storage System ◽  
Probabilistic Approach ◽  
Computing System ◽  
Full Data ◽  
System A ◽  
The Creation

The simulation of a data center for the storage and processing of data from the NICA detectors is an important step towards the creation of the NICA computing system. A model developed in the frame of the probabilistic approach to the solution enables decisions concerning a lower bound of the necessary resources for full data transfer of the detector records to the storage system.

scholarly journals A New Method for Reconstructing Data on a Single Failure Node in the Distributed Storage System Based on the MSR Code

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Miao Ye ◽  
Ruoyu Wei ◽  
Wei Guo ◽  
Qiuxiang Jiang ◽  
Hongbing Qiu ◽  
...  
Keyword(s):  
Genetic Algorithm ◽  
Distributed Storage ◽  
Storage System ◽  
Hybrid Genetic Algorithm ◽  
Repair Process ◽  
Single Node ◽  
Distributed Storage System ◽  
Erasure Code ◽  
Repair Strategy ◽  
The Cost

As a storage method for a distributed storage system, an erasure code can save storage space and repair the data of failed nodes. However, most studies that discuss the repair of fault nodes in the erasure code mode only focus on the condition that the bandwidth of heterogeneous links restricts the repair rate but ignore the condition that the storage node is heterogeneous, the cost of repair traffic in the repair process, and the influence of the failure of secondary nodes on the repair process. An optimal repair strategy based on the minimum storage regenerative (MSR) code and a hybrid genetic algorithm is proposed for single-node fault scenarios to solve the above problems. In this work, the single-node data repair problem is modeled as an optimization problem of an optimal Steiner tree with constraints considering heterogeneous link bandwidth and heterogeneous node processing capacity and takes repair traffic and repair delay as optimization objectives. After that, a hybrid genetic algorithm is designed to solve the problem. The experimental results show that under the same scales used in the MSR code cases, our approach has good robustness and its repair delay decreases by 10% and 55% compared with the conventional tree repair topology and star repair topology, respectively; the repair flow increases by 10% compared with the star topology, and the flow rate of the conventional tree repair topology decreases by 40%.

scholarly journals Efficient and selective catalytic hydroxylation of unsaturated plant oils: a novel method for producing anti-pathogens

BMC Chemistry ◽  
2021 ◽  
Vol 15 (1) ◽  
Author(s):  
Ahmed M. Senan ◽  
Binru Yin ◽  
Yaoyao Zhang ◽  
Mustapha M. Nasiru ◽  
Yong‐Mei Lyu ◽  
...  
Keyword(s):  
Antimicrobial Agents ◽  
In Vitro Study ◽  
Functional Method ◽  
Plant Oils ◽  
Main Process ◽  
Catalytic Method ◽  
Novel Method ◽  
Pharmaceutical Industries ◽  
Increasing Demand

AbstractWith the increasing demand for antimicrobial agents and the spread of antibiotic resistance in pathogens, the exploitation of plant oils to partly replace antibiotic emerges as an important source of fine chemicals, functional food utility and pharmaceutical industries. This work introduces a novel catalytic method of plant oils hydroxylation by Fe(III) citrate monohydrate (Fe3+-cit.)/Na2S2O8 catalyst. Methyl (9Z,12Z)-octadecadienoate (ML) was selected as an example of vegetable oils hydroxylation to its hydroxy-conjugated derivatives (CHML) in the presence of a new complex of Fe(II)-species. Methyl 9,12-di-hydroxyoctadecanoate 1, methyl-9-hydroxyoctadecanoate 2 and methyl (10E,12E)-octadecanoate 3 mixtures is produced under optimized condition with oxygen balloon. The specific hydroxylation activity was lower in the case of using Na2S2O8 alone as a catalyst. A chemical reaction has shown the main process converted of plantoils hydroxylation and (+ 16 Da) of OH- attached at the methyl linoleate (ML-OH). HPLC and MALDI-ToF-mass spectrometry were employed for determining the obtained products. It was found that adding oxidizing agents (Na2S2O8) to Fe3+ in the MeCN mixture with H2O would generate the new complex of Fe(II)-species, which improves the C-H activation. Hence, the present study demonstrated a new functional method for better usage of vegetable oils.Producing conjugated hydroxy-fatty acids/esters with better antipathogenic properties. CHML used in food industry, It has a potential pathway to food safety and packaging process with good advantages, fundamental to microbial resistance. Lastly, our findings showed that biological monitoring of CHML-minimum inhibitory concentration (MIC) inhibited growth of various gram-positive and gram-negative bacteria in vitro study. The produced CHML profiles were comparable to the corresponding to previousstudies and showed improved the inhibition efficiency over the respective kanamycin derivatives.

scholarly journals Micro grid for All India Institute of Medical Sciences, Madurai

Clean Energy ◽  
2021 ◽  
Vol 5 (2) ◽  
pp. 254-272
Author(s):  
C Palanichamy ◽  
P Naveen
Keyword(s):  
Tamil Nadu ◽  
Storage System ◽  
Healthcare Facility ◽  
Photovoltaic System ◽  
India Institute ◽  
Diesel Generator ◽  
Medical Sciences ◽  
System A ◽  
Utility Grid ◽  
The Government

Abstract In 2018, the Government of India approved the establishment of the New All India Institute of Medical Sciences (AIIMS) in Thoppur, Madurai, Tamil Nadu, India. As the most important amenity for continuing primary care and rescue response is a healthcare facility, a secure electricity supply becomes an imperative necessity. Hence, as the energy supplier for the new AIIMS, Madurai, this paper proposes a microgrid combined with the utility grid. The microgrid consists of a 4-MW photovoltaic system, a 1.8-MW wind-turbine energy-conversion system, a backup diesel generator capable of meeting the forecasted maximum demand and a 1-MW battery energy-storage system. The AIIMS Microgrid will have a service providing a capacity of 20 MVA following integration with the utility grid. The proposed microgrid would be the first attempt at healthcare facilities in India since its first day of work to ensure the availability of electricity. It would have a 9.8% return on investment, a 13.6% internal rate of return and a payback period of 6.75 years once it is operational, as well as an attractive levelized cost of energy (LCOE) of USD 0.07547/kWh. It would provide an environmentally friendly atmosphere by avoiding an annual emission of 6 261 132 kg of carbon dioxide, 27 362 kg of sulphur dioxide and 12 838 kg of nitrogen oxides as compared to power supplied entirely from the utility grid.

scholarly journals Cost Analysis of a Novel Method for Ecological Compensation—A Study of the Translocation of Dead Wood

2021 ◽  
Vol 13 (11) ◽  
pp. 6075
Author(s):  
Ola Lindroos ◽  
Malin Söderlind ◽  
Joel Jensen ◽  
Joakim Hjältén
Keyword(s):  
Cost Efficiency ◽  
Dead Wood ◽  
Road Transportation ◽  
Ecological Value ◽  
Ecological Compensation ◽  
Work Time ◽  
Delivery Times ◽  
Novel Method ◽  
Cost Efficient ◽  
The Cost

Translocation of dead wood is a novel method for ecological compensation and restoration that could, potentially, provide a new important tool for biodiversity conservation. With this method, substrates that normally have long delivery times are instantly created in a compensation area, and ideally many of the associated dead wood dwelling organisms are translocated together with the substrates. However, to a large extent, there is a lack of knowledge about the cost efficiency of different methods of ecological compensation. Therefore, the costs for different parts of a translocation process and its dependency on some influencing factors were studied. The observed cost was 465 SEK per translocated log for the actual compensation measure, with an additional 349 SEK/log for work to enable evaluation of the translocation’s ecological results. Based on time studies, models were developed to predict required work time and costs for different transportation distances and load sizes. Those models indicated that short extraction and insertion distances for logs should be prioritized over road transportation distances to minimize costs. They also highlighted a trade-off between costs and time until a given ecological value is reached in the compensation area. The methodology used can contribute to more cost-efficient operations and, by doing so, increase the use of ecological compensation and the benefits from a given input.

scholarly journals Supporting SLA via Adaptive Mapping and Heterogeneous Storage Devices in Ceph

Electronics ◽  
2021 ◽  
Vol 10 (7) ◽  
pp. 847
Author(s):  
Sopanhapich Chum ◽  
Heekwon Park ◽  
Jongmoo Choi
Keyword(s):  
Positive Impact ◽  
Distributed Storage ◽  
Storage System ◽  
Hard Disk Drives ◽  
Service Level Agreement ◽  
Service Level ◽  
Cost Ratio ◽  
Solid State Drives ◽  
Storage Devices ◽  
Management Scheme

This paper proposes a new resource management scheme that supports SLA (Service-Level Agreement) in a bigdata distributed storage system. Basically, it makes use of two mapping modes, isolated mode and shared mode, in an adaptive manner. In specific, to ensure different QoS (Quality of Service) requirements among clients, it isolates storage devices so that urgent clients are not interfered by normal clients. When there is no urgent client, it switches to the shared mode so that normal clients can access all storage devices, thus achieving full performance. To provide this adaptability effectively, it devises two techniques, called logical cluster and normal inclusion. In addition, this paper explores how to exploit heterogeneous storage devices, HDDs (Hard Disk Drives) and SSDs (Solid State Drives), to support SLA. It examines two use cases and observes that separating data and metadata into different devices gives a positive impact on the performance per cost ratio. Real implementation-based evaluation results show that this proposal can satisfy the requirements of diverse clients and can provide better performance compared with a fixed mapping-based scheme.

Sign in / Sign up

Export Citation Format

Share Document