scholarly journals Performance evaluation and resource optimization of cloud based parallel Hadoop clusters with an intelligent scheduler.

2018 ◽  
Vol 7 (4.20) ◽  
pp. 4220
Author(s):  
Manishankar S. ◽  
S. Sathayanarayana

Data generated from real time information systems are always incremental in nature. Processing of such a huge incremental data in large scale requires a parallel processing system like Hadoop based cluster. Major challenge that arises in all cluster-based system is how efficiently the resources of the system can be used. The research carried out proposes a model architecture for Hadoop cluster with additional components integrated such as super node who manages the clusters computations and a mediation manager who does the performance monitoring and evaluation. Super node in the system is equipped with intelligent or adaptive scheduler that does the scheduling of the job with optimal resources. The scheduler is termed intelligent as it automatically decides which resource to be taken for which computation, with the help of a cross mapping of resource and job with a genetic algorithm which finds the best matching resource. The mediation node deploys ganglia a standard monitoring tool for Hadoop cluster to collect and record the performance parameters of the Hadoop cluster. The system over all does the scheduling of different jobs with optimal usage of resources thus achieving better efficiency compared to the native capacity scheduler in Hadoop. The system is deployed on top of OpenNebula Cloud environment for scalability.     

2010 ◽  
Vol 143-144 ◽  
pp. 18-22
Author(s):  
Zhi Gang Ji

The system integrated RFID technique, wireless and mobile phone which apply in the intelligent guidance and book management of large scale of library, and to collate the real time information by different functional mobile agents to provide customized information and service, according to the favorite of each reader. Meanwhile, to display forgoing information by multimedia video of book introduction for readers to replace traditional looking for some particular book provide the customized knowledge search style and effective book introduction. In addition, our article set forth System Environment Aware and Recommending Mechanism which bring up considerations for customized information, besides to analyzing personal favorite, should think about the environmental factors in the library, such as, moving path of readers, service and so on such synthetic factors to have a complete analysis and consideration, so as to know what the most appropriate guided service and information is for readers and to make it as a customized intellectual guided service.


2019 ◽  
Author(s):  
Vinícius Röpke ◽  
Matheus Guarienti ◽  
Tiago Antonio Rizzetti

This article describes the optimization of the Zabbix network monitoring tool at the Hospital Universitário de Santa Maria, where existing servers had been presenting problems generated by the misconfiguration of the tool as a whole. The high amount of network assets monitored by the tool, and the abundance of data collected, slowed the machine, making it difficult to use when necessary. New network assets and locations to be added were analyzed to ensure item history. Thus it is possible to generate reports for statistics, and reduce the computational effort of the tool to bring real-time information of network problems. To solve the problems presented, and obtain concrete data of the environment, it was aimed to optimize the tool itself, reviewing items from each host on the machine, linking templates, adjusting the use of the housekeepinp function (acting to remove unused records in database level), among other processes to improve server performance. This facilitates the use of the tool.


Author(s):  
Melissa Ospina-Quiroga ◽  
Eduardo Mojica-Nava

The growing consumption of electricity as well as the progressive development of new technologies implies that the power system is increasingly automated with the purpose of having a more efficient and economical operation. This development drives the system to a Smart Grid, a large-scale cyber-physical network covering different energy generation technologies, storage and communications enabling real-time information exchange and control. In this work, we present an optimal distributed control based on the consensus+innovation technique, where each agent of the network obtains information from its neighbours. Simulations on a microgrid system based on an IEEE 14-Bus reference system demonstrate the effectiveness of the approach. Convergence is observed in the microgrid system under different scenarios in the physical and communications network.


Author(s):  
JUHANI HIRVONEN ◽  
REIJO LILJA ◽  
KARI IKONEN ◽  
JARMO NIHTINEN

This paper describes an image processing system that calculates real time information from the combustion process itself. The applications of the system on burner and supplemental fuel adjustment, and ignition trend monitoring are also discussed. Finally, new combustion control based on the image information and quantitative model of the furnace is discussed.


2013 ◽  
Vol 278-280 ◽  
pp. 1114-1118
Author(s):  
Pei Zhi Cui ◽  
Yan Sun ◽  
Guang Wei Wang ◽  
Ling Yun Sun

Data acquisition for simulation training system can provide reliable operation data sources for system displaying, monitoring, controlling and evaluation. It takes an important role in simulation training system. This paper studies and designs one kind based on database trigger mechanism document data acquisition method. This method can guarantee real-time information collection and write data to the remote target database table in real time. It’s suitable for the data processing system of information acquisition application.


Sign in / Sign up

Export Citation Format

Share Document