Hilbert Index-based Outlier Detection Algorithm in Metric Space

2016 ◽  
Vol 8 (4) ◽  
pp. 34-54
Author(s):  
Honglong Xu ◽  
Haiwu Rong ◽  
Rui Mao ◽  
Guoliang Chen ◽  
Zhiguang Shan
Keyword(s):  
Big Data ◽  
Data Management ◽  
Outlier Detection ◽  
Metric Space ◽  
Detection Algorithm ◽  
Analysis Framework ◽  
Distance Calculation ◽  
The World ◽  
Speed Up ◽  
Detection Speed

Big data is profoundly changing the lifestyles of people around the world in an unprecedented way. Driven by the requirements of applications across many industries, research on big data has been growing. Methods to manage and analyze big data to extract valuable information are the key of big data research. Starting from the variety challenge of big data, this dissertation proposes a universal big data management and analysis framework based on metric space. In this framework, the Hilbert Index-based Outlier Detection (HIOD) algorithm is proposed. HIOD can handle all datatypes that can be abstracted to metric space and achieve higher detection speed. Experimental results indicate that HIOD can effectively overcome the variety challenge of big data and achieves a 2.02 speed up over iORCA on average and, in certain cases, up to 5.57. The distance calculation times are reduced by 47.57% on average and up to 89.10%.

scholarly journals Index Based Hidden Outlier Detection in Metric Space

2016 ◽  
Vol 2016 ◽  
pp. 1-14
Author(s):  
Honglong Xu ◽  
Rui Mao ◽  
Hao Liao ◽  
He Zhang ◽  
Minhua Lu ◽  
...  
Keyword(s):  
Big Data ◽  
Outlier Detection ◽  
Metric Space ◽  
Experimental Results ◽  
Running Time ◽  
New Perspective ◽  
Candidate Set ◽  
Detection Speed

Useless and noise information occupies large amount of big data, which increases our difficulty to extract worthy information. Therefore outlier detection attracts much attention recently, but if two points are far from other points but are relatively close to each other, they are less likely to be detected as outliers because of their adjacency to each other. In this situation, outliers are hidden by each other. In this paper, we propose a new perspective of hidden outlier. Experimental results show that it is more accurate than existing distance-based definitions of outliers. Accordingly, we exploit a candidate set based hidden outlier detection (HOD) algorithm. HOD algorithm achieves higher accuracy with comparable running time. Further, we develop an index based HOD (iHOD) algorithm to get higher detection speed.

Opportunities and Implementation of Big Data Management in Academic Libraries

2021 ◽  
pp. 35-47
Author(s):  
Mahesh G. T. ◽  
Nandeesha B.
Keyword(s):  
Big Data ◽  
Data Management ◽  
Academic Libraries ◽  
Strong Association ◽  
Karnataka State ◽  
The World ◽  
The Government ◽  
Associated Data

Data has changed the world in an unbelievable way and made an impact on our lifestyles at an exceptional rate. Big data is now the latest science of exploring and forecasting human-machine behavior dealing with a massive amount of associated data. The study is intended to understand the intensity and the competencies of librarians in implementing big data initiative project in academic libraries by the Government of Karnataka State. The study also tries to understand the application of big data in these libraries; 68 (87.17%) librarians completed the survey out of 78 respondents. The results of the study showed a strong association, that is, 72 (92.30%) respondents had the essential competencies and 58 (75.64%) librarians ability, intensity, readiness in implementing big data in academic libraries.

Application of Big Data in Healthcare

2016 ◽  
pp. 149-168
Author(s):  
Md Rakibul Hoque ◽  
Yukun Bao
Keyword(s):  
Health Care ◽  
Big Data ◽  
Data Management ◽  
Healthcare Spending ◽  
Well Being ◽  
Healthcare Industry ◽  
The World ◽  
Hospital Systems ◽  
New Challenges

This chapter investigates the application, opportunities, challenges and techniques of Big Data in healthcare. The healthcare industry is one of the most important, largest, and fastest growing industries in the world. It has historically generated large amounts of data, “Big Data”, related to patient healthcare and well-being. Big Data can transform the healthcare industry by improving operational efficiencies, improve the quality of clinical trials, and optimize healthcare spending from patients to hospital systems. However, the health care sector lags far behind compared to other industries in leveraging their data assets to improve efficiencies and make more informed decisions. Big Data entails many new challenges regarding security, privacy, legal concerns, authenticity, complexity, accuracy, and consistency. While these challenges are complex, they are also addressable. The predominant ‘Big Data' Management technologies such as MapReduce, Hadoop, STORM, and others with similar combinations or extensions should be used for effective data management in healthcare industry.

Application of Big Data in Healthcare

Big Data ◽  
2016 ◽  
pp. 1189-1208 ◽  
Author(s):  
Md Rakibul Hoque ◽  
Yukun Bao
Keyword(s):  
Health Care ◽  
Big Data ◽  
Data Management ◽  
Healthcare Spending ◽  
Well Being ◽  
Healthcare Industry ◽  
The World ◽  
Hospital Systems ◽  
New Challenges

This chapter investigates the application, opportunities, challenges and techniques of Big Data in healthcare. The healthcare industry is one of the most important, largest, and fastest growing industries in the world. It has historically generated large amounts of data, “Big Data”, related to patient healthcare and well-being. Big Data can transform the healthcare industry by improving operational efficiencies, improve the quality of clinical trials, and optimize healthcare spending from patients to hospital systems. However, the health care sector lags far behind compared to other industries in leveraging their data assets to improve efficiencies and make more informed decisions. Big Data entails many new challenges regarding security, privacy, legal concerns, authenticity, complexity, accuracy, and consistency. While these challenges are complex, they are also addressable. The predominant ‘Big Data' Management technologies such as MapReduce, Hadoop, STORM, and others with similar combinations or extensions should be used for effective data management in healthcare industry.

The computational accelerator physics conference series: From its origins to the present day

2019 ◽  
Vol 34 (36) ◽  
pp. 1942025
Author(s):  
Robert D. Ryne
Keyword(s):  
Big Data ◽  
Data Management ◽  
Large Scale ◽  
Peak Performance ◽  
Accelerator Physics ◽  
Conference Series ◽  
The World ◽  
The Future

The first conference in what would become the International Computational Accelerator Physics (ICAP) series was held in 1988. At that time the most powerful computer in the world was a Cray YMP with 8 processors and a peak performance of 2 gigaflops. Today the fastest computer in the world has more than 2 million cores and a theoretical peak performance of nearly 200 petaflops. Compared to 1988, performance has increased by a factor of 100 million, accompanied by huge advances in memory, networking, big data management and analytics. By the time of the next ICAP in 2021 we will be at the dawn of the Exascale era. In this talk I will describe the advances in Computational Accelerator Physics that brought us to this point and describe trends in regard to large-scale accelerator simulation in the future.

Big data outlier detection model based on improved density peak algorithm

2020 ◽  
pp. 1-10
Author(s):  
Mengliang Shao ◽  
Deyu Qi ◽  
Huili Xue
Keyword(s):  
Big Data ◽  
Fault Detection ◽  
Outlier Detection ◽  
Clustering Algorithm ◽  
Detection Algorithm ◽  
Load Curve ◽  
Density Peak ◽  
Original Algorithm ◽  
Detection Model ◽  
Density Peak Clustering

Outlier detection is an important branch of data mining. This paper proposes an advanced fast density peak outlier detection algorithm based on the characteristics of big data. The algorithm is an outlier detection method based on the improved density peak clustering algorithm. This paper improves the original algorithm. From the perspective of outlier detection, although it is a clustering idea, it avoids the clustering process, reduces the time complexity of the cluster-based outlier detection algorithm, and absorbs. The outlier detection based on neighbors is not sensitive to data dimensions and other advantages. In the power industry, outlier detection can be used in areas such as grid fault detection, equipment fault detection, and power abnormality detection. The simulation experiment of outlier detection based on the daily load curve of single and multiple transformers in a certain province shows that the improved algorithm can effectively detect outliers in the data.

The beginning of phygital world

2020 ◽  
Vol 10 (2) ◽  
pp. 1-4
Author(s):  
Evgeny Soloviov ◽  
Alexander Danilov
Keyword(s):  
Big Data ◽  
Learning Styles ◽  
21St Century ◽  
Digital Technology ◽  
Aviation Education ◽  
Digital World ◽  
The World ◽  
Unique Experience ◽  
Smart Data

The Phygital word itself is the combination pf physical and digital technology application.This paper will highlight the detail of phygital world and its importance, also we will discuss why its matter in the world of technology along with advantages and disadvantages.It is the concept and technology is the bridge between physical and digital world which bring unique experience to the users by providing purpose of phygital world. It is the technology used in 21st century to bring smart data as opposed to big data and mix into the broader address of array of learning styles. It can bring new experience to every sector almost like, retail, medical, aviation, education etc. to maintain some reality in today’s world which is developing technology day to day. It is a general reboot which can keep economy moving and guarantee the wellbeing of future in terms of both online and offline.

Sign in / Sign up

Export Citation Format

Share Document