Incorporating Text OLAP in Business Intelligence

2011 ◽  
pp. 77-101 ◽  
Author(s):  
Byung-Kwon Park ◽  
Il-Yeol Song
Keyword(s):  
Information Retrieval ◽  
Text Mining ◽  
Business Intelligence ◽  
Multidimensional Analysis ◽  
Web Pages ◽  
Data Types ◽  
Text Documents ◽  
Text Data ◽  
Platform Architecture ◽  
Unstructured Text

As the amount of data grows very fast inside and outside of an enterprise, it is getting important to seamlessly analyze both data types for total business intelligence. The data can be classified into two categories: structured and unstructured. For getting total business intelligence, it is important to seamlessly analyze both of them. Especially, as most of business data are unstructured text documents, including the Web pages in Internet, we need a Text OLAP solution to perform multidimensional analysis of text documents in the same way as structured relational data. We first survey the representative works selected for demonstrating how the technologies of text mining and information retrieval can be applied for multidimensional analysis of text documents, because they are major technologies handling text data. And then, we survey the representative works selected for demonstrating how we can associate and consolidate both unstructured text documents and structured relation data for obtaining total business intelligence. Finally, we present a future business intelligence platform architecture as well as related research topics. We expect the proposed total heterogeneous business intelligence architecture, which integrates information retrieval, text mining, and information extraction technologies all together, including relational OLAP technologies, would make a better platform toward total business intelligence.

Document Clustering

2017 ◽  
pp. 264-281
Author(s):  
Harsha Patil ◽  
R. S. Thakur
Keyword(s):  
Text Mining ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Web Pages ◽  
Digital Form ◽  
Search Query ◽  
Text Documents ◽  
Keen Interest ◽  
Use Of Internet

As we know use of Internet flourishes with its full velocity and in all dimensions. Enormous availability of Text documents in digital form (email, web pages, blog post, news articles, ebooks and other text files) on internet challenges technology to appropriate retrieval of document as a response for any search query. As a result there has been an eruption of interest in people to mine these vast resources and classify them properly. It invigorates researchers and developers to work on numerous approaches of document clustering. Researchers got keen interest in this problem of text mining. The aim of this chapter is to summarised different document clustering algorithms used by researchers.

Text-Driven Reasoning and Multi-Structured Data Analytics for Business Intelligence

2016 ◽  
pp. 1-32 ◽  
Author(s):  
Lipika Dey ◽  
Ishan Verma
Keyword(s):  
Business Intelligence ◽  
Data Analytics ◽  
Statistical Significance ◽  
Heterogeneous Data ◽  
Structured Data ◽  
Text Data ◽  
Business Operations ◽  
Unstructured Text ◽  
Business Data ◽  
Heterogeneous Resources

Business Intelligence (BI) refers to an organization's capability to gather and analyze data about business operations and transactions in order to evaluate its performance. The abundance of information both within the enterprise and outside of it has necessitated a change in traditional Business Intelligence practices. There is a need to exploit heterogeneous resources. Text data like news, analyst reports, etc. helps in better interpretation of business data. In this chapter, the authors present a futuristic BI framework that facilitates acquisition, indexing, and analysis of heterogeneous data for extracting business intelligence. It enables integration of unstructured text data and structured business data seamlessly to generate insights. The authors propose methods that can help in extraction of events or significant happenings from both unstructured and structured data, correlate the events, and thereafter reason to generate insights. The insights extracted could be validated as cause-effect pairs based on the statistical significance of co-occurrence of events.

Dual Scaling in Data Mining from Text Databases

2006 ◽  
Vol 10 (4) ◽  
pp. 451-457 ◽  
Author(s):  
Junzo Watada ◽  
◽  
Keisuke Aoki ◽  
Masahiro Kawano ◽  
Muhammad Suzuri Hitam ◽  
...  
Keyword(s):  
Multivariate Analysis ◽  
Text Mining ◽  
Kansei Engineering ◽  
Semantic Meaning ◽  
Dual Scaling ◽  
Text Documents ◽  
Text Data ◽  
Text Document ◽  
Text Information ◽  
Quantification Model

The availability of multimedia text document information has disseminated text mining among researchers. Text documents, integrate numerical and linguistic data, making text mining interesting and challenging. We propose text mining based on a fuzzy quantification model and fuzzy thesaurus. In text mining, we focus on: 1) Sentences included in Japanese text that are broken down into words. 2) Fuzzy thesaurus for finding words matching keywords in text. 3) Fuzzy multivariate analysis to analyze semantic meaning in predefined case studies. We use a fuzzy thesaurus to translate words using Chinese and Japanese characters into keywords. This speeds up processing without requiring a dictionary to separate words. Fuzzy multivariate analysis is used to analyze such processed data and to extract latent mutual related structures in text data, i.e., to extract otherwise obscured knowledge. We apply dual scaling to mining library and Web page text information, and propose integrating the result in Kansei engineering for possible application in sales, marketing, and production.

The Method to Analyze Freely Described Data from Questionnaires

2009 ◽  
Vol 13 (3) ◽  
pp. 268-274 ◽  
Author(s):  
Masaomi Kimura ◽  
Keyword(s):  
Text Mining ◽  
Call Center ◽  
Clustering Algorithms ◽  
Web Pages ◽  
Research Papers ◽  
Text Data ◽  
Textual Data ◽  
The Way ◽  
Newspaper Articles

Text mining has been growing; mainly due to the need to extract useful information from vast amounts of textual data. Our target here is text data, a collection of freely described data from questionnaires. Unlike research papers, newspaper articles, call-center logs and web pages, which are usually the targets of text mining analysis, the freely described data contained in the questionnaire responses have specific characteristics, including a small number of short sentences forming individual pieces of data, while the wide variety of content precludes the applications of clustering algorithms used to classify the same. In this paper, we suggest the way to extract the opinions which are delivered by multiple respondents, based on the modification relationships included in each sentence in the freely described data. Certain applications of our method are also presented after the introduction of our approach.

scholarly journals Robust Face Recognition for Data Mining

2008 ◽  
pp. 3621-3629
Author(s):  
Brian C. Lovell ◽  
Shaokang Chen
Keyword(s):  
Data Mining ◽  
Interactive Television ◽  
Multimedia Data ◽  
The Internet ◽  
Data Types ◽  
Text Documents ◽  
Text Data ◽  
Internet Backbone ◽  
Large Databases ◽  
Robust Face Recognition

While the technology for mining text documents in large databases could be said to be relatively mature, the same cannot be said for mining other important data types such as speech, music, images and video. Yet these forms of multimedia data are becoming increasingly prevalent on the Internet and intranets as bandwidth rapidly increases due to continuing advances in computing hardware and consumer demand. An emerging major problem is the lack of accurate and efficient tools to query these multimedia data directly, so we are usually forced to rely on available metadata, such as manual labeling. Currently the most effective way to label data to allow for searching of multimedia archives is for humans to physically review the material. This is already uneconomic or, in an increasing number of application areas, quite impossible because these data are being collected much faster than any group of humans could meaningfully label them — and the pace is accelerating, forming a veritable explosion of non-text data. Some driver applications are emerging from heightened security demands in the 21st century, post-production of digital interactive television, and the recent deployment of a planetary sensor network overlaid on the Internet backbone.

Document Clustering

2018 ◽  
pp. 47-64 ◽  
Author(s):  
Harsha Patil ◽  
R. S. Thakur
Keyword(s):  
Text Mining ◽  
Clustering Algorithms ◽  
Document Clustering ◽  
Web Pages ◽  
Digital Form ◽  
Search Query ◽  
Text Documents ◽  
Keen Interest ◽  
Use Of Internet

As we know use of Internet flourishes with its full velocity and in all dimensions. Enormous availability of Text documents in digital form (email, web pages, blog post, news articles, ebooks and other text files) on internet challenges technology to appropriate retrieval of document as a response for any search query. As a result there has been an eruption of interest in people to mine these vast resources and classify them properly. It invigorates researchers and developers to work on numerous approaches of document clustering. Researchers got keen interest in this problem of text mining. The aim of this chapter is to summarised different document clustering algorithms used by researchers.

Intelligent Strategy and Security in Education

2019 ◽  
pp. 87-110
Author(s):  
Samson Oluwaseun Fadiya
Keyword(s):  
Social Networking ◽  
Query Language ◽  
Unstructured Data ◽  
Text Analytics ◽  
Text Documents ◽  
Text Data ◽  
Web Based ◽  
Data Usage ◽  
Unstructured Text ◽  
Common Problems

Text analytics applies to most businesses, particularly education segments; for instance, if association or university is suspicious about data secrets being spilt to contenders by the workers, text analytics investigation can help dissect many employees' email messages. The massive volume of both organized and unstructured data principally started from the web-based social networking (media) and Web 2.0. The investigation (analysis) of messages online, tweets, and different types of unstructured text data constitute what we call text analytics, which has been developed during the most recent few years in a way that does not shift, through the upheaval of various algorithms and applications being utilized for the processing of data alongside the protection and IT security. This chapter plans to find common problems faced when using the different medium of data usage in education, one can analyze their information through the perform of sentiment analysis using text analytics by extracting useful information from text documents using IBM's annotation query language (AQL).

scholarly journals An Efficient Topic Modeling Approach for Text Mining and Information Retrieval through K-means Clustering

2020 ◽  
Vol 39 (1) ◽  
pp. 213-222
Author(s):  
Junaid Rashid ◽  
Syed Muhammad Adnan Shah ◽  
Aun Irtaza
Keyword(s):  
Information Retrieval ◽  
Text Mining ◽  
Topic Modeling ◽  
Clustering Algorithm ◽  
Latent Dirichlet Allocation ◽  
Semantic Analysis ◽  
State Of The Art ◽  
Text Documents ◽  
New Perspective ◽  
Better Than

Topic modeling is an effective text mining and information retrieval approach to organizing knowledge with various contents under a specific topic. Text documents in form of news articles are increasing very fast on the web. Analysis of these documents is very important in the fields of text mining and information retrieval. Meaningful information extraction from these documents is a challenging task. One approach for discovering the theme from text documents is topic modeling but this approach still needs a new perspective to improve its performance. In topic modeling, documents have topics and topics are the collection of words. In this paper, we propose a new k-means topic modeling (KTM) approach by using the k-means clustering algorithm. KTM discovers better semantic topics from a collection of documents. Experiments on two real-world Reuters 21578 and BBC News datasets show that KTM performance is better than state-of-the-art topic models like LDA (Latent Dirichlet Allocation) and LSA (Latent Semantic Analysis). The KTM is also applicable for classification and clustering tasks in text mining and achieves higher performance with a comparison of its competitors LDA and LSA.

scholarly journals Layer-Assisted Neural Topic Modeling over Document Networks

2021 ◽  
Author(s):  
Yiming Wang ◽  
Ximing Li ◽  
Jihong Ouyang
Keyword(s):  
Text Classification ◽  
Topic Modeling ◽  
Link Prediction ◽  
Topic Model ◽  
Web Pages ◽  
Text Documents ◽  
Text Data ◽  
Generative Process ◽  
Network Links ◽  
Scientific Papers

Neural topic modeling provides a flexible, efficient, and powerful way to extract topic representations from text documents. Unfortunately, most existing models cannot handle the text data with network links, such as web pages with hyperlinks and scientific papers with citations. To resolve this kind of data, we develop a novel neural topic model , namely Layer-Assisted Neural Topic Model (LANTM), which can be interpreted from the perspective of variational auto-encoders. Our major motivation is to enhance the topic representation encoding by not only using text contents, but also the assisted network links. Specifically, LANTM encodes the texts and network links to the topic representations by an augmented network with graph convolutional modules, and decodes them by maximizing the likelihood of the generative process. The neural variational inference is adopted for efficient inference. Experimental results validate that LANTM significantly outperforms the existing models on topic quality, text classification and link prediction..

Sign in / Sign up

Export Citation Format

Share Document