Using Semantically-Extended LDA Topic Model for Semantic Tagging

2016 ◽  
Vol 10 (04) ◽  
pp. 503-525
Author(s):  
Mehdi Allahyari ◽  
Krys Kochut
Keyword(s):  
Topic Model ◽  
Web Pages ◽  
Challenging Problem ◽  
Huge Amount ◽  
Web Documents ◽  
Aggregate Information ◽  
Probabilistic Topic Model ◽  
Statistical Topic Models ◽  
Great Step ◽  
The Web

The volume of documents and online resources has been increasing significantly on the Web for many years. Effectively, organizing this huge amount of information has become a challenging problem. Tagging is a mechanism to aggregate information and a great step towards the Semantic Web vision. Tagging aims to organize, summarize, share and search the Web resources in an effective way. One important problem facing tagging systems is to automatically determine the most appropriate tags for Web documents. In this paper, we propose a probabilistic topic model that incorporates DBpedia knowledge into the topic model for tagging Web pages and online documents with topics discovered in them. Our method is based on integration of the DBpedia hierarchical category network with statistical topic models, where DBpedia categories are considered as topics. We have conducted extensive experiments on two different datasets to demonstrate the effectiveness of our method.

A FRAME WORK FOR WEB INFORMATION EXTRACTION AND ANALYSIS

2013 ◽  
Vol 7 (2) ◽  
pp. 574-579 ◽  
Author(s):  
Dr Sunitha Abburu ◽  
G. Suresh Babu
Keyword(s):  
Information Extraction ◽  
Data Extraction ◽  
Research Work ◽  
Web Pages ◽  
Web Documents ◽  
E Learning ◽  
Structured Information ◽  
Frame Work ◽  
Effective Decision ◽  
The Web

Day by day the volume of information availability in the web is growing significantly. There are several data structures for information available in the web such as structured, semi-structured and unstructured. Majority of information in the web is presented in web pages. The information presented in web pages is semi-structured.  But the information required for a context are scattered in different web documents. It is difficult to analyze the large volumes of semi-structured information presented in the web pages and to make decisions based on the analysis. The current research work proposed a frame work for a system that extracts information from various sources and prepares reports based on the knowledge built from the analysis. This simplifies  data extraction, data consolidation, data analysis and decision making based on the information presented in the web pages.The proposed frame work integrates web crawling, information extraction and data mining technologies for better information analysis that helps in effective decision making.   It enables people and organizations to extract information from various sourses of web and to make an effective analysis on the extracted data for effective decision making.  The proposed frame work is applicable for any application domain. Manufacturing,sales,tourisum,e-learning are various application to menction few.The frame work is implemetnted and tested for the effectiveness of the proposed system and the results are promising.

scholarly journals Extraction of Meaningful Information from the Web: a Brief Survey

2018 ◽  
Vol 7 (4.19) ◽  
pp. 1041
Author(s):  
Santosh V. Chobe ◽  
Dr. Shirish S. Sane
Keyword(s):  
Information Extraction ◽  
Relevant Information ◽  
Unstructured Data ◽  
Web Pages ◽  
Extraction Techniques ◽  
Web Documents ◽  
Meaningful Information ◽  
The Web

There is an explosive growth of information on Internet that makes extraction of relevant data from various sources, a difficult task for its users. Therefore, to transform the Web pages into databases, Information Extraction (IE) systems are needed. Relevant information in Web documents can be extracted using information extraction and presented in a structured format.By applying information extraction techniques, information can be extracted from structured, semi-structured, and unstructured data. This paper presents some of the major information extraction tools. Here, advantages and limitations of the tools are discussed from a user’s perspective.  

scholarly journals Improving Performance of DOM in Semi-structured Data Extraction using WEIDJ Model

2018 ◽  
Vol 9 (3) ◽  
pp. 752 ◽  
Author(s):  
Ily Amalina Ahmad Sabri ◽  
Mustafa Man
Keyword(s):  
Data Extraction ◽  
Extraction Process ◽  
Structured Data ◽  
Web Pages ◽  
Web Page ◽  
Web Data ◽  
Web Documents ◽  
Web Extraction ◽  
Comparison Time ◽  
The Web

<p>Web data extraction is the process of extracting user required information from web page. The information consists of semi-structured data not in structured format. The extraction data involves the web documents in html format. Nowadays, most people uses web data extractors because the extraction involve large information which makes the process of manual information extraction takes time and complicated. We present in this paper WEIDJ approach to extract images from the web, whose goal is to harvest images as object from template-based html pages. The WEIDJ (Web Extraction Image using DOM (Document Object Model) and JSON (JavaScript Object Notation)) applies DOM theory in order to build the structure and JSON as environment of programming. The extraction process leverages both the input of web address and the structure of extraction. Then, WEIDJ splits DOM tree into small subtrees and applies searching algorithm by visual blocks for each web page to find images. Our approach focus on three level of extraction; single web page, multiple web page and the whole web page. Extensive experiments on several biodiversity web pages has been done to show the comparison time performance between image extraction using DOM, JSON and WEIDJ for single web page. The experimental results advocate via our model, WEIDJ image extraction can be done fast and effectively.</p>

IMPROVED WEB PAGE IDENTIFICATION METHOD USING NEURAL NETWORKS

2011 ◽  
Vol 10 (01) ◽  
pp. 87-114 ◽  
Author(s):  
ALI SELAMAT ◽  
ZHI SAM LEE ◽  
MOHD AIZAINI MAAROF ◽  
SITI MARIYAM SHAMSUDDIN
Keyword(s):  
Neural Networks ◽  
Feature Selection ◽  
Weighting Scheme ◽  
Web Pages ◽  
Web Page ◽  
Term Weighting ◽  
Web Documents ◽  
Web Contents ◽  
Feature Selection Approach ◽  
The Web

In this paper, an improved web page classification method (IWPCM) using neural networks to identify the illicit contents of web pages is proposed. The proposed IWPCM approach is based on the improvement of feature selection of the web pages using class based feature vectors (CPBF). The CPBF feature selection approach has been calculated by considering the important term's weight for illicit web documents and reduce the dependency of the less important term's weight for normal web documents. The IWPCM approach has been examined using the modified term-weighting scheme by comparing it with several traditional term-weighting schemes for non-illicit and illicit web contents available from the web. The precision, recall, and F1 measures have been used to evaluate the effectiveness of the proposed IWPCM approach. The experimental results have shown that the proposed improved term-weighting scheme has been able to identify the non-illicit and illicit web contents available from the experimental datasets.

scholarly journals An Enhanced Web Document Search Engine using a Semantic Network

2016 ◽  
Author(s):  
Sang Thanh Thi Nguyen ◽  
Tuan Thanh Nguyen
Keyword(s):  
Web Search ◽  
Latent Dirichlet Allocation ◽  
Topic Model ◽  
Semantic Network ◽  
Daily Basis ◽  
Web Pages ◽  
Web Crawler ◽  
Web Document ◽  
Modelling Techniques ◽  
The Web

With the rapid advancement of ICT technology, the World Wide Web (referred to as the Web) has become the biggest information repository whose volume keeps growing on a daily basis. The challenge is how to find the most wanted information from the Web with a minimum effort. This paper presents a novel ontology-based framework for searching the related web pages to a given term within a few given specific websites. With this framework, a web crawler first learns the content of web pages within the given websites, then the topic modeller finds the relations between web pages and topics via key words found on the web pages using the Latent Dirichlet Allocation (LDA) technique. After that, the ontology builder establishes an ontology which is a semantic network of web pages based on the topic model. Finally, a reasoner can find the related web pages to a given term by making use of the ontology. The framework and related modelling techniques have been verified using a few test websites and the results convince its superiority over the existing web search tools.

Content-based analysis to detect Arabic web spam

2012 ◽  
Vol 38 (3) ◽  
pp. 284-296 ◽  
Author(s):  
Mohammed Al-Kabi ◽  
Heider Wahsheh ◽  
Izzat Alsmadi ◽  
Emad Al-Shawakfa ◽  
Abdullah Wahbeh ◽  
...  
Keyword(s):  
Decision Tree ◽  
Search Engines ◽  
Daily Basis ◽  
Classification Algorithms ◽  
Web Pages ◽  
Huge Amount ◽  
Dynamic Nature ◽  
Search Results ◽  
Web Spam ◽  
The Web

Search engines are important outlets for information query and retrieval. They have to deal with the continual increase of information available on the web, and provide users with convenient access to such huge amounts of information. Furthermore, with this huge amount of information, a more complex challenge that continuously gets more and more difficult to illuminate is the spam in web pages. For several reasons, web spammers try to intrude in the search results and inject artificially biased results in favour of their websites or pages. Spam pages are added to the internet on a daily basis, thus making it difficult for search engines to keep up with the fast-growing and dynamic nature of the web, especially since spammers tend to add more keywords to their websites to deceive the search engines and increase the rank of their pages. In this research, we have investigated four different classification algorithms (naïve Bayes, decision tree, SVM and K-NN) to detect Arabic web spam pages, based on content. The three groups of datasets used, with 1%, 15% and 50% spam contents, were collected using a crawler that was customized for this study. Spam pages were classified manually. Different tests and comparisons have revealed that the Decision Tree was the best classifier for this purpose.

A Machine Learning-Based Model to Evaluate Readability and Assess Grade Level for the Web Pages

2020 ◽  
Author(s):  
Muralidhar Pantula ◽  
K S Kuppusamy
Keyword(s):  
Machine Learning ◽  
Grade Level ◽  
Web Pages ◽  
Word Count ◽  
Web Page ◽  
Web Documents ◽  
Browser Extension ◽  
Statistical Measures ◽  
Proposed Model ◽  
The Web

Abstract Evaluating readability of web documents has gained attention due to several factors such as improving the effectiveness of writing and to reach a wider spectrum of audience. Current practices in this direction follow several statistical measures in evaluating readability of the document. In this paper, we have proposed a machine learning-based model to compute readability of web pages. The minimum educational standards required (grade level) to understand the contents of a web page are also computed. The proposed model classifies the web pages into highly readable, readable or less readable using specified feature set. To classify a web page with the aforementioned categories, we have incorporated the features such as sentence count, word count, syllable count, type-token ratio and lexical ambiguity. To increase the usability of the proposed model, we have developed an accessible browser extension to perform the assessments of every web page loaded into the browser.

Probability Association Approach in Automatic Image Annotation

2008 ◽  
pp. 615-626
Author(s):  
Feng Xu ◽  
Yu-Jin Zhang
Keyword(s):  
Large Scale ◽  
Image Annotation ◽  
Public Life ◽  
Image Database ◽  
Web Pages ◽  
Challenging Problem ◽  
Semantic Concepts ◽  
Automated Image Annotation ◽  
High Level ◽  
The Web

Content-based image retrieval (CBIR) has wide applications in public life. Either from a static image database or from the Web, one can search for a specific image, generally browse to make an interactive choice, and search for a picture to go with a broad story or to illustrate a document. Although CBIR has been well studied, it is still a challenging problem to search for images from a large image database because of the well-acknowledged semantic gap between low-level features and high-level semantic concepts. An alternative solution is to use keyword-based approaches, which usually associate images with keywords by either manually labeling or automatically extracting surrounding text from Web pages. Although such a solution is widely adopted by most existing commercial image search engines, it is not perfect. First, manual annotation, though precise, is expensive and difficult to extend to large-scale databases. Second, automatically extracted surrounding text might by incomplete and ambiguous in describing images, and even more, surrounding text may not be available in some applications. To overcome these problems, automated image annotation is considered as a promising approach in understanding and describing the content of images.

Techniques for Refreshing Images in Web Documents

2011 ◽  
Vol 403-408 ◽  
pp. 1008-1013 ◽  
Author(s):  
Divya Ragatha Venkata ◽  
Deepika Kulshreshtha
Keyword(s):  
Search Engine ◽  
Major Part ◽  
Web Pages ◽  
End User ◽  
Web Page ◽  
Web Documents ◽  
Client Server ◽  
Web Document ◽  
Server Architecture ◽  
The Web

In this paper, we put forward a technique for keeping web pages up-to-date, later used by search engine to serve the end user queries. A major part of the Web is dynamic and hence, a need arises to constantly update the changed web documents in search engine’s repository. In this paper we used the client-server architecture for crawling the web and propose a technique for detecting changes in web page based on the content of the images present if any in web documents. Once it is being identified that the image embedded in the web document is changed then the previous copy of the web document present in the search engine’s database/repository is replaced with the changed one.

Orthographic Errors in Web Pages: Toward Cleaner Web Corpora

2006 ◽  
Vol 32 (3) ◽  
pp. 295-340 ◽  
Author(s):  
Christoph Ringlstetter ◽  
Klaus U. Schulz ◽  
Stoyan Mihov
Keyword(s):  
Corpus Linguistics ◽  
Knowledge Bases ◽  
High Accuracy ◽  
Linguistic Knowledge ◽  
Web Pages ◽  
Public Repository ◽  
Web Documents ◽  
Grammatical Errors ◽  
Orthographic Errors ◽  
The Web

Since the Web by far represents the largest public repository of natural language texts, recent experiments, methods, and tools in the area of corpus linguistics often use the Web as a corpus. For applications where high accuracy is crucial, the problem has to be faced that a non-negligible number of orthographic and grammatical errors occur in Web documents. In this article we investigate the distribution of orthographic errors of various types in Web pages. As a by-product, methods are developed for efficiently detecting erroneous pages and for marking orthographic errors in acceptable Web documents, reducing thus the number of errors in corpora and linguistic knowledge bases automatically retrieved from the Web.

Sign in / Sign up

Export Citation Format

Share Document