Towards a Spatial Instance Learning Method for Deep Web Pages

2011 ◽  
pp. 270-285 ◽  
Author(s):  
Ermelinda Oro ◽  
Massimo Ruffolo
Keyword(s):  
Deep Web ◽  
Web Pages ◽  
Learning Method

scholarly journals Cross-Fertilizing Deep Web Analysis and Ontology Enrichment

2017 ◽  
Author(s):  
Marilena Oita ◽  
Antoine Amarilli ◽  
Pierre Senellart
Keyword(s):  
Domain Knowledge ◽  
Deep Web ◽  
Web Pages ◽  
Complete Understanding ◽  
Specific Knowledge ◽  
Domain Specific ◽  
Domain Specific Knowledge ◽  
Web Crawlers ◽  
New Perspective ◽  
The Impact

Deep Web databases, whose content is presented as dynamically-generated Web pages hidden behind forms, have mostly been left unindexed by search engine crawlers. In order to automatically explore this mass of information, many current techniques assume the existence of domain knowledge, which is costly to create and maintain. In this article, we present a new perspective on form understanding and deep Web data acquisition that does not require any domain-specific knowledge. Unlike previous approaches, we do not perform the various steps in the process (e.g., form understanding, record identification, attribute labeling) independently but integrate them to achieve a more complete understanding of deep Web sources. Through information extraction techniques and using the form itself for validation, we reconcile input and output schemas in a labeled graph which is further aligned with a generic ontology. The impact of this alignment is threefold: first, the resulting semantic infrastructure associated with the form can assist Web crawlers when probing the form for content indexing; second, attributes of response pages are labeled by matching known ontology instances, and relations between attributes are uncovered; and third, we enrich the generic ontology with facts from the deep Web.

A COMPETITIVE NEURAL NETWORK APPROACH TO WEB-PAGE CATEGORIZATION

2001 ◽  
Vol 09 (06) ◽  
pp. 731-741 ◽  
Author(s):  
ZHI-QIANG LIU ◽  
YA-JUN ZHANG
Keyword(s):  
Neural Network ◽  
Real World ◽  
Gradient Descent ◽  
Web Pages ◽  
Learning Method ◽  
Network Approach ◽  
Web Page ◽  
Real World Data ◽  
Neural Network Approach ◽  
Competitive Neural Network

Recently many techniques, e.g., Google or AltaVista, are available for classifying well-organized, hierarchical crisp categories from human constructed web pages such as that in Yahoo. However, given the current rate of web-page production, there is an urgent need of classifiers that are able to autonomously classify web-page categories that have overlaps. In this paper, we present a competitive learning method for this problem, which based on a new objective function and gradient descent scheme. Experimental results on real-world data show that the approach proposed in this paper gives a better performance in classifying randomly-generated, knowledge-overlapped categories as well as hierarchical crisp categories.

Search Engine

The Dark Web ◽  
2018 ◽  
pp. 359-374
Author(s):  
Dilip Kumar Sharma ◽  
A. K. Sharma
Keyword(s):  
Computer Networks ◽  
Search Engines ◽  
Web Search ◽  
Relevant Information ◽  
Vital Role ◽  
Deep Web ◽  
Telecommunication Networks ◽  
Web Pages ◽  
Web Crawler ◽  
Main Components

ICT plays a vital role in human development through information extraction and includes computer networks and telecommunication networks. One of the important modules of ICT is computer networks, which are the backbone of the World Wide Web (WWW). Search engines are computer programs that browse and extract information from the WWW in a systematic and automatic manner. This paper examines the three main components of search engines: Extractor, a web crawler which starts with a URL; Analyzer, an indexer that processes words on the web page and stores the resulting index in a database; and Interface Generator, a query handler that understands the need and preferences of the user. This paper concentrates on the information available on the surface web through general web pages and the hidden information behind the query interface, called deep web. This paper emphasizes the Extraction of relevant information to generate the preferred content for the user as the first result of his or her search query. This paper discusses the aspect of deep web with analysis of a few existing deep web search engines.

Search Engine

2011 ◽  
Vol 3 (2) ◽  
pp. 38-51 ◽  
Author(s):  
Dilip Kumar Sharma ◽  
A. K. Sharma
Keyword(s):  
Computer Networks ◽  
Search Engines ◽  
Web Search ◽  
Relevant Information ◽  
Vital Role ◽  
Deep Web ◽  
Telecommunication Networks ◽  
Web Pages ◽  
Web Crawler ◽  
Main Components

ICT plays a vital role in human development through information extraction and includes computer networks and telecommunication networks. One of the important modules of ICT is computer networks, which are the backbone of the World Wide Web (WWW). Search engines are computer programs that browse and extract information from the WWW in a systematic and automatic manner. This paper examines the three main components of search engines: Extractor, a web crawler which starts with a URL; Analyzer, an indexer that processes words on the web page and stores the resulting index in a database; and Interface Generator, a query handler that understands the need and preferences of the user. This paper concentrates on the information available on the surface web through general web pages and the hidden information behind the query interface, called deep web. This paper emphasizes the Extraction of relevant information to generate the preferred content for the user as the first result of his or her search query. This paper discusses the aspect of deep web with analysis of a few existing deep web search engines.

Web Form Entrance Detection and Automatic Form Filling

2013 ◽  
Vol 380-384 ◽  
pp. 2712-2715
Author(s):  
Wen Qian Shang
Keyword(s):  
Research Area ◽  
Deep Web ◽  
Experimental Results ◽  
Web Pages ◽  
Information Mining ◽  
Form Analysis ◽  
Entire Process ◽  
Web Information

At present, deep web information mining is a considerably potential research area. How to get this massive and valuable information hidden after the database is need to be studied further. So this paper presents an approach that includes web pages analysis, getting forms, form analysis, automatic form filling, automatic form submission and acquiring returning pages. The aim is to let the computer automatically complete this process. The experimental results show the feasibility of this method. It can automatically complete the entire process.

Crawling the Deep Web Using Asynchronous Advantage Actor Critic Technique

2021 ◽  
Author(s):  
Kapil Madan ◽  
Rajesh Bhatia
Keyword(s):  
Value Function ◽  
General Purpose ◽  
Deep Web ◽  
Web Pages ◽  
Coverage Ratio ◽  
Digital World ◽  
Learning Technique ◽  
Policy Gradient ◽  
Structured Information ◽  
Surface Web

In the digital world, World Wide Web magnitude is expanding very promptly. Now a day, a rising number of data-centric websites require a mechanism to crawl the information. The information accessible through hyperlinks can easily be retrieved with general-purpose search engines. A massive chunk of the structured information is invisible behind the search forms. Such immense information is recognized as the deep web and has structured information as compared to the surface web. Crawling the content of deep web is very challenging and requires filling the search forms with suitable queries. This paper proposes an innovative technique using an Asynchronous Advantage Actor-Critic (A3C) to explore the unidentified deep web pages. It is based on the policy gradient deep reinforcement learning technique that parameterizes the policy and value function based on the reward system. A3C has one coordinator and various agents. These agents learn from different environments, update the local gradients to a coordinator, and produce a more stable system. The proposed technique has been validated with Open Directory Project (ODP). The experimental outcome shows that the proposed technique outperforms most of the prevailing techniques based on various metrics such as average precision-recall, average harvest rate, and coverage ratio.

scholarly journals Efficient Methodology for Deep Web Data Extraction

2021 ◽  
Vol 12 (1S) ◽  
pp. 286-293
Author(s):  
Shilpa Deshmukh, Et. al.
Keyword(s):  
Information Extraction ◽  
Data Extraction ◽  
Deep Web ◽  
Web Pages ◽  
Web Data ◽  
Web Information Extraction ◽  
Web Data Extraction ◽  
Web Information ◽  
Assessment Measure ◽  
Enormous Number

Deep Web substance are gotten to by inquiries submitted to Web information bases and the returned information records are enwrapped in progressively created Web pages (they will be called profound Web pages in this paper). Removing organized information from profound Web pages is a difficult issue because of the fundamental mind boggling structures of such pages. As of not long ago, an enormous number of strategies have been proposed to address this issue, however every one of them have characteristic impediments since they are Web-page-programming-language subordinate. As the mainstream two-dimensional media, the substance on Web pages are constantly shown routinely for clients to peruse. This inspires us to look for an alternate path for profound Web information extraction to beat the constraints of past works by using some fascinating normal visual highlights on the profound Web pages. In this paper, a novel vision-based methodology that is Visual Based Deep Web Data Extraction (VBDWDE) Algorithm is proposed. This methodology basically uses the visual highlights on the profound Web pages to execute profound Web information extraction, including information record extraction and information thing extraction. We additionally propose another assessment measure amendment to catch the measure of human exertion expected to create wonderful extraction. Our investigations on a huge arrangement of Web information bases show that the proposed vision-based methodology is exceptionally viable for profound Web information extraction.

Semi-Supervised Classification with Co-Training for Deep Web

2010 ◽  
Vol 439-440 ◽  
pp. 183-188 ◽  
Author(s):  
Wei Fang ◽  
Zhi Ming Cui
Keyword(s):  
Supervised Classification ◽  
Deep Web ◽  
Unlabeled Data ◽  
Supervised Machine Learning ◽  
Web Pages ◽  
The Novel ◽  
Training Process ◽  
Novel Approach ◽  
The Cost

The main problems in Web Pages classification are lack of labeled data, as well as the cost of labeling the unlabeled data. In this paper we discuss the application of semi-supervised machine learning method co-training on classification of Deep Web query interfaces to boost the performance of a classifier. Then, Bayes and Maxim Entropy algorithm are co-operated to incorporate labeled data with unlabeled data in training process incrementally. Our experiment results show the novel approach has a promising performance.

Sign in / Sign up

Export Citation Format

Share Document