Automatic Generation of Ontology for Extracting Hidden Web Pages

2017 ◽  
pp. 127-139 ◽  
Author(s):  
Manvi ◽  
Komal Kumar Bhatia ◽  
Ashutosh Dixit
Keyword(s):  
Automatic Generation ◽  
Web Pages ◽  
Hidden Web

Automatic generation of agents for collecting hidden Web pages for data extraction

2004 ◽  
Vol 49 (2) ◽  
pp. 177-196 ◽  
Author(s):  
Juliano Palmieri Lage ◽  
Altigran S. da Silva ◽  
Paulo B. Golgher ◽  
Alberto H.F. Laender
Keyword(s):  
Data Extraction ◽  
Automatic Generation ◽  
Web Pages ◽  
Hidden Web

scholarly journals Automated Web Application ...

10.29007/vs62 ◽  
2018 ◽  
Author(s):  
Priyank Bhojak ◽  
Vatsal Shah ◽  
Kanu Patel ◽  
Deven Gol
Keyword(s):  
Dynamic Analysis ◽  
Open Source ◽  
Web Application ◽  
Web Applications ◽  
Black Box ◽  
Web Pages ◽  
Software Bugs ◽  
Hidden Web ◽  
A New Technique ◽  
Input Validation

The rate of web application threats is growing more and more now in days. The most of software bugs are result from inappropriate input validation. It should lead to attack of confidential information, breaking of knowledge integrity. We develop a scanner for detecting SQ injection and XSS type software-bugs which is based on hidden web crawl and make open source scanner with the aim of hidden web crawling which may be require authentication. In this research paper we presents a new technique to find vulnerability which include advantages of black-box analysis of different web pages. And at the end we shows evaluation table which mention comparison of our scanner with two other web scanner tool. So finally this paper additionally shows how easy it is to scan web application bugs with dynamic analysis and retrieve hidden web pages from web applications.

scholarly journals A scale for crawler effectiveness on the client-side hidden web

2012 ◽  
Vol 9 (2) ◽  
pp. 561-583 ◽  
Author(s):  
Víctor Prieto ◽  
Manuel Álvarez ◽  
Rafael López-García ◽  
Fidel Cacheda
Keyword(s):  
Open Source ◽  
Search Engines ◽  
Web Site ◽  
Point Of View ◽  
Web Pages ◽  
Common Features ◽  
Hidden Web ◽  
Client Side ◽  
The Web

The main goal of this study is to present a scale that classifies crawling systems according to their effectiveness in traversing the ?clientside? Hidden Web. First, we perform a thorough analysis of the different client-side technologies and the main features of the web pages in order to determine the basic steps of the aforementioned scale. Then, we define the scale by grouping basic scenarios in terms of several common features, and we propose some methods to evaluate the effectiveness of the crawlers according to the levels of the scale. Finally, we present a testing web site and we show the results of applying the aforementioned methods to the results obtained by some open-source and commercial crawlers that tried to traverse the pages. Only a few crawlers achieve good results in treating client-side technologies. Regarding standalone crawlers, we highlight the open-source crawlers Heritrix and Nutch and the commercial crawler WebCopierPro, which is able to process very complex scenarios. With regard to the crawlers of the main search engines, only Google processes most of the scenarios we have proposed, while Yahoo! and Bing just deal with the basic ones. There are not many studies that assess the capacity of the crawlers to deal with client-side technologies. Also, these studies consider fewer technologies, fewer crawlers and fewer combinations. Furthermore, to the best of our knowledge, our article provides the first scale for classifying crawlers from the point of view of the most important client-side technologies.

scholarly journals Real-time Monitoring of Greenhouse Climate Control Using the Internet

2001 ◽  
Vol 11 (4) ◽  
pp. 639-643 ◽  
Author(s):  
Niels Ehler ◽  
Jesper M. Aaslyng
Keyword(s):  
Real Time ◽  
Automatic Generation ◽  
Web Pages ◽  
Climatic Data ◽  
Climate Control ◽  
Dynamic Information ◽  
Ongoing Research ◽  
Greenhouse Climate ◽  
Internet Application ◽  
And Control

The possibility of constructing an Internet application that would enable greenhouse users to track climate and control parameters from any Internet-connected computer was investigated. By constructing a set of HTML-templates, dynamic information from the control-system databases was integrated in real-time, and was uploaded to a common net-server by automatic generation of web pages using software developed during the project. Good performance, reliability and security were obtained and the technology proved to be an efficient way of supplying a broad range of users not only with climatic data but also with results from ongoing research.

scholarly journals IHWC: intelligent hidden web crawler for harvesting data in urban domains

2021 ◽  
Author(s):  
Sawroop Kaur ◽  
Aman Singh ◽  
G. Geetha ◽  
Xiaochun Cheng
Keyword(s):  
Large Scale ◽  
Smart Cities ◽  
Quality Data ◽  
Web Pages ◽  
Web Crawler ◽  
Harvest Rate ◽  
Web Interfaces ◽  
Hidden Web ◽  
Special Cases ◽  
Daunting Task

AbstractDue to the massive size of the hidden web, searching, retrieving and mining rich and high-quality data can be a daunting task. Moreover, with the presence of forms, data cannot be accessed easily. Forms are dynamic, heterogeneous and spread over trillions of web pages. Significant efforts have addressed the problem of tapping into the hidden web to integrate and mine rich data. Effective techniques, as well as application in special cases, are required to be explored to achieve an effective harvest rate. One such special area is atmospheric science, where hidden web crawling is least implemented, and crawler is required to crawl through the huge web to narrow down the search to specific data. In this study, an intelligent hidden web crawler for harvesting data in urban domains (IHWC) is implemented to address the relative problems such as classification of domains, prevention of exhaustive searching, and prioritizing the URLs. The crawler also performs well in curating pollution-related data. The crawler targets the relevant web pages and discards the irrelevant by implementing rejection rules. To achieve more accurate results for a focused crawl, ICHW crawls the websites on priority for a given topic. The crawler has fulfilled the dual objective of developing an effective hidden web crawler that can focus on diverse domains and to check its integration in searching pollution data in smart cities. One of the objectives of smart cities is to reduce pollution. Resultant crawled data can be used for finding the reason for pollution. The crawler can help the user to search the level of pollution in a specific area. The harvest rate of the crawler is compared with pioneer existing work. With an increase in the size of a dataset, the presented crawler can add significant value to emission accuracy. Our results are demonstrating the accuracy and harvest rate of the proposed framework, and it efficiently collect hidden web interfaces from large-scale sites and achieve higher rates than other crawlers.

Sign in / Sign up

Export Citation Format

Share Document