scholarly journals Log File Data Extraction or Mining

2021 ◽  
Vol 9 (VI) ◽  
pp. 4802-4806
Author(s):  
Sayalee Ghule
Keyword(s):  
Web Mining ◽  
Data Extraction ◽  
Web Server ◽  
Basic Data ◽  
The Internet ◽  
Web Pages ◽  
Time Stamp ◽  
Ip Address ◽  
Log File ◽  
Web Server Logs

Log records contain data generally Client Title, IP Address, Time Stamp, Get to Ask, number of Bytes Exchanged, Result Status, URL that Intimated, and Client Chairman. The log records are kept up by the internet servers. By analysing these log records gives a flawless thought to the client. The wide Web may be a solid store of web pages that gives the Net clients piles of information. With the change in the number and complexity of Websites, the degree of the net has gotten to be massively wide. Web Utilization Mining may be a division of web mining that consolidates the application of mining strategies to web server logs in coordination to expel the behaviour of clients. Log records contain basic data around the execution of a framework. This data is frequently utilized for investigating, operational profiling, finding quirks, recognizing security dangers, measuring execution,

APLIKASI PENCARI INFORMASI PERMINTAAN LAYANAN WEB YANG ERROR MENGGUNAKAN ALGORITMA BOYER-MOORE

2016 ◽  
Vol 1 (1) ◽  
pp. 001
Author(s):  
Harry Setya Hadi
Keyword(s):  
Data Storage ◽  
Computer Network ◽  
Web Server ◽  
Access Methods ◽  
Ip Address ◽  
String Searching ◽  
Common Process ◽  
Web Server Logs ◽  
Different Parts ◽  
The Web

String searching is a common process in the processes that made the computer because the text is the main form of data storage. Boyer-Moore is the search string from right to left is considered the most efficient methods in practice, and matching string from the specified direction specifically an algorithm that has the best results theoretically. A system that is connected to a computer network that literally pick a web server that is accessed by multiple users in different parts of both good and bad aim. Any activity performed by the user, will be stored in Web server logs. With a log report contained in the web server can help a web server administrator to search the web request error. Web server log is a record of the activities of a web site that contains the data associated with the IP address, time of access, the page is opened, activities, and access methods. The amount of data contained in the resulting log is a log shed useful information.

scholarly journals User Identification in the Process of Web Usage Data Preprocessing

2019 ◽  
Vol 14 (09) ◽  
pp. 21 ◽  
Author(s):  
Jozef Kapusta ◽  
Michal Munk ◽  
Dominik Halvoník ◽  
Martin Drlík
Keyword(s):  
User Behavior ◽  
Web Server ◽  
Web Browser ◽  
Ip Address ◽  
User Session ◽  
Log Files ◽  
The Us ◽  
Log File ◽  
Default Form

If we are talking about user behavior analytics, we have to understand what the main source of valuable information is. One of these sources is definitely a web server. There are multiple places where we can extract the necessary data. The most common ways are to search for these data in access log, error log, custom log files of web server, proxy server log file, web browser log, browser cookies etc. A web server log is in its default form known as a Common Log File (W3C, 1995) and keeps information about IP address; date and time of visit; ac-cessed and referenced resource. There are standardized methodologies which contain several steps leading to extract new knowledge from provided data. Usu-ally, the first step is in each one of them to identify users, users’ sessions, page views, and clickstreams. This process is called pre-processing. Main goal of this stage is to receive unprocessed web server log file as input and after processing outputs meaningful representations which can be used in next phase. In this pa-per, we describe in detail user session identification which can be considered as most important part of data pre-processing. Our paper aims to compare the us-er/session identification using the STT with the identification of user/session us-ing cookies. This comparison was performed concerning the quality of the se-quential rules generated, i.e., a comparison was made regarding generation useful, trivial and inexplicable rules.

scholarly journals User’s Behaviour inside a Digital Library

Data Mining ◽  
2013 ◽  
pp. 1312-1319
Author(s):  
Marco Scarnò
Keyword(s):  
Data Mining ◽  
Digital Library ◽  
Web Mining ◽  
Dynamic Change ◽  
Web Server ◽  
Scientific Field ◽  
Data Mining Techniques ◽  
Log File ◽  
The Web ◽  
The Way

CASPUR allows many academic Italian institutions located in the Centre-South of Italy to access more than 7 million articles through a digital library platform. The behaviour of its users were analyzed by considering their “traces”, which are stored in the web server log file. Using several web mining and data mining techniques the author discovered a gradual and dynamic change in the way articles are accessed. In particular there is evidence of a journal browsing increase in comparison to the searching mode. Such phenomenon were interpreted using the idea that browsing better meets the needs of users when they want to keep abreast about the latest advances in their scientific field, in comparison to a more generic searching inside the digital library.

Semi-Structured Data Extraction from Heterogenous Sources

2000 ◽  
pp. 83-102 ◽  
Author(s):  
Xiaoying Gao ◽  
Leon Sterling
Keyword(s):  
Knowledge Management ◽  
World Wide ◽  
Data Extraction ◽  
Research Problem ◽  
The Internet ◽  
Web Pages ◽  
Accessible Information ◽  
The World ◽  
Machine Readable ◽  
The Universe

The World Wide Web is known as the “universe of network-accessible information, the embodiment of human knowledge” (W3C, 1999). Internet-based knowledge management aims to use the Internet as the world wide environment for knowledge publishing, searching, sharing, reusing, and integration, and to support collaboration and decision making. However, knowledge on the Internet is buried in documents. Most of the documents are written in languages for human readers. The knowledge contained therein cannot be easily accessed by computer programs such as knowledge management systems. In order to make the Internet “machine readable,” information extraction from Web pages becomes a crucial research problem.

Extracting Knowledge from Web Data

2014 ◽  
Vol 7 (4) ◽  
pp. 27-41
Author(s):  
Hanane Ezzikouri ◽  
Mohamed Fakir ◽  
Cherki Daoui ◽  
Mohamed Erritali
Keyword(s):  
Web Mining ◽  
User Behavior ◽  
Data Extraction ◽  
Research Area ◽  
Web Usage Mining ◽  
Web Data ◽  
Main Research ◽  
Web Log ◽  
Web Usage ◽  
Log File

The user behavior on a website triggers a sequence of queries that have a result which is the display of certain pages. The Information about these queries (including the names of the resources requested and responses from the Web server) are stored in a text file called a log file. Analysis of server log file can provide significant and useful information. Web Mining is the extraction of interesting and potentially useful patterns and implicit information from artifacts or activity related to the World Wide Web. Web usage mining is a main research area in Web mining focused on learning about Web users and their interactions with Web sites. The motive of mining is to find users' access models automatically and quickly from the vast Web log file, such as frequent access paths, frequent access page groups and user clustering. Through Web Usage Mining, several information left by user access can be mined which will provide foundation for decision making of organizations, Also the process of Web mining was defined as the set of techniques designed to explore, process and analyze large masses of consecutive information activities on the Internet, has three main steps: data preprocessing, extraction of reasons of the use and the interpretation of results. This paper will start with the presentation of different formats of web log files, then it will present the different preprocessing method that have been used, and finally it presents a system for “Web content and Usage Mining'' for web data extraction and web site analysis using Data Mining Algorithms Apriori, FPGrowth, K-Means, KNN, and ID3.

Building and Evaluating an Adaptive Smart Web Pages

2012 ◽  
Vol 3 (1) ◽  
pp. 30
Author(s):  
Mona M. Abu Al-Khair ◽  
M. Koutb ◽  
H. Kelash
Keyword(s):  
Web Server ◽  
Web Pages ◽  
Server Logs ◽  
Web Server Logs ◽  
The Web

Each year the number of consumers and the variety of their interests increase. As a result, providers are seeking ways to infer the customer's interests and to adapt their websites to make the content of interest more easily accessible. Assume that past navigation behavior as an indicator of the user's interests. Then, the records of this behavior, kept in the web-server logs, can be mined to extract the user's interests. On this principal, recommendations can be generated, to help old and new website's visitors to find the information about their interest faster.

BUILDING A KNOWLEDGE BASE FOR IMPLEMENTING A WEB-BASED COMPUTERIZED RECOMMENDATION SYSTEM

2007 ◽  
Vol 16 (05) ◽  
pp. 793-828 ◽  
Author(s):  
JUAN D. VELÁSQUEZ ◽  
VASILE PALADE
Keyword(s):  
Knowledge Base ◽  
Web Site ◽  
Web Mining ◽  
Recommendation System ◽  
The Internet ◽  
Web Pages ◽  
Web Based ◽  
Web Logs ◽  
Mining Tools ◽  
The Web

Understanding the web user browsing behaviour in order to adapt a web site to the needs of a particular user represents a key issue for many commercial companies that do their business over the Internet. This paper presents the implementation of a Knowledge Base (KB) for building web-based computerized recommender systems. The Knowledge Base consists of a Pattern Repository that contains patterns extracted from web logs and web pages, by applying various web mining tools, and a Rule Repository containing rules that describe the use of discovered patterns for building navigation or web site modification recommendations. The paper also focuses on testing the effectiveness of the proposed online and offline recommendations. An ample real-world experiment is carried out on a web site of a bank.

scholarly journals Data Mining applied on Web Robots Detection: A Systematic Mapping

2021 ◽  
Author(s):  
Ramon Abilio ◽  
Cristiano Garcia ◽  
Victor Fernandes
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Learning Algorithms ◽  
Web Server ◽  
Machine Learning Algorithms ◽  
Web Pages ◽  
Systematic Mapping ◽  
Mining Methods ◽  
Server Logs ◽  
Web Server Logs

Browsing on Internet is part of the world population’s daily routine. The number of web pages is increasing and so is the amount of published content (news, tutorials, images, videos) provided by them. Search engines use web robots to index web contents and to offer better results to their users. However, web robots have also been used for exploiting vulnerabilities in web pages. Thus, monitoring and detecting web robots’ accesses is important in order to keep the web server as safe as possible. Data Mining methods have been applied to web server logs (used as data source) in order to detect web robots. Then, the main objective of this work was to observe evidences of definition or use of web robots detection by analyzing web server-side logs using Data Mining methods. Thus, we conducted a systematic Literature mapping, analyzing papers published between 2013 and 2020. In the systematic mapping, we analyzed 34 studies and they allowed us to better understand the area of web robots detection, mapping what is being done, the data used to perform web robots detection, the tools, and algorithms used in the Literature. From those studies, we extracted 33 machine learning algorithms, 64 features, and 13 tools. This study is helpful for researchers to find machine learning algorithms, features, and tools to detect web robots by analyzing web server logs.

Web Harvesting

2017 ◽  
pp. 351-378 ◽  
Author(s):  
B. Umamageswari ◽  
R. Kalpana
Keyword(s):  
Web Mining ◽  
Data Extraction ◽  
Web Pages ◽  
Extraction Techniques ◽  
Web Data ◽  
Web Data Extraction ◽  
Region Extraction ◽  
Universal Applicability ◽  
Web Harvesting

Web mining is done on huge amounts of data extracted from WWW. Many researchers have developed several state-of-the-art approaches for web data extraction. So far in the literature, the focus is mainly on the techniques used for data region extraction. Applications which are fed with the extracted data, require fetching data spread across multiple web pages which should be crawled automatically. For this to happen, we need to extract not only data regions, but also the navigation links. Data extraction techniques are designed for specific HTML tags; which questions their universal applicability for carrying out information extraction from differently formatted web pages. This chapter focuses on various web data extraction techniques available for different kinds of data rich pages, classification of web data extraction techniques and comparison of those techniques across many useful dimensions.

Sign in / Sign up

Export Citation Format

Share Document