scholarly journals YOUTUBE DATA ANALYSIS USING HADOOP FRAMEWORK

2021 ◽  
Vol 5 (11) ◽  
Author(s):  
Ashwini T ◽  
Sahana LM ◽  
Mahalakshmi E ◽  
Shweta S Padti
Keyword(s):  
Big Data ◽  
Application Programming Interface ◽  
Structured Data ◽  
Unstructured Data ◽  
Distributed File System ◽  
Hadoop Distributed File System ◽  
Application Programming ◽  
Programming Interface ◽  
Hadoop Framework ◽  
Social Media Tool

— Analysis of consistent and structured data has seen huge success in past decades. Where the analysis of unstructured data in the form of multimedia format remains a challenging task. YouTube is one of the most used and popular social media tool. The main aim of this paper is to analyze the data that is generated from YouTube that can be mined and utilized. API (Application Programming Interface) and going to be stored in Hadoop Distributed File System (HDFS). Dataset can be analyzed using MapReduce. Which is used to identify the video categories in which most number of videos are uploaded. The objective of this paper is to demonstrate Hadoop framework, to process and handle big data there are many components. In the existing method, big data can be analyzed and processed in multiple stages by using MapReduce. Due to huge space consumption of each job, Implementing iterative map reduce jobs is expensive. A Hive method is used to analyze the big data to overcome the drawbacks of existing methods, which is the state-ofthe-art method. The hive works by extracting the YouTube information by generating API (Application Programming Interface) key and uses the SQL queries.

scholarly journals Mining and YouTube Data Analysis using Hadoop

2020 ◽  
Vol 9 (3) ◽  
pp. 1461-1465
Keyword(s):  
Real Time ◽  
Application Programming Interface ◽  
Data Set ◽  
Apache Hadoop ◽  
Informed Decisions ◽  
Hadoop Distributed File System ◽  
Application Programming ◽  
Using Data ◽  
Programming Interface ◽  
Social Media Tool

Analysis of structured and consistent data has seen remarkable success in past decades. Whereas, the analysis of unstructured data in the form of multimedia format remains a challenging task. YouTube is one of the most popular and used social media tool. It reveals the community feedback through comments for published videos, number of likes, dislikes, number of subscribers for a particular channel. The main objective of this work is to demonstrate by using Hadoop concepts, how data generated from YouTube can be mined and utilized to make targeted, real time and informed decisions. In our paper, we analyze the data to identify the top categories in which the most number of videos are uploaded. This YouTube data is publicly available and the YouTube data set is described below under the heading Data Set Description. The dataset will be fetched from the Google using the YouTube API (Application Programming Interface) and going to be stored in Hadoop Distributed File System (HDFS). Using MapReduce we are going to analyze the dataset to identify the video categories in which most number of videos are uploaded. The objective of this paper is to demonstrate Apache Hadoop framework concepts and how to make targeted, real-time and informed decisions using data gathered from YouTube.

scholarly journals Using Natural Language Preprocessing Architecture (NLPA) for Big Data Text Sources

2020 ◽  
Vol 2020 ◽  
pp. 1-13
Author(s):  
María Novo-Lourés ◽  
Reyes Pavón ◽  
Rosalía Laza ◽  
David Ruano-Ordas ◽  
Jose R. Méndez
Keyword(s):  
Big Data ◽  
Natural Language ◽  
Semantic Information ◽  
Application Programming Interface ◽  
Unstructured Data ◽  
Application Programming ◽  
Text Preprocessing ◽  
Programming Interface ◽  
Proper Analysis

During the last years, big data analysis has become a popular means of taking advantage of multiple (initially valueless) sources to find relevant knowledge about real domains. However, a large number of big data sources provide textual unstructured data. A proper analysis requires tools able to adequately combine big data and text-analysing techniques. Keeping this in mind, we combined a pipelining framework (BDP4J (Big Data Pipelining For Java)) with the implementation of a set of text preprocessing techniques in order to create NLPA (Natural Language Preprocessing Architecture), an extendable open-source plugin implementing preprocessing steps that can be easily combined to create a pipeline. Additionally, NLPA incorporates the possibility of generating datasets using either a classical token-based representation of data or newer synset-based datasets that would be further processed using semantic information (i.e., using ontologies). This work presents a case study of NLPA operation covering the transformation of raw heterogeneous big data into different dataset representations (synsets and tokens) and using the Weka application programming interface (API) to launch two well-known classifiers.

scholarly journals Integración de herramientas para la toma de decisiones en la congestión vehicular

DYNA ◽  
2018 ◽  
Vol 85 (205) ◽  
pp. 363-370
Author(s):  
Nelson Ivan Herrera-Herrera ◽  
Sergio Luján-Mora ◽  
Estevan Ricardo Gómez-Torres
Keyword(s):  
Big Data ◽  
Application Programming Interface ◽  
Apache Hadoop ◽  
Toma De Decisiones ◽  
Application Programming ◽  
La Red ◽  
Apache Pig ◽  
Programming Interface

Este estudio tiene como finalidad presentar un análisis de la utilización e integración de herramientas tecnológicas que ayudan a tomar decisiones en situaciones de congestión vehicular. La ciudad de Quito-Ecuador es considerada como un caso de estudio para el trabajo realizado. La investigación se presenta en función del desarrollo de una aplicación, haciendo uso de herramientas Big Data (Apache Flume, Apache Hadoop, Apache Pig), que permiten el procesamiento de gran cantidad de información que se requiere recolectar, almacenar y procesar. Uno de los aspectos innovadores de la aplicación es el uso de la red social Twitter como fuente de origen de datos. Para esto se utilizó su interfaz de programación de aplicaciones (Application Programming Interface, API), la cual permite tomar datos de esta red social en tiempo real e identificar puntos probables de congestión. Este estudio presenta resultados de pruebas realizadas con la aplicación, durante un período de 9 meses.

scholarly journals Sentiment Analysis on Overseas Tweets on the Impact of COVID-19 in Indonesia

2021 ◽  
Vol 5 (2) ◽  
pp. 304-313
Author(s):  
Tigor Nirman Simanjuntak ◽  
Setia Pramana
Keyword(s):  
Big Data ◽  
Application Programming Interface ◽  
Text Format ◽  
Python Programming Language ◽  
Application Programming ◽  
Python Programming ◽  
Negative Sentiment ◽  
Analytical Tools ◽  
The Impact ◽  
Programming Interface

This study aims to conduct analysis to determine the trend of sentiment on tweets about Covid-19 in Indonesia from the Twitter accounts overseas on big data perspective. The data was obtained from Twitter in the period of April 2020, with the word query "Indonesian Corona Virus" from foreign user accounts in English. The process of retrieving data comes from Twitter tweets by crawling the text using Twitter's API (Application Programming Interface) by employing Python programming language. Twitter was chosen because it is very fast and easy to spread through status updates from and among the user accounts. The number of tweets obtained was 8,740 in text format, with a total engagement of 217,316. The data was sorted from the tweets with the largest to smallest engagement, then cleaned from unnecessary fonts and symbols as well as typo words and abbreviations. The sentiment classification was carried out by analytical tools, extracting information with text mining, into positive, negative, and neutral polarity. To sharpen the analysis, the cleaned data was selected only with the largest engagement until those with 100 engagements; then was grouped into 30 sub-topics to be analyzed. The interesting facts are found that most tweets and sub-topics were dominated by the negative sentiment; and some unthinkable sub-topics were talked by many users.

scholarly journals Practical Training of Students on the Extraction and Analysis of Big Data

2018 ◽  
Vol 3 (2) ◽  
pp. 361 ◽  
Author(s):  
Prokhorov I.V. ◽  
Kochetkov O.T. ◽  
Filatov A.A.
Keyword(s):  
Big Data ◽  
Information Systems ◽  
Information Security ◽  
Application Programming Interface ◽  
The Internet ◽  
Practical Training ◽  
Security Systems ◽  
Application Programming ◽  
Data Scientist ◽  
Programming Interface

The article deals with questions of studies, development and practical use in teaching complex laboratory work on extracting and analyzing big data to train specialists in the specialty 10.05.04 "Information and Analytical Security Systems", direction of training "Information sSecurity of Financial and Economic Structures" in the framework of the educational discipline "Distributed Automated Information Systems". Keywords: big data, data scientist, extraction, processing and analysis of big data, information security of financial and economic structures, the Internet, Yandex, Google, application programming interface –API.

scholarly journals The File System Recommendations to Reduce the Space and Time Parameters in Hadoop File Storage and Map Reduce Processing of Big Data Applications

2020 ◽  
Vol 9 (10) ◽  
pp. 353-356
Keyword(s):  
Big Data ◽  
Data Processing ◽  
Data Storage ◽  
File System ◽  
Distributed File System ◽  
Map Reduce ◽  
Space And Time ◽  
File Storage ◽  
Hadoop Distributed File System ◽  
Hadoop Framework

The study of Hadoop Distributed File System (HDFS) and Map Reduce (MR) are the key aspects of the Hadoop framework. The big data scenarios like Face Book (FB) data processing or the twitter analytics such as storing the tweets and processing the tweets is other scenario of big data which can depends on Hadoop framework to perform the storage and processing through which further analytics can be done. The point here is the usage of space and time in the processing of the above-mentioned huge amounts of the data definitely leads to higher amounts of space and time consumption of the Hadoop framework. The problem here is usage of huge amounts of the space and at the same time the processing time is also high which need to be reduced so as to get the fastest response from the framework. The attempt is important as all the other eco system tools also depends on HDFS and MR so as to perform the data storage and processing of the data and alternative architecture so as to improve the usage of the space and effective utilization of the resources so as to reduce the time requirements of the framework. The outcome of the work is faster data processing and less space utilization of the framework in the processing of MR along with other eco system tools like Hive, Flume, Sqoop and Pig Latin. The work is proposing an alternative framework of the HDFS and MR and the name we are assigning is Unified Space Allocation and Data Processing with Metadata based Distributed File System (USAMDFS).

scholarly journals Scraping and Analysing YouTube Trending Videos for BI

2021 ◽  
Author(s):  
Sowmiya K ◽  
Supriya S ◽  
R. Subhashini
Keyword(s):  
Large Scale ◽  
Application Programming Interface ◽  
Unstructured Data ◽  
Huge Amount ◽  
Channel Process ◽  
The Past ◽  
Data Process ◽  
Application Programming ◽  
Video Format ◽  
Programming Interface

Analysis of structured data has seen tremendous success in the past. However, large scale of an unstructured data have been analysed in the form of video format remains a challenging area. YouTube, a Google company, has over a billion users and it used to generate billions of views. Since YouTube data is getting created in a very huge amount with a lot of views and with an equally great speed, there is a huge demand to store the data, process the data and carefully study the data in this large amount of it usable. The project utilizes the YouTube Data API (Application Programming Interface) which allows the applications or websites to incorporate functions in which they are used by YouTube application to fetch and view the information. The Google Developers Console which is used to generate an unique access key which is further required to fetch the data from YouTube public channel. Process the data and finally data stored in AWS. This project extracts the meaningful output of which can be used by the management for analysis, these methodologies helpful for business intelligence.

Sistem Pengawasan Aktifitas Penggunaan Smartphone Android

2018 ◽  
Vol 9 (1) ◽  
pp. 24-31
Author(s):  
Rudianto Rudianto ◽  
Eko Budi Setiawan
Keyword(s):  
Application Programming Interface ◽  
Third Party ◽  
Image Content ◽  
Image File ◽  
Application Programming ◽  
Index Terms ◽  
Programming Interface

Availability the Application Programming Interface (API) for third-party applications on Android devices provides an opportunity to monitor Android devices with each other. This is used to create an application that can facilitate parents in child supervision through Android devices owned. In this study, some features added to the classification of image content on Android devices related to negative content. In this case, researchers using Clarifai API. The result of this research is to produce a system which has feature, give a report of image file contained in target smartphone and can do deletion on the image file, receive browser history report and can directly visit in the application, receive a report of child location and can be directly contacted via this application. This application works well on the Android Lollipop (API Level 22). Index Terms— Application Programming Interface(API), Monitoring, Negative Content, Children, Parent.

Sign in / Sign up

Export Citation Format

Share Document