Hadoop Based Generic Template for Performing Sentiment Analysis Using Apache PIG

Every day over 2.5 quintillion data is generated using various channels like online surveys, transactional data tracking, social media monitoring, etc. Out of these majority of the data is generated using social media platforms. This raw data contains information that can be used for industrial, economic, social and business purposes. To facilitate this, sentiment analysis has become a prospect for various tech-based industry giants to review and analyze their products. Hadoop has been established as one of the best tools for storing, processing, and streaming data in the market. In this paper, we present a generic approach to performing sentiment analysis using Apache PIG which classifies the given data taken from a dataset to either positive or negative to get the people’s sentiment over an object or an issue.

The research of social processes at the university using big data

The volume of information in the 21st century is growing at a rapid pace. Big data technologies are used to process modern information. This article discusses the use of big data technologies to implement monitoring of social processes. Big data has its characteristics and principles, which reflect here. In addition, we also discussed big data applications in some areas. Particular attention in this article pays to the interactions of big data and sociology. For this, there consider digital sociology and computational social sciences. One of the main objects of study in sociology is social processes. The article shows the types of social processes and their monitoring. As an example, there is implemented monitoring of social processes at the university. There are used following technologies for the realization of social processes monitoring: products 1010data (1010edge, 1010connect, 1010reveal, 1010equities), products of Apache Software Foundation (Apache Hive, Apache Chukwa, Apache Hadoop, Apache Pig), MapReduce framework, language R, library Pandas, NoSQL, etc. Despite this, this article examines the use of the MapReduce model for social processes monitoring at the university.

Twitter data analysis using hadoop ecosystems and apache zeppelin

The day-to-day life of the people doesn't depend only on what they think, but it is affected and influenced by what others think. The advertisements and campaigns of the favourite celebrities and mesmerizing personalities influence the way people think and see the world. People get the news and information at lightning speed than ever before. The growth of textual data on the internet is very fast. People express themselves in various ways on the web every minute. They make use of various platforms to share their views and opinions. A huge amount of data is being generated at every moment on this process. Being one of the most important and well-known social media of the present time, millions of tweets are posted on Twitter every day. These tweets are a source of very important information and it can be made use for business, small industries, creating government policies, and various studies can be performed by using it. This paper focuses on the location from where the tweets are posted and the language in which the tweets are written. These details can be effectively analysed by using Hadoop. Hadoop is a tool that is used to analyze distributed big data, streaming data, timestamp data and text data. With the help of Apache Flume, the tweets can be collected from Twitter and then sink in the HDFS (Hadoop Distributed File System). These raw data then analyzed using Apache Pig and the information available can be made use for social and commercial purposes. The result will be visualized using Apache Zeppelin.

An Overview of Apache Pig and Apache Hive

Ever since the enhancement of technology has taken place, the data is growing at an alarming rate. The most prominent factor of data growth is the “Social Media”, leads to the origination of a tremendous amount of data called Big Data. Big Data is a term used for data sets that are extremely large in size as well as complicated to store and process using traditional database processing applications. A saviour to deal with Big Data is “Hadoop” and two major components of Hadoop which are HDFS (Distributed Storage) and Map Reduce(Parallel Processing). Apache Pig and Hive is an essential part of the Hadoop Ecosystem. This paper covers an overview of both Apache Pig and Hive with their architecture. As Hadoop, no doubt is doing tremendously great work by storing and processing the huge volume of data but there are more frameworks now a days to increase the efficiency of Hadoop framework which are basically seen as the layers of Hadoop or a part of Apache Hadoop project. And that is why this paper includes the two most important layers namely Apache Pig and Apache Hive.