Big Data Testing Framework for Recommendation Systems in e-Science and e-Commerce Domains

2021 ◽  
Author(s):  
Meryem Uzun-Per ◽  
Ali Burak Can ◽  
Ahmet Volkan Gurel ◽  
Mehmet S. Aktas
Keyword(s):  
Big Data ◽  
Recommendation Systems ◽  
Testing Framework

scholarly journals Towards an Automated Testing Framework for Big Data

2019 ◽  
Vol 9 (1S3) ◽  
pp. 479-485
Keyword(s):  
Big Data ◽  
Performance Testing ◽  
Data Migration ◽  
Functional Testing ◽  
Security Testing ◽  
Testing Framework ◽  
Strategy Map ◽  
Data Ecosystem ◽  
Testing Module ◽  
Migration Testing

Big data testing services are to deliver end to end testing methodologies which address our big data challenges. The testing module includes two types of functionalities. One is functional testing and second is non- functional testing. The functional testing should be accomplished at every stage of big data processing. Functional testing is nothing but the big data sources extraction testing, data migration testing and big data ecosystem. Testing which completes ETL test strategy, Map job reduce validation, multicore Data integration validation and data duplication check. On the other side the non-functional testing is to ensure that there are no quality defeat in data and no performance related issues. It covers the area for security testing, performance testing which solve the problem of monitoring and identify bottlenecks.

scholarly journals An Optimal Enhancement of the Dynamic Features of Recommender Systems

2019 ◽  
Vol 8 (2S4) ◽  
pp. 51-55
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Recommender Systems ◽  
Data Analytics ◽  
Recommendation Systems ◽  
Big Data Analytics ◽  
The Internet ◽  
User Requirements ◽  
Dynamic Features ◽  
Dynamic Nature

Recommendation systems come under the domain of Data mining and Big Data analytics. It is useful tool that is used to predict the ratings or preferences of a user from a pool of resources. The preferences of user are dynamic in nature. The immeasurable usage of internet is having a great impact on the way we deal our lives and communicate with each other. As a result, the requirements of user browsing the internet are changing radically. Recommender Systems (RSs) provide a technology that helps users in finding relevant or preferential information among the pool of information using internet. This paper puts forward not only the issues related to the dynamic nature of user’ requirements but also the changes in the systems’ contents. The Recommendation Systems which involves the above stated issues are termed as Dynamic Recommender Systems (DRSs). This paper first defines the concept of DRS and then explores the various parameters that is taken into account in developing a DRS. This paper also discusses the scope of contributions in this field and concludes citing in possible extensions that can improve the dynamic qualities of recommendation systems in future.

Big Data Mining Using Collaborative Filtering

Web Services ◽  
2019 ◽  
pp. 702-711
Author(s):  
Anu Saini
Keyword(s):  
Data Mining ◽  
Big Data ◽  
Collaborative Filtering ◽  
Recommendation System ◽  
Recommendation Systems ◽  
Big Data Mining ◽  
Detailed Survey ◽  
Past Behavior

Today every big company, like Google, Flipkart, Yahoo, Amazon etc., is dealing with the Big Data. This big data can be used to predict the recommendation for the user on the basis of their past behavior. Recommendation systems are used to provide the recommendation to the users. The author presents an overview of various types of recommendation systems and how these systems give recommendation by using various approaches of Collaborative Filtering. Various research works that employ collaborative filtering for recommendations systems are reviewed and classified by the authors. Finally, this chapter focuses on the framework of recommendation system of big data along with the detailed survey on the use of the Big Data mining in collaborative filtering.

scholarly journals Scalable Recommendation Using Large Scale Graph Partitioning With Pregel and Giraph

2020 ◽  
Vol 14 (4) ◽  
pp. 42-61
Author(s):  
Gourav Bathla ◽  
Himanshu Aggarwal ◽  
Rinkle Rani
Keyword(s):  
Big Data ◽  
Social Networking Sites ◽  
Large Scale ◽  
Information Overload ◽  
Social Trust ◽  
Recommendation Systems ◽  
Large Scale Data ◽  
Social Big Data ◽  
The Social ◽  
Recommendation Accuracy

Social Big Data is generated by interactions of connected users on social network. Sharing of opinions and contents amongst users, reviews of users for products, result in social Big Data. If any user intends to select products such as movies, books, etc., from e-commerce sites or view any topic or opinion on social networking sites, there are a lot of options and these options result in information overload. Social recommendation systems assist users to make better selection as per their likings. Recent research works have improved recommendation systems by using matrix factorization, social regularization or social trust inference. Furthermore, these improved systems are able to alleviate cold start and sparsity, but not efficient for scalability. The main focus of this article is to improve scalability in terms of locality and throughput and provides better recommendations to users with large-scale data in less response time. In this article, the social big graph is partitioned and distributed on different nodes based on Pregel and Giraph. In the proposed approach ScaleRec, partitioning is based on direct as well as indirect trust between users and comparison with state-of-the-art approaches proves that statistically better partitioning quality is achieved using proposed approach. In ScaleRec, hyperedge and transitive closure are used to enhance social trust amongst users. Experiment analysis on standard datasets such as Epinions and LiveJournal proves that better locality and recommendation accuracy is achieved by using ScaleRec.

scholarly journals Design of Personalized Recommendation Algorithms based on Big Data Framework

2021 ◽  
Vol 2138 (1) ◽  
pp. 012025
Author(s):  
Fang Liu
Keyword(s):  
Big Data ◽  
Information Overload ◽  
Recommendation Systems ◽  
Recall Rate ◽  
Personalized Recommendation ◽  
Processing Technologies ◽  
Data Framework ◽  
Recommendation Algorithms ◽  
Data Environment ◽  
Better Than

Abstract The issue of information overload has become increasingly prominent since there are various kinds of data generated daily. A good recommendation systems can better deal with such problems. However, traditional recommendation systems for a single machine are suffering from the computing bottleneck in the environment of massive data. An individual recommendation algorithm is unable to gratify desiring users. To tackle this problem, we designed and implemented three kinds of recommendation algorithms based on big data framework in this paper. Besides, we improved the traditional recommendation algorithms leveraging the prevailing big data processing technologies. Finally, we evaluated the efficiency of the algorithm through recall rate, precision rate and coverage. Experiments show that the hybrid model-based recommendation algorithms which can be applied to the bulk data environment are better than the single recommendation algorithms.

scholarly journals Recommendation Systems in the Big Data Era

2020 ◽  
Vol 8 (12s3) ◽  
pp. 80-85
Keyword(s):  
Social Media ◽  
Big Data ◽  
Recommendation System ◽  
Recommendation Systems ◽  
Rapid Progression ◽  
Specific Product ◽  
Hybrid Approaches ◽  
Social Media Platforms ◽  
Use Of Social Media ◽  
The Way

Rapid progression in technology and increasing use of social media platforms like Facebook, Instagram and Twitter has altered the way of articulating people’s judgment, observation and sentiments about specific product, services, and more. This leads to the production and accumulation of massive amount of data. Recommendation systems are getting impetus when it comes to find insights from this data to make decisions that can be represented in various statistical and graphical forms. They have proven useful in predicting or recommending products ranging from food, movies, restaurants etc. This paper presents an overview about recommendation systems and a review of generation of recommendation methods based on categories like contentbased, collaborative, and hybrid approaches. The paper will enlist the limitations which the present recommendation system faces and the possible improvements required in their capabilities to fit into a wider range of application areas.

scholarly journals The Recommended Online Social Networks (OSN) and Big Data System Using Large Scale Graph Partitioning with Mahout and PowerGraph

2020 ◽  
Author(s):  
Ford Lumban Gaol ◽  
Tokuro Matsuo
Keyword(s):  
Big Data ◽  
Online Social Networks ◽  
Social Networking Sites ◽  
Large Scale ◽  
Information Overload ◽  
Social Trust ◽  
Recommendation Systems ◽  
Large Scale Data ◽  
Social Big Data ◽  
Recommendation Accuracy

Abstract Introduction : Social Big data is generated by interactions of connected users on social network. Sharing of opinions and contents amongst users, reviews of users for products, result in Social Big data. If any user intends to select product such as movies, books etc. from e-commerce sites or view any topic or opinion on social networking sites, there are a lot of options and these options result in information overload. Case Description : Social recommendation systems assist users to make better selection as per their likings. Recent research works have improved recommendation systems by using matrix factorization, social regularization or social trust inference. Furthermore, these improved systems are able to alleviate cold start and sparsity but not efficient for scalability. Discussion and Evaluation: The main focus of this paper is to improve scalability and provide better recommendations to users with large-scale data in less response time. We have partitioned social big graph and distributed it on different nodes based on Mahout and PowerGraph like system. Conclusion : In our approach, partitioning is based on direct as well as indirect trust between users and comparison with state-of-the-art approaches proves that statistically better partitioning quality is achieved using our approach. In our proposed approach ScaleRec, hyperedge and transitive closure are used to enhance social trust amongst users. Experiment analysis on standard datasets proves that better locality and recommendation accuracy is achieved by using our proposed approach.

Sign in / Sign up

Export Citation Format

Share Document