scholarly journals Progress in Recommender Systems Research: Crisis? What Crisis?

AI Magazine ◽  
2022 ◽  
Vol 42 (3) ◽  
pp. 43-54
Author(s):  
Paolo Cremonesi ◽  
Dietmar Jannach
Keyword(s):  
Recommender Systems ◽  
Recent Work ◽  
Scientific Method ◽  
Systems Research ◽  
Accuracy Measures

Scholars in algorithmic recommender systems research have developed a largely standardized scientific method, where progress is claimed by showing that a new algorithm outperforms existing ones on or more accuracy measures. In theory, reproducing and thereby verifying such improvements is easy, as it merely involves the execution of the experiment code on the same data. However, as recent work shows, the reported progress is often only virtual, because of a number of issues related to (i) a lack of reproducibility, (ii) technical and theoretical flaws, and (iii) scholarship practices that are strongly prone to researcher biases. As a result, several recent works could show that the latest published algorithms actually do not outperform existing methods when evaluated independently. Despite these issues, we currently see no signs of a crisis, where researchers re-think their scientific method, but rather a situation of stagnation, where researchers continue to focus on the same topics. In this paper, we discuss these issues, analyze their potential underlying reasons, and outline a set of guidelines to ensure progress in recommender systems research.

scholarly journals Evaluating recommender systems for AI-driven biomedical informatics

2020 ◽  
Author(s):  
William La Cava ◽  
Heather Williams ◽  
Weixuan Fu ◽  
Steve Vitale ◽  
Durga Srivatsan ◽  
...  
Keyword(s):  
Septic Shock ◽  
Recommender Systems ◽  
Data Science ◽  
Expert Knowledge ◽  
State Of The Art ◽  
Optimal Algorithm ◽  
Supplementary Information ◽  
Biomedical Data ◽  
Classification Problems ◽  
Systems Research

Abstract Motivation Many researchers with domain expertise are unable to easily apply machine learning (ML) to their bioinformatics data due to a lack of ML and/or coding expertise. Methods that have been proposed thus far to automate ML mostly require programming experience as well as expert knowledge to tune and apply the algorithms correctly. Here, we study a method of automating biomedical data science using a web-based AI platform to recommend model choices and conduct experiments. We have two goals in mind: first, to make it easy to construct sophisticated models of biomedical processes; and second, to provide a fully automated AI agent that can choose and conduct promising experiments for the user, based on the user’s experiments as well as prior knowledge. To validate this framework, we conduct an experiment on 165 classification problems, comparing to state-of-the-art, automated approaches. Finally, we use this tool to develop predictive models of septic shock in critical care patients. Results We find that matrix factorization-based recommendation systems outperform metalearning methods for automating ML. This result mirrors the results of earlier recommender systems research in other domains. The proposed AI is competitive with state-of-the-art automated ML methods in terms of choosing optimal algorithm configurations for datasets. In our application to prediction of septic shock, the AI-driven analysis produces a competent ML model (AUROC 0.85±0.02) that performs on par with state-of-the-art deep learning results for this task, with much less computational effort. Availability and implementation PennAI is available free of charge and open-source. It is distributed under the GNU public license (GPL) version 3. Supplementary information Supplementary data are available at Bioinformatics online.

scholarly journals Recommendation as Collaboration in Web Search

AI Magazine ◽  
2011 ◽  
Vol 32 (3) ◽  
pp. 35-45 ◽  
Author(s):  
Barry Smyth ◽  
Jill Freyne ◽  
Maurice Coyle ◽  
Peter Briggs
Keyword(s):  
Recommender Systems ◽  
Search Engines ◽  
Web Search ◽  
Online Information ◽  
Information Discovery ◽  
Proactive Approach ◽  
Systems Research ◽  
Traditional Approaches ◽  
New Generation

Recommender systems now play an important role in online information discovery, complementing traditional approaches such as search and navigation, with a more proactive approach to discovery that is informed by the users interests and preferences. To date recommender systems have been deployed within a variety of e-commerce domains, covering a range of products such as books, music, movies, and have proven to be a successful way to convert browsers into buyers. Recommendation technologies have a potentially much greater role to play in information discovery however and in this article we consider recent research that takes a fresh look at web search as a fertile platform for recommender systems research as users demand a new generation of search engines that are less susceptible to manipulation and more responsive to searcher needs and preferences.

scholarly journals Escaping the McNamara Fallacy: Towards more Impactful Recommender Systems Research

AI Magazine ◽  
2020 ◽  
Vol 41 (4) ◽  
pp. 79-95
Author(s):  
Dietmar Jannach ◽  
Christine Bauer
Keyword(s):  
Artificial Intelligence ◽  
Recommender Systems ◽  
Real World ◽  
Paradigm Shift ◽  
Prediction Accuracy ◽  
Research Community ◽  
Research Culture ◽  
Research Gaps ◽  
Systems Research ◽  
Research Questions

Recommender systems are among today’s most successful application areas of artificial intelligence. However, in the recommender systems research community, we have fallen prey to a McNamara fallacy to a worrying extent: In the majority of our research efforts, we rely almost exclusively on computational measures such as prediction accuracy, which are easier to make than applying other evaluation methods. However, it remains unclear whether small improvements in terms of such computational measures matter greatly and whether they lead us to better systems in practice. A paradigm shift in terms of our research culture and goals is therefore needed. We can no longer focus exclusively on abstract computational measures but must direct our attention to research questions that are more relevant and have more impact in the real world. In this work, we review the various ways of how recommender systems may create value; how they, positively or negatively, impact consumers, businesses, and the society; and how we can measure the resulting effects. Through our analyses, we identify a number of research gaps and propose ways of broadening and improving our methodology in a way that leads us to more impactful research in our field.

scholarly journals FINN.no Slates Dataset: A new Sequential Dataset Logging Interactions, all Viewed Items and Click Responses/No-Click for Recommender Systems Research

2021 ◽  
Author(s):  
Simen Eide ◽  
David S Leslie ◽  
Arnoldo Frigessi ◽  
Joakim Rishaug ◽  
Helge Jenssen ◽  
...  
Keyword(s):  
Recommender Systems ◽  
Systems Research

scholarly journals Human-Centered Recommender Systems: Origins, Advances, Challenges, and Opportunities

AI Magazine ◽  
2021 ◽  
Vol 42 (3) ◽  
pp. 31-42
Author(s):  
Joseph Konstan ◽  
Loren Terveen
Keyword(s):  
Machine Learning ◽  
Recommender Systems ◽  
Human Performance ◽  
Optimization Techniques ◽  
Research And Practice ◽  
Systems Research ◽  
Challenges And Opportunities ◽  
Human Decision ◽  
Performance Statistics ◽  
System Designs

From the earliest days of the field, Recommender Systems research and practice has struggled to balance and integrate approaches that focus on recommendation as a machine learning or missing-value problem with ones that focus on machine learning as a discovery tool and perhaps persuasion platform. In this article, we review 25 years of recommender systems research from a human-centered perspective, looking at the interface and algorithm studies that advanced our understanding of how system designs can be tailored to users objectives and needs. At the same time, we show how external factors, including commercialization and technology developments, have shaped research on human-centered recommender systems. We show how several unifying frameworks have helped developers and researchers alike incorporate thinking about user experience and human decision-making into their designs. We then review the challenges, and the opportunities, in today’s recommenders, looking at how deep learning and optimization techniques can integrate with both interface designs and human performance statistics to improve recommender effectiveness and usefulness

A collaborative filtering similarity measure based on potential field

Kybernetes ◽  
2016 ◽  
Vol 45 (3) ◽  
pp. 434-445 ◽  
Author(s):  
Yajun Leng ◽  
Qing Lu ◽  
Changyong Liang
Keyword(s):  
Recommender Systems ◽  
Similarity Measure ◽  
Potential Field ◽  
Similarity Measures ◽  
Nearest Neighbors ◽  
Dense Matrix ◽  
Content Type ◽  
Systems Research ◽  
Practical Engineering ◽  
Potential Matrix

Purpose – Collaborative recommender systems play a crucial role in providing personalized services to online consumers. Most online shopping sites and many other applications now use the collaborative recommender systems. The measurement of the similarity plays a fundamental role in collaborative recommender systems. Some of the most well-known similarity measures are: Pearson’s correlation coefficient, cosine similarity and mean squared differences. However, due to data sparsity, accuracy of the above similarity measures decreases, which makes the formation of inaccurate neighborhood, thereby resulting in poor recommendations. The purpose of this paper is to propose a novel similarity measure based on potential field. Design/methodology/approach – The proposed approach constructs a dense matrix: user-user potential matrix, and uses this matrix to compute potential similarities between users. Then the potential similarities are modified based on users’ preliminary neighborhoods, and k users with the highest modified similarity values are selected as the active user’s nearest neighbors. Compared to the rating matrix, the potential matrix is much denser. Thus, the sparsity problem can be efficiently alleviated. The similarity modification scheme considers the number of common neighbors of two users, which can further improve the accuracy of similarity computation. Findings – Experimental results show that the proposed approach is superior to the traditional similarity measures. Originality/value – The research highlights of this paper are as follows: the authors construct a dense matrix: user-user potential matrix, and use this matrix to compute potential similarities between users; the potential similarities are modified based on users’ preliminary neighborhoods, and k users with the highest modified similarity values are selected as the active user’s nearest neighbors; and the proposed approach performs better than the traditional similarity measures. The manuscript will be of particular interests to the scientists interested in recommender systems research as well as to readers interested in solution of related complex practical engineering problems.

The Methodology of Actuarial Science

1999 ◽  
Vol 5 (1) ◽  
pp. 115-195 ◽  
Author(s):  
J.M. Pemberton
Keyword(s):  
United Kingdom ◽  
Recent Work ◽  
Scientific Method ◽  
Empirical Method ◽  
Actuarial Science ◽  
The United Kingdom ◽  
Formal Study ◽  
Key Points ◽  
Actuarial Methods ◽  
Recent Developments

ABSTRACTThis paper considers actuarial science within the context of the framework provided by the formal study of scientific method. A review of key points of recent developments within the methodology (study of method) of science and the methodology of economics is presented. A characterisation of actuarial science and its methods is then developed using as inputs the United Kingdom actuarial education syllabus and recent work of the profession, most notably Bell et al., (1998). The methods of actuarial science are then considered within the framework provided by formal methodology to propose an articulation of the methodology of actuarial science. This methodology is explored in relation to that of other sciences, and some of the implications and opportunities for actuarial science which arise from this investigation are identified. The paper concludes that actuarial science has a distinctive and potentially powerful empirical method of applied approximation. This methodological analysis is intended, in part, to add to the momentum of the programme concerned with furthering the use of actuarial methods within broader spheres (eg Nowell et al., 1996).

scholarly journals A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research

2021 ◽  
Vol 39 (2) ◽  
pp. 1-49
Author(s):  
Maurizio Ferrari Dacrema ◽  
Simone Boglio ◽  
Paolo Cremonesi ◽  
Dietmar Jannach
Keyword(s):  
Recommender Systems ◽  
Systems Research

scholarly journals The effect of visualisation on user experience in recommender systems

2021 ◽  
Vol 26 (3) ◽  
Author(s):  
Wen-Yau Liang ◽  
Chun-Che Huang ◽  
Tzu-Liang Tseng ◽  
Zih-Yan Wang ◽  
◽  
...  
Keyword(s):  
Decision Making ◽  
Recommender Systems ◽  
User Experience ◽  
Recommender System ◽  
User Satisfaction ◽  
Business Environment ◽  
Perceived Usefulness ◽  
Continuance Intention ◽  
Systems Research ◽  
Proposed Model

Introduction. Measuring user experience, though natural in a business environment, is often challenging for recommender systems research. How recommender systems can substantially improve consumers’ decision making is well understood; but the influence of specific design attributes of the recommender system interface on decision making and other outcome measures is far less understood. Method. This study provides the first empirical test of post-acceptance model adaption for information system continuance in the context of recommender systems. Based on the proposed model, two presentation types (with or without using tag cloud) are compared. An experimental design is used and a questionnaire is developed to analyse the data. Analysis. Data were analysed using SPSS and SmartPLS (partial least squares path modeling method). Statistical methods used for the questionnaire on user satisfaction were a reliability analysis, a validity analysis and T-tests. Results. The results demonstrate that the proposed model is supported and that the visual recommender system can indeed significantly enhance user satisfaction and continuance intention. Conclusions. In order to improve the satisfaction or continuance intention of users, it is required to improve the perceived usefulness, effectiveness and visual attractiveness of a recommender system.

Sign in / Sign up

Export Citation Format

Share Document