A Characterisation and Framework for User-Centric Factors in Evaluation Methods for Recommender Systems

Researchers have worked on-finding e-commerce recommender systems evaluation methods that contribute to an optimal solution. However, existing evaluations methods lack the assessment of user-centric factors such as buying decisions, user experience and user interactions resulting in less than optimum recommender systems. This paper investigates the problem of adequacy of recommender systems evaluation methods in relation to user-centric factors. Published work has revealed limitations of existing evaluation methods in terms of evaluating user satisfaction. This paper characterizes user-centric evaluation factors and then propose a user-centric evaluation conceptual framework to identify and expose a gap within literature. The researchers used an integrative review approach to formulate both the characterization and the conceptual framework for investigation. The results reveal a need to come up with a holistic evaluation framework that combines system-centric and user-centric evaluation methods as well as formulating computational user-centric evaluation methods. The conclusion reached is that, evaluation methods for e-commerce recommender systems lack full assessment of vital factors such as: user interaction, user experience and purchase decisions. A full consideration of these factors during evaluation will give birth to new types of recommender systems that predict user preferences using user decision-making process profiles, and that will enhance user experience and increase revenue in the long run.