Rough Set Based Aggregation for Effective Evaluation of Web Search Systems
Rank aggregation is the process of generating a single aggregated ranking for a given set of rankings. In industrial environment, there are many applications where rank aggregation must be applied. Rough set based rank aggregation is a user feedback based technique which mines ranking rules for rank aggregation using rough set theory. In this chapter, the authors discuss rough set based rank aggregation technique in light of Web search evaluation. Since there are many search engines available, which can be used by used by industrial houses to advertise their products, Web search evaluation is essential to decide which search engines to rely on. Here, the authors discuss the limitations of rough set based rank aggregation and present an improved version of the same, which is more suitable for aggregation of different techniques for Web search evaluation. In the improved version, the authors incorporate the confidence of the rules in predicting a class for a given set of data. They validate the mined ranking rules by comparing the predicted user feedback based ranking with the actual user feedback based ranking. They show their experimental results pertaining to the evaluation of seven public search engines using improved version of rough set based aggregation for a set of 37 queries.