Abstract
Average reciprocal hit rank (ARHR) is a commonly used metric for ranking evaluation of top-n recommender systems. However, it suffers from an important shortcoming that it cannot be applied when the user has multiple preferences at a time. In order to overcome this problem, a modified version of ARHR metric is introduced and applied to grocery shopping domain by conducting a series of experiments on real-life data. The results show that the proposed measure is feasible for ranking evaluation of Top-N recommender systems in the cases where the users have multiple preferences at a time or a specific time interval.
Similar content being viewed by others
References
Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: GroupLens: an open architecture for collaborative filtering of netnews. In: Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work, pp. 175–186. ACM (1994)
Hill, W., Stead, L., Rosenstein, M., Furnas, G.: Recommending and evaluating choices in a virtual community of use. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 194–201. ACM Press/Addison-Wesley Publishing Co. (1995)
Deshpande, M., Karypis, G.: Item-based top-N recommendation algorithms. ACM Trans. Inf. Syst. 22, 143–177 (2004)
Cremonesi, P., Koren, Y., Turrin, R.: Performance of recommender algorithms on top-n recommendation tasks. In: Proceedings of the Fourth ACM Conference on Recommender Systems, pp. 39–46. ACM (2010)
Ostuni, V.C., Di Noia, T., Di Sciascio, E., Mirizzi, R.: Top-N recommendations from implicit feedback leveraging linked open data. In: Proceedings of the 7th ACM Conference on Recommender Systems, pp. 85–92. ACM (2013)
Li, W., Matejka, J., Grossman, T., Konstan, J.A., Fitzmaurice, G.: Design and evaluation of a command recommendation system for software applications. ACM Trans. Comput. Hum. Interact. (TOCHI) 18, 6 (2011)
Bobadilla, J., Ortega, F., Hernando, A., Gutiérrez, A.: Recommender systems survey. Knowl. Based Syst. 46, 109–132 (2013)
Moffat, A., Zobel, J.: Rank-biased precision for measurement of retrieval effectiveness. ACM Trans. Inf. Syst. 27, 27 (2009)
Zheng, N., Li, Q.D.: A recommender system based on tag and time information for social tagging systems. Expert Syst. Appl. 38, 4575–4587 (2011)
Ning, X., Karypis, G.: Slim: sparse linear methods for top-n recommender systems. In: 2011 IEEE 11th International Conference on Data Mining (ICDM), pp. 497–506. IEEE (2011)
Cheng, Y., Yin, L.A., Yu, Y.: LorSLIM: low rank sparse linear methods for top-n recommendations. In: 2014 IEEE International Conference on Data Mining (ICDM), pp. 90–99. IEEE (2014)
Kang, Z., Cheng, Q.: Top-N recommendation with novel rank approximation. arXiv preprint (2016). arXiv:1602.07783
Chapelle, O., Metlzer, D., Zhang, Y., Grinspan, P.: Expected reciprocal rank for graded relevance. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 621–630. ACM (2009)
Kekalainen, J.: Binary and graded relevance in IR evaluations - comparison of the effects on ranking of IR systems. Inf. Process. Manage. 41, 1019–1033 (2005)
Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Analysis of recommendation algorithms for e-commerce. In: Proceedings of the 2nd ACM Conference on Electronic Commerce, pp. 158–167. ACM (2000)
Cumby, C., Fano, A., Ghani, R., Krema, M.: Predicting customer shopping lists from point-of-sale purchase data. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 402–409. ACM (2004)
Cumby, C., Fano, A., Ghani, R., Krema, M.: Building intelligent shopping assistants using individual consumer models. In: Proceedings of the 10th International Conference on Intelligent User Interfaces, pp. 323–325. ACM (2005)
Acknowledgements
This work is partially supported by the Scientific and Technological Research Council of Turkey (TUBITAK).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Peker, S., Kocyigit, A. (2016). mRHR: A Modified Reciprocal Hit Rank Metric for Ranking Evaluation of Multiple Preferences in Top-N Recommender Systems. In: Dichev, C., Agre, G. (eds) Artificial Intelligence: Methodology, Systems, and Applications. AIMSA 2016. Lecture Notes in Computer Science(), vol 9883. Springer, Cham. https://doi.org/10.1007/978-3-319-44748-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-44748-3_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44747-6
Online ISBN: 978-3-319-44748-3
eBook Packages: Computer ScienceComputer Science (R0)