An experimental comparison of click position-bias models

N Craswell, O Zoeter, M Taylor, B Ramsey - Proceedings of the 2008 …, 2008 - dl.acm.org
N Craswell, O Zoeter, M Taylor, B Ramsey
Proceedings of the 2008 international conference on web search and data mining, 2008dl.acm.org
Search engine click logs provide an invaluable source of relevance information, but this
information is biased. A key source of bias is presentation order: the probability of click is
influenced by a document's position in the results page. This paper focuses on explaining
that bias, modelling how probability of click depends on position. We propose four simple
hypotheses about how position bias might arise. We carry out a large data-gathering effort,
where we perturb the ranking of a major search engine, to see how clicks are affected. We …
Search engine click logs provide an invaluable source of relevance information, but this information is biased. A key source of bias is presentation order: the probability of click is influenced by a document's position in the results page. This paper focuses on explaining that bias, modelling how probability of click depends on position. We propose four simple hypotheses about how position bias might arise. We carry out a large data-gathering effort, where we perturb the ranking of a major search engine, to see how clicks are affected. We then explore which of the four hypotheses best explains the real-world position effects, and compare these to a simple logistic regression model. The data are not well explained by simple position models, where some users click indiscriminately on rank 1 or there is a simple decay of attention over ranks. A 'cascade' model, where users view results from top to bottom and leave as soon as they see a worthwhile document, is our best explanation for position bias in early ranks
ACM Digital Library