There’s an awful lot of attention is being given to users and click data at the moment (despite the fact that analysing query logs dates back to 10 years ago). “Tailoring Click Models to User Goals” by Guo, Li, Faloutsos (Carnegie University) is from the 2009 workshop on Web Search Click Data so it’s considered important enough that a reasonable number of researchers are dedicating themselves to the subject. Those in the business of online marketing are also interested in these metrics because they can give valuable information on what kind of keywords draw in users and behavioural analysis can help further optimise campaigns (in short).
What’s a click model:
“Click models provide a principled way of understanding user interaction with web search results in a query session. They usually incorporate user behavior assumptions, such as the examination hypothesis and the cascade hypothesis, to specify how examination and clicks at different positions depend on each other.”
Their question:
“Usually the average user behavior pattern is summarized in a small set of global parameters. Can we fit multiple models with different user behavior parameters on a click data set?”
Method:
They tailored click models for user goals (in web search), using query term classification. They fitted 2 click models for navigational (user is looking for a particular website) and informational (user is looking for particular information) queries. Both types of query dislay different user behaviour. Their achieved a better prediction than other methods. They also propose an evaluation metric called “Search relevance score” (SRS) which gives statistics to assess performance. It’s calculated using the expected examined document relevance.
User behaviour:
Users’ click behaviour depends on a multitude of factors, not simple eye-tracking, or information found in logs, even document relevance alone cannot give accurate answers as to what a user’s goal is and what makes them click on particular results.
Navigational and informational queries:
They are identified based on the number of clicks on each position and they’re sorted in descending order to form a click vector. This allows them to visualise the click distribution over positions. They also record the time elapsed before the click happens.
Findings:
They found that navigational queries have higher scores than informational queries. Navigational queries perform better than informational ones. They think that search engines might tweak their rankings for some head queries to place some navigational queries on top. The more popular terms are for informational queries the more the SRS curve drops. They think that maybe there are biases introduced by query re-submission or the fraction of no-click sessions that were ignored.
Conclusion:
They showed that by tailoring click models to user goals, better performance is achieved.
“A hidden assumption in the derivation of SRS…is that document impression, the identity of all documents in the search result, is already known. And examination probabilities are computed a priori without knowledge of the click events. This represents the average user behavior under the particular click model. Therefore, if we want to derive SRS for a particular user…”
Why should you care?
Being able to predict user behaviour based a variety of different factors is of benefit to everyone, marketers and researchers alike. Understanding the user should really be our priority as without this understanding we are unable to design precise and efficient systems, and also unable to efficiently optimise websites for usability and seo. If most users come to a site on a navigational query and we know what variables are involved in this process, we can assess what kind of site visitors are expecting. We may well provide a lot of information, like on this blog, and then notice that people are coming through to get somewhere else, maybe another page of the site even. This calls for a modification in design for example amongst other things.
