Nov 21 2011
People hear about the algorithm so much they don’t realize the other side of Google’s ranking matrix – human raters.
A few years ago, conversion optimizers wondered whether their work to improve a landing page’s conversion rate also influenced search rank. Well, today it most likely does because of human raters.
Human raters play a vital role in Google’s quest to provide relevant and quality search results. Human raters test proposed changes to the algorithm before it is rolled out and it seems likely that they rate web pages already in the live index as well. Live user behavior with search results is also monitored.
As you can imagine, Google’s human quality raters are most concerned with understanding search user intent. For example, when a user enters a search term like “jersey” do they mean “New Jersey,” “NFL Jersey,” “Jersey Shore,” or the open source software? Human raters help determine the dominant interpretation of a particular search query. Human raters also help interpret:
- Action intent
- Informational intent
- Navigational intent
A searcher who is in action mode is trying to achieve a goal – it might be to buy jewelry for their mother, to join a parenting support group, to watch a video or to download music for their iphone. So users are trying to find a web page that will help them take action to achieve that goal.
An informational searcher is seeking knowledge from an authoritative source, such as how to unclog a kitchen drain, identify the author of a book or learn about the behavior of honeybees. Users are trying to find information from a credible source.
A navigational searcher wants to find a website or a specific web page, such as Facebook or gmail. Instead of using the address bar in their browser, they do a search for it. Users are trying to find an exact website.
Human raters rate search results based on user intent and the utility of the landing page. Here are some of the things they might say in their ratings for search results that do not qualify as the dominant interpretation.
This site/page would be helpful for most users, because:
- The page is on an authoritative website that answers this question very well
- This Italian restaurant has hundreds of reviews
- This is a reputable site for completing this type of transaction
- There are many low quality lyric pages on the web but we have confidence in the accuracy of these lyrics because they are on the artist’s official website
It’s unclear what percentage of search results human raters rate, but it is clear that they actually visit the web page in question and have clear guidelines on how to rate web pages for specific types of search queries.
In addition to assigning a rating to a web page, human raters also flag web pages for various things, such as spam. If a human rater believes a page has been designed with deceptive techniques, it will receive a spam flag.
In case you thought you could treat search engines like a machine, this should be a wake up call to begin viewing them as living organisms in which a set of human eyes examines web pages for relevance to specific search queries. Google’s human raters might influence the pages that rank for specific search terms by evaluating its relevance to user intent and rating them accordingly.