People hear about the algorithm so much they don’t 
realize the other side of Google’s ranking matrix – human raters.
A few years ago, conversion optimizers wondered whether their work to improve a landing page’s conversion rate also influenced search rank. Well, today it most likely does because of human raters.
Human raters play a vital role in Google’s quest to provide relevant and quality search results. Human raters test proposed changes to the algorithm before it is rolled out and it seems likely that they rate web pages already in the live index as well. Live user behavior with search results is also monitored.
As you can imagine, Google’s human quality raters are most concerned with understanding search user intent. For example, when a user enters a search term like “jersey” do they mean “New Jersey,” “NFL Jersey,” “Jersey Shore,” or the open source software? Human raters help determine the dominant interpretation of a particular search query. Human raters also help interpret:
- Action intent
- Informational intent
- Navigational intent
A searcher who is in action mode is trying to achieve a goal – it might be to buy jewelry for their mother, to join a parenting support group, to watch a video or to download music for their iphone. So users are trying to find a web page that will help them take action to achieve that goal.
An informational searcher is seeking knowledge from an authoritative source, such as how to unclog a kitchen drain, identify the author of a book or learn about the behavior of honeybees. Users are trying to find information from a credible source.
A navigational searcher wants to find a website or a specific web page, such as Facebook or gmail. Instead of using the address bar in their browser, they do a search for it. Users are trying to find an exact website.
Human raters rate search results based on user intent and the utility of the landing page. Here are some of the things they might say in their ratings for search results that do not qualify as the dominant interpretation.
This site/page would be helpful for most users, because:
- The page is on an authoritative website that answers this question very well
- This Italian restaurant has hundreds of reviews
- This is a reputable site for completing this type of transaction
- There are many low quality lyric pages on the web but we have confidence in the accuracy of these lyrics because they are on the artist’s official website
It’s unclear what percentage of search results human raters rate, but it is clear that they actually visit the web page in question and have clear guidelines on how to rate web pages for specific types of search queries.
In addition to assigning a rating to a web page, human raters also flag web pages for various things, such as spam. If a human rater believes a page has been designed with deceptive techniques, it will receive a spam flag.
In case you thought you could treat search engines like a machine, this should be a wake up call to begin viewing them as living organisms in which a set of human eyes examines web pages for relevance to specific search queries. Google’s human raters might influence the pages that rank for specific search terms by evaluating its relevance to user intent and rating them accordingly.
7 replies to "Google’s Quality Raters Are Looking At Your Site"
Hey Tom, this makes me wonder a few things….don’t expect you to know the answers to these….just thinking out loud here.
1- how do the human raters get assigned websites to rate? If 5% of sites are rated, I wonder how they are selected and what might make you more likely to get audited. And if there were factors that made you more likely for an audit, what steps could you take to avoid (or encourage) an audit?
2- I’ve heard many people worried about the negative impacts, but I’m excited about the positive impacts. I wonder if this could actually help bolster your rankings. Especially, if you website looks really cool to the human eye, but maybe doesn’t fair as well when viewed by machines.
3- What if you should be ranking for certain keywords logically, but perhaps are not because they don’t appear on your site. Will the human eye help? I actually think this is very common as many people write without SEO in mind and they may not spell out certain keywords that might actually best describe the content.
4- How will this affect backlinks? If you have some low quality backlinks, will they go lower? Or is their risk that google penalize you for trying to build backlinks that human raters deem less than awesome?
Just my thoughts! -Geoff
Hi Geoff, you have some great questions. I’m not sure if an audit is a prerequisite for getting a human rater to view your site. My guess is if your site ranks in the top 10 for a competitive search term, it will probably be reviewed by a human rater.
I agree, if human raters think a site’s esthetics make it more relevant for a particular search term, it could make a difference in search rank.
If keywords don’t appear on your site, can you still rank for them due to human raters? Maybe so, especially if the domainant interpretation is a brand name or a navigational search.
My guess is human raters deal with certain types of search queries that they are highly qualified to rate for user intent. I’m not sure if they deal with backlinks so much, but they can certainly get Google’s Webspam team involved by flagging a page for spam.
What an interesting revolution–I just tweeted to my fellow Tweeters. I also wonder about the linking since people link to your site that you can’t always control. I know I get quite a few goofy links that try to link to my blog. Everyone feels that links are valuable and tries to get them from the best sources possible. I would hate to see something happen with this live person review that is similar to googles recent “Proximity Lockout” regarding city centers–this has hurt many professional businesses that are close to the central city but are not actually downtown.
Hi G, I wouldn’t worry about links from places you can’t control, that won’t hurt your rankings; otherwise, you could simply point some poor quality links at your competitor’s site and take them down.
If you believe Google is trying to do what is best for users, you have to view human raters as a good thing, but it’s not clear whether their ratings affect search rankings yet.
Some marketers impacted by proximity lockout have found in one case that helping Google Places get accurate info also helped them get better placement in the maps listings. I’m not saying you’ll get better rankings by helping Google verify data in your local area, but users will want that and it can only help the legit businesses in the area.
If you believe Google is trying to do what is best for users, you have to view human raters as a good thing, but it’s not clear whether their ratings affect search rankings yet.Thanks a lot for sharing this..
I hear so many different theories on Google rankings. I was considering using Backlinks Phillipines for paid backlinks, is this something I should consider.
Hi Johnny, first ask yourself this question about everything you are doing to increase your site’s organic search rank: how would I explain what I am doing to Google’s Web Spam Team if they saw it?
Second, Backlinks Phillipines seems to be involved in setting up social media profiles that include a link to your site. Not only is this bad for the web and probably violates the TOS of these social media sites, but Google tends to count these links for a short period of time and then discounts them. So, people who do this are on a perpetual rollercoaster ride – as soon as they get off they lose their rankings.
There far better ways to build links that Google will accept and count. Check out my Link Building Strategy Brief.