The Faults of Human Review
Danny Sullivan's recently made a post highlighting the downside of human review for search engines:
[Tim] Mayer reminded that what's relevant for a query can often change over time. Google's Udi Manber, vice president of engineering, made similar remarks when I spoke with him about human-crafted results when I was visiting at Google yesterday.
One example he pointed out was how Google's human quality reviewers -- people that Google pays to provide a human double-check on the quality of its results, so they can then better tune the search algorithm -- started to downgrade results for [cars] when information about the movie Cars started turning up. The algorithm had picked up that the movie was important to that term before some of the human reviewers were aware of it.
Obviously human review is used at all major search engines, but even when outsourcing reviews humans have limits just like with producing content. Even if Google has 10,000 quality raters those people can only be trained to find and rate certain things.
Comments
Google needs to get some humans onto the serps for "convert mov to flv".
One of the most appalling set of results I have seen for a while. Basically the top 10 results are a minefield of banners, Adsense and thin content.
you have a huge collection of information buddy, good work.
Take a look on my blog for Celebrity Gossips and News:http://www.thecelebrityblogs.blogspot.com
I think the whole idea of having people manually populate search results is absurd. It's like taking a step backwards in time.
hey aaron do you still have the first version of the back link analyzer please? for some reason i am more comfortable with it then the newer version? Thanx
The faults of any human reviewing anything are always going to be there, differing from person to person. The fault of your SEObook keyword tool is that it uses an American spell checker. In Australia we spell certain words differently. Checking the differences in spelling using Google Trends and the differences in the regions of the US and Aus you can clearly see that we do spell certain words differently than yourself. Your keyword tool did not recognise my spelling and showed me the American version which is completely irrelevant to my cause.
Humans may have faults, but every piece of software and code has been created by humans. The only way there could be no fault at all is if every person on the planet added their mind to the code. But then, I fear, we would have a serious problem where everything would be cancelled out by everything else. The Ying/Yang theory.
Everything can be debated by someone. Peoples beliefs are too narrow minded these days.
Down with advertising websites! There are more of them on the net now than there is actual content.
Adam: "I think the whole idea of having people manually populate search results is absurd. It's like taking a step backwards in time."
Tell that to Jason who just started Mahalo.com
On the subject of human edit results, isn't MAHALO just the lousiest internet business plan since the bubble? Have investors lost their marbles again? I checked it out for some relevant searches to my area and the results were all same and all big branded multinationals, rather than the specialists. Come on - is Walmart really the best place I can buy flowers? And there's no local context - I'm in the UK! They haven't even considered the scope of the information they need to provide. Are they going to research florists in every town and city across the US or even the world. And don't get me started on timeliness. How up-to-date will those results be? Worst still, the whole plan works on a mass market appeal, when most searches will never hear of Mahalo. Forget humna edited, probably Google's CSE's are the way forward for more relevancy. Because, you have to tackle relevancy a niche at a time!
I don't have much of a problem with human edited search as long as it remains relevant. One could say that sites like delicious or stumbleupon is a human edited search engine. Both are user recommended content with plenty of relevancy for any given topic.
Add new comment