Google's search quality evaluation process site may have been around for years.

SearchBistro recently posted a 22 page PDF titled General Guidelines on Random-Query Evaluation that was last revised on December 31, 2003. In addition to posting the Random-Query Evaluation PDF, Henk van Ess has recently posted:

• examples of offensive (or low quality) sites

• some whitelisted sites:
  

  Here is a non-exhaustive "white list" of the sites whose pages are not to be rated as Offensive (nor as Erroneous):
  Kelkoo, Shopping.com, dealtime.com, bizrate.com, bizrate.lycos.com, dooyoo.com;

• tips for rating sites:
  

  If it's a machine-generated, no added value affiliates, it's Spam. If it provides some unique values, for example, customer feedback, local information, it should be rated on the merit scale even if it has some affiliates. Similarly, if the game site allows you to download a game, without being intrusive (i.e. install a spyware without notice), it should be rated on the merit scale, instead of Spam.

• How reviewers communicate to come up with solutions when review quality scores are far apart from one another

Search Classification Types:
The Google review guide classifies searches as being

• navigational (example: a search for United Airlines)

• informational (example: how do I..)
• transactional (example: buy 18K White Gold Omega Watch)
• any mixture of the above categories.

Resource Quality Rating:
Google then asks raters to classify sites listed in random queries using the following categories:

• Vital
  • most queries, especially generic type queries do not have a Vital result.

  • Vital result example: search for Ask Jeeves returns www.ask.com.
• Useful
  • these should have some of the following characteristics (although it likely will not exhibit all of them): comprehensive, quality, answers the search query with precission, timly, authoritative.

  • This is the highest rating attainable for most pages on most search queries.
  • Useful result example: search for USA Patriot Act returning the ACLU page covering the USA Patriot Act.
  • For some plural queries, such as Newspapers in Scotland, the best results may be lists of related sites. Reviewers must also check some links on the page to ensure the page is functional.
• Relevant
  • One step down from Useful. Relevant results may satisfy only one important facet of a query, whereas Useful results are expected to be more broad and thorough.

  • Results that would have been Vital if a more common interpretation did not overshadow it are considered relevant.
• Not Relevant
  • Not Relevant results are related to the topic but do not help users.

  • If a person searching for Real Estate finds a San Diego Real Estate website that would probably not be relevant since most people searching for that do not live in or want to move specifically to San Diego.
  • As the San Diego example is too narrow geographically other sites could also be too narrow in other non location based ways, such as being outdated or too specific to a subset idea of the query.
• Off Topic
  • Is not a useful page. Irrelevant.

  • Usually occurs when text matching algorithms do not account for some terms that can have multiple meanings.
• Offensive
  • Pages or sites that often do not hold merit on any query.

  • Example Offensive sites: spyware, unrequested porn, AdSense scraper and other keyword net type sites, etc.
• Erronious
• Didn't Load
• Foreign Language
• Unrated

Vital to Offensive are in order of quality. The higher the better. Erronious through Unrated are cast as non votes. When in doubt between rating values raters are expected to rate at the lower of the two rating values.

Why this is Important:
By learning how and what they want evaluators to look for it makes it easier to understand how to deliver what the search engines want.

This post was a quick review of General Guidelines on Random-Query Evaluation. If you are heavily interested in SEO it is well worth your time to read the original document, which lists many more examples and is in far greater detail than this post.

Random Thoughts:
With how relatively low the wages are for these positions ($10 - $20 an hour) you have to wonder:

• why it took so long for this information to come out

• if some of these people are using the information they gained from participating in other ways
• if these people know anything about Google's business model, and how much THEY could be making on a per click basis if they created well cited content that fit Google's guidelines.
• and a far off tangent! what would happen if Google's business model made self employment too profitable to where they could not afford to pay workers

Not too long ago there were many reports of French trying to rival the Google print program. The International Herald Tribune reports the same thing is now happening in Germany.

Then this year, when Google started wooing publishers to sign on for its own digital book project, that German executive, Matthias Ulmer, decided the time was ripe to seize control with a homegrown counterattack.

Now Ulmer and a five-member task force of the German book trade association Börsenverein are organizing their own digital indexing project, Volltextsuche Online. The effort of the 6,000-member association of booksellers and publishers comes in reaction to Google's plans, unveiled in December, to start digitizing books in the world, with the first step being major university library collections in the United States.

Ultimately variety is going to be important to keep the free flow of information possible. A few companies controlling the information supply is a scary thought, though it looks as though it is where we are headed.

The scalability of search and requested ad networks requires that anyone jumping into the market either

• creates a strong brand in a niche, or

• jumps in big

It will be hard for Google and others to appease publishers as they try to convince them to allow others to control free copies of their content, which at the least will transform the publishing business model and could eventually undermine large segments of it.

Even if some of the uprising forces have little effect on the outcome being a leader in an outraged group helps market the leaders as being market leaders. An article with "challenges Google" in the title is bound to get syndicated thousands of times.

If you can find some angle where you can go against Google which others find nobel it might be some of the cheapest marketing you ever experience.

Whether or not people view Google's desire to control information as evil it is hard to deny that they are at the forefront of pushing others to modernize data.

So yesterday I posted about Corey Rudl dying, and a couple people sent me anonymous hate messages. I just posted about it on a group post in passing. I figured it would be interesting for some people, but I did not know the guy well. Some people took great offense to my mention, going so far as calling it pathetic.

My Corey Rudl Experience:
A while ago I bought his course because I was going to make a marketing review site. While I am interesting in marketing, I probably am no longer interested in creating such a site. I have not yet read his course, but I did watch the upsell DVD series.

I do find it interesting how attached many people were to him. But I also found the followthrough on some of his techniques less than desirable. For example:

• After buying his course weeks later I got more emails asking me why I had not bought it yet

• His DVD contained information about SEO that was nearly a decade outdated. Others have proved that you can make audio or videos that stay relevant as time passes. His info was generic and dated. He owed it to his customers to give them accurate information.
• Occassionally when people order my ebook the order messes up. I tell them sorry and send them a download link promptly.

  After people copied my ebook and placed it on their site to sell or give away I bought Corey's ebook program to make it harder to steal and redistribute my ebook. I never got a download link. When I told them that I got an email telling me I was all screwed up and I needed to whitelist their email address.

• I acted as though I was interested in buying a mentorship program that I probably did not need just to listen to the sales messages. I never followed through with it, because I ended up finding some of the hard sales techniques offensive.
• When I got an affiliate check and letter the letter said "dear valued affiliate". how hard is it to personalize that? You already have my data if you are writing me a check.
• One of my first SEO customers bought his affiliate software and course and whatnot. He was losing a ton of money. Within a couple months of meeting me that same customer started to make many thousands of dollars profit per month. At that point my personalized service cost less than what my friend considered old school outdated techniques. I was selling a service and not a dream.

Personal Biases:
The above was my own personal experience with what I knew of Corey Rudl. It does not necissarily make him good or bad, just states that my personal experience was not that great. He still got some of my money though, so he must have been doing something right.

I did admire how well recognized he was and how good he was at selling stuff. Often though many of the most successful people that sell courses, services, and products about living a dream grow detatched from reality.

Not saying that his stuff is bad, just that as time passes successful people tend to become detached from reality. I wrote an article about blinding success long before I was in any way successful (which I still don't really consider myself so today in the grand scheme of things). It is one of the things I fear most. In many markets you have to build a reputation before you can compete, but after you build that reputation the things that are true to you are not true to the average person. To me the just make quality content (without mentioning social interaction) advice many SEOs give parallels the problem of people forgetting where people came from.

Before I get any more hate mail for kicking a dead guy, that is not the goal of this post. The goal of this post is to show why my perceptions may not have matched some of my readers, and perhaps why his death did not mean as much to me as it did to some of the readers of this site.

Selling a Dream:
When people learn new subjects they need somewhere to start. Even if a large amount of the things I say are without tact or in some way wrong, hopefully I am generally helping people.

I think think the same way with Corey, but on a much larger scale. I am sure thousands of people are doing well due in large part to Corey's guidance.

With a decade of experience and front runner marketing techniques it makes sense that many people are attached to Corey and sad to see him go. The guy did die doing what he loved though, and most people who live to 100 do not expereince as much enjoyment as he probably did in his short life. Perhaps that is another reason I was not as sad as some people were about the issue. Most people never get to do what they really want to. At a young age he already did.

Sorry:
Was I wrong to post about Corey dying? I don't think so. Could it have been done with more compassion? Yes.

On another site I covered the topic more in depth, but I did not think that most readers of this blog would want in depth coverage.

When you get lots of readers and do not get lots of feedback it is easy to do things that offend some of them. If I offended you sorry. Here is a good thread about Corey if you want to post.

the search engine gaming ad lawsuit continues...

major search engine websites lost their motion to strike core allegations in plaintiffs' complaint in the case of Cisneros et al v. Yahoo! et al (San Francisco County Superior Court). These allegations ask the Court to provide a remedy against the search engine's alleged illegal advertisement of Internet gambling. The Court made a ruling that allows the case to move forward.

If you are interested in buying an on topic online casino ad on this page please send me an email.

Jakob Nielsen's Alertbox celebrates 10 years. An interesting excerpt:

In 1997, I wrote the "Do Websites Have Increasing Returns?" column, discussing the relative value of big and small websites. I predicted that small sites would generate 75% of the Web's total value because they can be more targeted than big sites. ...

Most current discussions of the long tail underestimate the non-hits and assume that each point on the curve has the same value. But on the Web, being small means that you can better target your content and thus provide higher value per unit than more generic services.

Jakob also mentions a few of his hits and misses and points out some of his Alertbox articles which he feels deserve far more traffic than they get.

I have a bit of a hard time balancing self worth and ego, but it is interesting to think that hundreds of thousands of people could read your not well received work and still view it as deserving of more attention.

It seems to me that whether a person calls something usability, long tail, conversion marketing, SEO, story telling, brand building, or whatever, the end goal is to create enough value to extract profits while serving customers needs.

Usually most of the tips and information can be generic in nature (ex: track results or increase usability) - or deeply specific using some random vocabulary set (ex: Use location based keyword modifiers and bid for third position on Overture for expensive terms. Daypart your bid prices or ad display times to match the optimal point on the profit elasticity curve.) - because most of these terms and ideas are geared toward creating websites or systems which specifically target the needs of a small group of people. It is easy to fill the needs and desires of a small group of people.

It also makes me wonder if I should broaden or shift some of my interests (and more importantly, the way I market them) to a label other than SEO.

When in the UK I asked a ton of questions about demographics, law enforcement, power generation, social services, transportation, and the like. I find it fascinating to watch how some systems scale. It will be amazing if / when people figure out something that can beat out AdWords. MSN's new product may offer more data, but it may be confusing and just a bit ahead of its time.