A Thought Experiment on Google Whitelisting Websites

Google has long maintained that "the algorithm" is what controls rankings, except for sites which are manually demoted for spamming, getting hacked, delivering spyware, and so on.

At the SMX conference it was revealed that Google uses white listing:

Google and Bing admitted publicly to having ‘exception lists’ for sites that were hit by algorithms that should not have been hit. Matt Cutts explained that there is no global whitelist but for some algorithms that have a negative impact on a site in Google’s search results, Google may make an exception for individual sites.

The idea that "sites rank where they deserve, with the exception of spammers" has long been pushed to help indemnify Google from potential anti-competitive behavior. Google's marketing has further leveraged the phrase "unique democratic nature of the web" to highlight how PageRank originally worked.

But why don't we conduct a thought experiment for the purpose of thinking through the differences between how Google behaves and how Google doesn't want to be perceived as behaving.

Let's cover the negative view first. The negative view is that either Google has a competing product or a Google engineer dislikes you and goes out of his way to torch your stuff simply because you are you and he dislikes you & is holding onto a grudge. Given Google's current monopoly-level marketshare in most countries, such would be seen as unacceptable if Google was just picking winners and losers based on their business interests.

The positive view is that "the algorithm handles almost everything, except some edge cases of spam." Let's break down that positive view a bit.

  • Off the start, consider that Google engineers write the algorithms with set goals and objectives in mind.
    • Google only launched universal search after Google bought Youtube. Coincidence? Not likely. If Google had rolled out universal search before buying Youtube then they likely would have increased the price of Youtube by 30% to 50%.
    • Likewise, Google trains some of their algorithms with human raters. Google seeds certain questions & desired goals in the minds of raters & then uses their input to help craft an algorithm that matches their goals. (This is like me telling you I can't say the number 3, but I can ask you to add 1 and 2 then repeat whatever you say :D)
  • At some point Google rolls out a brand-filter (or other arbitrary algorithm) which allows certain favored sites to rank based on criteria that other sites simply can not match. It allows some sites to rank with junk doorway pages while demoting other websites.
  • To try to compete with that, some sites are forced to either live in obscurity & consistently shed marketshare in their market, or be aggressive and operate outside the guidelines (at least in spirit, if not in a technical basis).
  • If the site operates outside the guidelines there is potential that they can go unpenalized, get a short-term slap on the wrist, or get a long-term hand issued penalty that can literally last for up to 3 years!
  • Now here is where it gets interesting...
    • Google can roll out an automated algorithm that is overly punitive and has a significant number of false positives.
    • Then Google can follow up by allowing nepotistic businesses & those that fit certain criteria to quickly rank again via whitelisting.
    • Sites which might be doing the same things as the whitelisted sites might be crushed for doing the exact same thing & upon review get a cold shoulder.

You can see that even though it is claimed "TheAlgorithm" handles almost everything, they can easily interject their personal biases to decide who ranks and who does not. "TheAlgorithm" is first and foremost a legal shield. Beyond that it is a marketing tool. Relevancy is likely third in line in terms of importance (how else could one explain the content farm issue getting so out of hand for so many years before Google did something about it).

Doorway Pages Ranking in Google in 2011?

When Google did the Panda update they highlighted that not only did some "low quality" sites get hammered, but that some "high quality" sites got a boost. Matt Cutts said: "we actually came up with a classifier to say, okay, IRS or Wikipedia or New York Times is over on this side, and the low-quality sites are over on this side."

Here is the problem with that sort of classification system: doorway pages.

The following Ikea page was ranking page 1 in the search results for a fairly competitive keyword.

Once you strip away the site's navigation there are literally only 20 words on that page. And the main body area "content" for that page is a link to a bizarre, confusing, and poor-functioning flash tour which takes a while to load.

If you were trying to design the worst possible user experience & wanted to push the "minimum viable product" page into the search results then you really couldn't possibly do much worse that that Ikea page is (at least not without delivering malware and such).

I am not accusing Ikea of doing anything spammy. They just have terrible usability on that page. Their backlinks to that page are few in number & look just about as organic as they could possibly come. But not that long ago companies like JC Penny and Overstock were demoted by Google for building targeted deep links (that they needed in order to rank, but were allegedly harming search relevancy & Google user experience). Less than a month later Google arbitrarily changed their algorithm to where other branded sites simply didn't need many (or in some cases any) deep links to get in the game, even if their pages were pure crap. Google Handling Flash.

We are told the recent "content farm" update was to demote low quality content. If that is the case, then how does a skeleton of a page like that rank so high? How did that Ikea page go from ranking on the third page of Google's results to the first one? I think Google's classifier is flashing a new set of exploits for those who know what to look for.

A basic tip? If you see Google ranking an information-less page like that on a site you own, that might be a green light to see how far you can run with it. Give GoogleBot the "quality content" it seeks. Opportunity abound!

Google Penalized BeatThatQuote.com

Shortly after we found out that Google was to purchase Beat That Quote we highlighted how Google purchased a website that was deeply engaged in numerous nefarious black hat SEO practices. A friend just pinged me to confirm that Google has penalized the domain by removing it from the search results.

From a competition & market regulation perspective that was a smart move for Google. They couldn't leave it in the search results while justifying handing out penalties to any of its competitors. As an added bonus, the site is building up tons of authoritative links in the background from all the buzz about being bought by Google. Thus when Google allows it to rank again in 30 days it will rank better than ever.

Based on their web dominance which generates such a widespread media buzz, Google adds millions of Pounds worth of inbound links to any website they buy.

The message Google sends to the market with this purchase is that you should push to get the attention of Google's hungry biz dev folks before you get scrutiny from their search quality team. After the payday the penalty is irrelevant because you already have cash in hand & the added links from the press mentioning the transaction will more than offset any spam links you remove. Month 1 revenues might be slightly lower, but months 2 through x will be far higher.

Google Buys BeatThatQuote, a UK Comparison Site Violating Google's Guidelines

It looks like Money.co.uk was right on the money:

BeatThatQuote.com today was sold to Google for GBP37.7 million. We think this deal is a tremendous opportunity for our company to develop new and innovative options for personal finance in the UK. Our team is excited about becoming a part of Google. We look forward to working with their engineers to create new tools making it easier for consumers to choose the right financial products. We think

What is screwed up about this is that Google is engaging in *major* channel conflict. Not only is there some gray area background stuff:

BeatThatQuote.com's ad prompted 101 complaints to the Advertising Standards Authority, with 65 objecting that the commercial "trivialised, condoned or encouraged bullying in the workplace".

But now they have to consider SEO as well. I highlighted how it was a bit unjust when Google arbitrarily chose to whack one site while letting another get away with worse just because the founder was good at public relations, but how can Google police Google's guidelines when Google is the one breaking them?

Doorway Pages / Gateway Sites

Remember how Overstock.com was recently penalized for offering discounts in exchange for links? BeatThatQuote partnered with Oxfam to create CompareForGood.com. The homepage consists of a bunch of links into BeatThatQuote.com. If you look at those links using our server header checker you will see some 301 redirects. Of course doorway pages are considered spam & we know that Google has torched some other affiliate programs for using 301 redirects.

Such links are doing 301 redirects

Sure that is fairly vanilla...a bit gray area. Certainly not straight up black hat.

....unlike...

Buying Links That Pass PageRank

Raise your hand if you have read a post by Matt Cutts on the dangers of buying links that flow PageRank?

Ok. Now keep your hand up if you have read a dozen of them.

Remember how Matt Cutts stated that some bloggers got torched for selling a single link? Remember the long NYT article about how Google is cracking down on link buying & they penalized JC Penny for it?

With that in mind, can anyone explain why Google's newest purchase buying links like

Not so much a categorized listing with an editorial review...just a paid link for the sake of buying links to flow PageRank.

That one is only totally flagrant.

A bit off color. Like comment spamming.

Sorta like the link exchanges in German.

But some are even more outrageous. Consider that BeatThatQuote is buying links from pages with ad sections like

Paid Blog Reviews

Remember those "evil" paid reviews Matt Cutts wrote of? Plenty of those to go around ;)

In fact, some of the paid blog links were in place so long that BeatThatQuote got thank you's for advertising for over a year straight.

I don't have a decade of spam fighting experience like Google does. But is it too much to suggest that before Google buys *any* website they should do a basic compliance audit to verify that the site is operating within Google's TOS. I am an independent SEO and it took me all of 2 minutes to find numerous FLAGRANT violations.

What sort of message does Google send the market with the above behavior?

How Can Google Police Anyone?

Google has on multiple occasions penalized other UK based finance sites for SEO issues & link buying. But now that Google owns one of the horses in the race, and that horse has been found to be using anabolic steroids, can they legitimately penalize any of their new competitors?

If I had a UK finance site I would go on a link buying binge right now. Google can't penalize you for it because they are doing the same damn thing. And if they do penalize it for DOING THE EXACT SAME THING GOOGLE IS DOING then you know you have a legitimate gripe for the media, and I have no doubt Microsoft would be willing to help pick up the legal tab.

Google Eats Microsoft's Lunch Again!

Ultimately this is a body blow to Microsoft. Microsoft started to gain momentum in search through verticalization, but has since backed off. Meanwhile Google took Microsoft's ball and ran it in for a touchdown (acquiring MetaWebs, trying to buy ITA Software, and buying BeatThatQuote). And now one of MSN's portal ad partners is owned by Google:

Head of partnerships at MSN, Phil Coxon said, “At Microsoft Advertising, we’re passionate about collaborating with brands to create compelling advertising campaigns. By providing new and exclusive content that appeals to consumers, this partnership both enhances the overall MSN user experience as well as providing a great platform for BeatThatQuote to engage with their target audience on a meaningful level. This deal builds on our previous partnership with BeatThatQuote, which led to a 400% increase in revenue generated from insurance products. We’re delighted to continue to build on this relationship with this new campaign.”

Oooops.

Update

After a Googler read this blog post, it appears that Beat That Quote has been penalized.

Almost Everything is Unprofitable

Clowning Around

There is a saying in the bond trading market that if you don't know who the clown in a deal is then look in the mirror because it is probably you. Business is the same way. Almost everyone gets taken for a ride at least once.

What is Ignorance?

Ignorance is often viewed as a condescending word, but it is how we are all born. It is only through learning and experience we are able to do much more than survive. Any time you enter a new market or use a new strategy you start out from behind. You are the sucker who is losing money. Rarely does the new guy win just by showing up, or just by copying someone else's existing strategy. There has to be some point of differentiation.

A Brutal Uphill Climb

The leader has more data, more connections, more links, more capital, higher visitor value, and the algorithms have another layer of karma built over the top of them as well. Matt Cutts described part of the Panda update as "we actually came up with a classifier to say, okay, IRS or Wikipedia or New York Times is over on this side, and the low-quality sites are over on this side."

Roadblocks & Pot Holes Are Everywhere

Based on those sorts of disadvantages, why would anyone want to try SEO? Well in almost any other business model similar roadblocks and pain points exist, and SEO allows one to build momentum over time without it being an all or nothing risk. The slow buildup can lead you toward success in ways you may not have anticipated. And the cost of failure is often little more than time. Plus you gain knowledge even when something fails.

Is SEO Really Any Different?

I think Chris Dixon is one of the smarter entrepreneurs and angle investors out there, but was disappointed to see him write a post titled SEO is no longer a viable marketing strategy for startups:

I talk to lots of startups and almost none that I know of post-2008 have gained significant traction through SEO (the rare exceptions tend to be focused on content areas that were previously un-monetizable). Google keeps its ranking algorithms secret, but it is widely believed that inbound links are the preeminent ranking factor. This ends up rewarding sites that are 1) older and have built up years of inbound links 2) willing to engage in aggressive link building, or what is known as black-hat SEO.

A similar blog headline flipped around might read like "Most VC funded companies fail & founders get hosed on equity dilution, so getting funded is no longer a viable company formation strategy for startups." Of course something like that would be laughable, but it is no less absurd than saying SEO is no longer viable.

Sure coming from behind is hard, but the above misses that

  • Google has grown more aggressive in monetizing their search results through increased verticalization and navigational aids
  • many of the most profitable SEO plays are reinvesting into growth
  • most people who are successful with SEO do not like to attribute their success to it because doing so creates additional risks & more competition

Unique Market Approaches

Even treading water in a market where competitors are reinvesting profits & the market maker is tilting the table is quite respectable. If you want to come from behind and exactly clone someone else's business model, it won't likely be profitable. But that is why people attack markets from different perspectives. This is no different than why there are many different graphs. Chris isn't trying to beat Google in creating another link graph, but is looking at different signals.

Tectonic Shifts in Relevancy

Likewise marketing strategies can be vastly different between different companies and different projects within a company. Certain types of pages & certain types of websites rise and fall as the algorithms are adjusted to close down opportunistic loopholes. But as they make certain things harder they make other things easier. The whole content farm model was only enabled by an excessive weighting on domain authority & the introduction of rel=nofollow.

That opportunity may have fallen by the wayside. Many content mills just got hit pretty hard.

Was The Pain Really That Bad?

But for all the bluster about how it was one of the biggest changes in years, most of the content farms are only down maybe 20% to 50% in terms of traffic & revenues.

Sure that is a lot of revenue to disappear, but when you are operating at 80% net margins you can do that without it destroying your company. And this doesn't even take into account that many of these sites had a clean double over the past year. So if you grow 100% then lose 50% you are still even year on year, in spite of being penalized. Not bad in an environment where tons of businesses are going bankrupt offline.

And of course those sites getting whacked create opportunity for other folks, who build sites using different strategies.

A Cautionary Tale

About a half-decade ago a CEO of a start up contacted me & had us build a few links for them. Then they had to get their VCs approval for doing a full in-depth strategic review because it was going to cost well into 5 figures. Their VC investors didn't believe in SEO!

So that killed the project.

This company had a multi-lingual site where their leading market's content was only accessible through a drop down form where the URLs did not change. Fixing that issue to make the site crawlable would have produced more revenues in the first few months than the cost of our contract. But the VC didn't think SEO was valuable. They never got that tip. And for businesses which have network effects built in, losing $x today can easily be $10x or $20x a few years out.

Current Market Leaders Were Yesterday's Gray Area Marketers

Mr. Dixon also highlights how established TripAdvisor is, but when they were founded they were once the small dog just starting out. His article also fails to mention that TripAdvisor was Text-Link-Ads largest customer. In other words, they came from behind, took a calculated risk, and won. They backed off from the risks when the risks started to exceed the opportunity.

Not long after TripAdvisor started collecting consumer reviews, eHow was sold for $100,000. That turned out to be quite profitable for the buyers! And eHow was also known for aggressive & spammy link building against Google's guidelines. In fact, one of their largest competitors highlighted the lack of this information in their S1 filing:

The entire 250+ page document is devoid of any discussion of incoming links which is the cornerstone of search engine optimization. By reading through the lines, it appears that they have two primary sources for link development for their owned and operated sites: (1) from their “undeveloped websites” and (2) from their content partner sites. Although these two initiatives alone are generally not financially profitable, they are successful approaches to maximizing the incoming link equity in their owned and operated properties.

The point is that start ups shouldn't avoid all risk, but they should pick and choose their spots. The above sites are billion Dollar enterprises because they worked in the gray area to catch up & build a lead, and then pulled away from risk after they had a strong market position.

As time passes the opportunities change, but they don't really disappear.

When Best Practices Lead to Miserable Failures

How often do you ever hear the phrase worst practices? Probably never.

Everything is a best practice approach, right up until things change.

Consider AdSense websites.

Hey Look, a Case Study!

When you look at some of the biggest losers in the Google content farm update, many of them happened to be premium AdSense publishers which were even used by Google as case studies! For instance, Hub Pages or EzineArticles.

Everyone thinks that their content is the cream of the crop & that they will bounce back:

We are confident that over time the proven quality of our writers' content will be attractive to users. We have faith in Google's ability to tune results post major updates and are optimistic that the cream will rise back to the top in the coming weeks, which has been our experience with past updates - Paul Edmondson

The problem is that for many businesses there will be no bounce back. Some are simply over. The web has evolved & the algorithm has moved beyond them.

Where is the Much Needed Disclaimer?

What makes this worse is that when Google gives a site their premium AdSense feed & sets something up as a case study others will see that as an explicit endorsement.

THIS IS HOW YOU SHOULD DO IT.

Even after Google torches the companies that follow Google suggested best practices those case studies live on, offering what now amounts to maps to Google hell.

Adding Insult to Injury

What makes such filters/penalties even more infuriating is that in some cases when your site is slapped with a negative karma penalty, others who steal your content & wrap it in AdSense will outrank you, since their site does not yet have a negative karma penalty against it. :)

Individually the splog sites may not live long, but collectively they can keep outranking you to ensure you are invisible for your own words, even if you poured years of your life into creating something beautiful & important.

As Cult of Mac reports:

As we noted yesterday, Cult of Mac was collateral damage in Google’s war on crappy content farms. For some inexplicable reason, we got downgraded when Google tweaked its algorithms last Thursday.

But today we’re back in. We’re on Google News (a very important source of daily traffic) as well as Google’s general search results. However, we still get outranked by some of the scraper sites that steal our content, so not everything’s perfect.

That part in bold is the most outrageous part of this new "algorithmic" approach. When Google whacks your site then someone who steals your content will outrank you. And most sites stealing content *are* monetized via Google's DoubleClick & AdSense ads.

That whooshing sound you just heard was MFA sploggers making a mad dash to steal content from the list of currently penalized sites.

Cult of Mac is lucky they had enough pull with the press to get reconsidered. Most webmasters who got hit did not & anyone who has contracts based on set traffic levels or tight margins which just turned negative are in a pretty crappy situation. Yet another example of the importance of not fueling growth with debt & the importance of profit margins and a cash-on-hand safety net.

Who Are the Opportunistic Maximizers?

The problem with such an approach of maximizing everything you do to suck peak revenue out of the pageview is that things can change on a whim. I have seen some of Google's 1 on 1 AdSense optimization advice they sent to a friend of mine. I told my friend that the optimization advise was at best short-term opportunism that would end up crushing them in the long run if they actually implemented it.

Google doesn't care if following their advice torches your site if it makes them a bit more money, because ultimately there is another person standing in line waiting to follow.

My friend is lucky that they realized my advice was more trustworthy than the advice they were getting direct from Google. If they listened to Google back then their business might be destroyed today.

Google likes to position SEOs as exploiters out for the quick buck, but what honest analysis shows is that it is Google which is pushing the boundaries in terms of:

Google AdSense has a "get rich quick" ad category. That is something you won't find on our website (and one of the reasons we will never put AdSense ads on this site)!

AdSense Heatmaps? Look Out Below!

One of the worst hit sites in the AdSense farm update was WiseGeek. Sure WiseGeek must have had something like a 20% ad clickthrough rate. But with traffic falling 75%, maybe they would have been better off building a cleaner experience with a 5% CTR.

That said, they were simply following Google AdSense best practices:

Collateral Damage

What was the most profitable best practices based approach suddenly falls short. And the results are not always predictable. When Google decided to attack content farms who honestly knew that:

  1. somehow eHow.com would survive
  2. yet somehow Google's "algorithmic" approach would punt 10,000's of smaller websites that have far higher content quality

In advance of the solution I was fairly certain eHow would survive, but what I underestimated was the Google engineers. Or rather the ignorance of same. I simply couldn't imagine such a content farm algorithm going live that missed eHow and decimated the lives of so many independent webmasters.

I guess we can simply view this as an extension of Google's you can have any web you want so long as it is corporate TM policy. I think Brett Tabke said it best in a recent AdSense thread:

When the rules and the enforcements are made up by monopolies in a make believe world - there is no cheating.

The only "cheatings" is when it gets outside the lines of the law. - Brett Tabke

After the farmer update layoffs are already happening. Not only for monolithic useless content mill websites, but even in organizations where the content is pure as the driven snow.

AskTheBuilder is yet another Google AdSense case study. In spite of being a niche player well regarded in his community, Sistrix data shows the site off 87% after the most recent Google update!

Who Caused the Content Farm Problem?

Everyone likes to vilify the content farms and scrapers (and they deserve it) but the real villain behind all of this is CPC/CPM based advertising.

Can you imagine a world where your attention was sold off based on how long you stayed on a page rather then how often you switched pages? If google wants to fix their search results, they should focus on fixing adsense. The technology to more accurately measure a viewer's exposure to an ad are there, it just needs a trustworthy player to bring it to market. Someone trusted by both users and advertisers.

Google made click/impression-based advertising appealing to both groups and it made them what they are now. It's time to get away from it. - po

Smokescreen & Misdirection

I have long highlighted that Google's algorithmic-centric approach was blindingly hypocritical & that I felt the approach was nothing more than a scammy cover though which they can selectively exercise editorial discretion while claiming that "the algorithm did it."

Consider the following scenario:

  • roll in an algorithm that aggressively penalizes tons of borderline edge cases
  • see who complains to the media & has connections with the media
  • fix the rankings of those who you like & those with sway, while ignoring the rest

Can You Trust Google?

All of this leads to the obvious question: can you trust Google?

The short answer is yes.

The long answer is you can *always* expect Google to do what is in the best interest of Google. As they plow into field after field (payments, local, mobile, ecommerce, mortgage, credit cards, travel, weddings, fashion, etc.) & use their search dominance to manipulate other markets one would have to be blind to view Google as anything other than a competitor.

Maybe not today. Maybe not tomorrow. But some day they will come. And it is never fun when it happens to you. :(

Until that day may come, if you always follow their best practices, just remember ... ;)

Don't say you were not warned!

SEO Book Reopened

We are now fully open. Our membership area of the site has 2 separate tiers to it. The first gives you access to our tools, training modules, and newsletters, while running at $69 a month. The second level of access gives you access to everything in the first tier and access to our private member forums, while running $300 per month.

If you do not have a free/basic account set up yet then you can go here and set up your account and advanced permissions at the same time.

If you already have a free account you need to login here & then upgrade here. If for some reason that doesn't work, you may need to first reset your password, then login here, then upgrade here.

Google Kills eHow Competitors, eHow Rankings Up

Economics Drive Literally Everything

Media is all about profit margins. eHow was originally founded in 1998 & had $36 million in venture capital behind it. But the original cost structure was flawed due to high content costs. The site failed so badly that it was sold in 2004 for $100,000. The original site owners had GoogleBot blocked. Simply by unblocking GoogleBot and doing basic SEO to existing content the site had a revenue run rate of $4 million Dollars within 2 years, which allowed the site to be flipped for a 400-fold profit.

Demand Media bought it in 2006 and has pushed to scale it while making cheaper and lower quality content. Demand Media has since gone public with a $1.5+ billion valuation based largely on eHow. Just prior to the DMD IPO Google's Matt Cutts wrote a warning about content mills on the official Google blog.

Seeing Value Where Others do Not

People are arguing if Demand Media is overvalued at its current valuation, and at one point the NYT was debating buying Demand Media by rolling About.com into Demand & own 49% of the combined company. But the salient point to me is that we are talking about something that was bought for only $100,000 7 years ago. Sure the opportunities today may be smaller scale and different, but if you see value where others do not then recycling something that has been discarded can be quite valuable.

In this 6-page article about the eHow way on page 6 there was a recycling tip any SEO can do with the help of some OCR software:

The key, he said, was to keep costs low. If possible, don't pay for the intellectual content. Look for material, he urged, on which the copyright has expired. Any book published in the U.S. before 1923 was available.
He said he was in the process of turning vanloads of old books into websites. With a few hours of labor, for example, you could take a turn-of-the-century Creole cookbook and transform it into the definitive site for vintage Creole recipes. Google's AdSense program would then load the thing up with ads for shrimp and cooking pots and spices and direct people looking for shrimp recipes to your website.

A spin on the above, LoveToKnow has published a 1911 encyclopedia online. And ArticlesBase (an article farm which built up its popularity by linking to contributior sites) now slaps nofollow on all outbound links & is pulling in a cool $500k per month!

How did ArticlesBase grow to that size? It and Ezinearticles were a couple of the "selected few" which lasted through the last major burn down of article directories about 3 or 4 years ago. But it seems their model has peaked after this last Google update.

Search is Political

Content farms are proving to be a political issue in search. They are beginning to replace "the evil SEO" in the eye of the press as "what creates spam." Rich Skrenta created a spam clock which stated that a million spam pages are created every hour. He then followed up by banning 20 content farms from the Blekko search results & burning spam man. ;)

Microsoft also got in on the bashing, with Harry Shum highlighting that Google was funding web pollution. When Blekko's model is based on claiming Google polluted the web with crap, Microsoft says the same thing, and there is a rash of end user complaints, there are few independent experts the media can call upon to talk about Google - unless they decide to talk to SEOs, who tend to be quite happy to highlight Google's embarrassing hypocrisy. Freelance writers may claim that marketing is what screwed up the web, but ultimately Google has nobody credible and well known to defend them at this point. The only people who can defend Google's approach are those who have a taste in the revenue stream. Hence why Google had to act against content farms.

Always Be Selling

Demand Media's CEO is the consummate sales professional, when Google first warned about content farms Mr. Rosenblatt he used the above to disclaim that Google means "duplicate content" when they write about content farms. Then Demand quickly scrambled after they were caught publishing plagiarized content the following day. :)

Google stepping up their public relations smear campaigns against Bing and others is leaving Google looking either hypocritical or ignorant in many instances, like when a Google engineer lambasted an ad network without realizing Google was serving the scam ad.

Social Answers?

While on the content farm topic, it is worth mentioning that Answers.com was bought for $127 million & there is also a bunch of news about Ask's strategy in the Ask section near the bottom of this newsletter. On the social end of the answer farm model, Facebook was rumored to be looking into the space & Twitter bought a social answer service called Fluther. Even Groupon seems to be looking at the space. Quora is well hyped on TechCrunch, but will have a hard time expanding beyond the tech core it has developed.

High Quality Answer Communities?

At first glance StackExchange's growth looks exciting, but it has basically gone nowhere outside of the programming niche. In my opinion they are going to need to find subject matter experts to lead some of their niche sites & either pay those experts or give them equity in the sites if they want to lead in other markets. Worse yet, few people are as well educated about online schemes as programmers, so the other sites will not only lack leadership, but will also be much harder to police. Just look at the junk on Yahoo! Answers! There are Wordpress themes and open source CMS tools for QnA sites, but I would pick a tight niche early if I was going to build one as the broader sites seem to be full of spam and the niche sites struggle to cross the chasm. As of writing this, fewer than 50 Mahalo answers pages currently indexed in Google have over 100 views. It flat out failed, even with financial bribes and a PR spin man behind it.

A Warning

A Google engineer nicknamed moultano stated the following on Hacker News:

At the organizational level, Google is essentially chaos. In search quality in particular, once you've demonstrated that you can do useful stuff on your own, you're pretty much free to work on whatever you think is important. I don't think there's even a mechanism for shifting priorities like that.
We've been working on this issue for a long time, and made some progress. These efforts started long before the recent spat of news articles. I've personally been working on it for over a year. The central issue is that it's very difficult to make changes that sacrifice "on-topic-ness" for "good-ness" that don't make the results in general worse. You can expect some big changes here very shortly though.

A good example of the importance of padding out results with junk on-topic content to aid perceived relevancy can be seen by looking at the last screenshot of a search result here. Blekko banned the farms, but without them there is not much relevant content that is precisely on-topic. (In other words, content farms may be junk, but it is hard to have the same level of perceived relevancy without them).

New Signals

Google created a Chrome plugin to solicit end user feedback on content mills, but that will likely only reach tech savvy folks & the feedback is private. Google can claim to use any justification for removing sites they do not like though, just like they do with select link buying engagements. Look the other way where it is beneficial, and remove those which you personally dislike.

In a recent WSJ article Amit Singhal was quoted as saying new signals have been added to Google's relevancy algorithms:

Singhal did say that the company added numerous “signals,” or factors it would incorporate into its algorithm for ranking sites. Among those signals are “how users interact with” a site. Google has said previously that, among other things, it often measures whether users click the “back” button quickly after visiting a search result, which might indicate a lack of satisfaction with the site.

In addition, Google got feedback from the hundreds of people outside the company that it hires to regularly evaluate changes. These “human raters” are asked to look at results for certain search queries and questions such as, “Would you give your credit card number to this site?” and “Would you take medical advice for your children from those sites,” Singhal said.

Evolving the Model

One interesting way to evolve the content farm model is through the use of tight editorial focus, a core handful of strong editors, and wiki software. WikiHow was launched by a former eHow owner, and when you consider how limited their relative resources are, their traffic level and perceived editorial quality are quite high. Jack Herrick has struck how-to gold twice!

Going Political?

AOL purchased The Huffington Post for $315 million. Here are some reviews of that purchase. The following analysis is a bit rough, but I still think it is spot on - contrary to popular belief, most of Huffington Post's pageviews are still driven by their professionally sourced content.

Editors who have a distaste for pageview journalism are already quitting AOL. But if you are interested in the content farm business model, AOL's business plan was leaked publicly. Oooops. :D

Conflating Scraper Sites vs Content Farms

In addition to general content farms, Google is fighting a war against scraper websites. One such algorithmic update has already been done against sites repurposing content, and the content farm algorithm just recently went live & whacked a bunch of content farms. Check out the top losers from Sistrix's data.

Notice any content farms missing from the above list? Maybe the biggest one? Here is a list of some of eHow's closest competing sites (based on keywords, from SEM Rush). The ones in red got pummeled, the ones in yellow dropped as well & were fellow Demand Media sites, and the ones in green gained traffic.

Getting Hammered

Jason "will do anything for ink" Calacanis recently gave an about face speech claiming people need to step away from the content farm business model, and in doing so admitted that roughly everything he said about Mahalo over the past couple years was a complete lie. Surprise, surprise. The interesting bit is that the start up community - which used to fawn over his huckster PR driven ploys - no longer buys them. Jason claimed to have "pivoted" his business model again, but once again we see more garbage content. His credibility has been spent. And so have his rankings! Sistrix shows that not only is he ranking for fewer keywords, but that the graph has skewed downward to worse average positions.

After the Crash, What is Next?

The biggest content farms like Ask & eHow will still do well in the short run. Over the long run I see Google bringing the results of content farms to the attention of book publishers & then working to slowly rotate out from farmed content to published book content. Most readers do not know that most book writers are lucky if they earn $10,000 writing a 300 or 400 page tome. Publishers tell book authors that with the additional exposure they can often sell lots more other things, but unless the content is highly targeted that might not back out well for the author. But that cheap content is far better structured and far more vetted than the mill stuff is.

Over the past week I have been seeing more ebooks in the search results, though I am not entirely sure if that is just because I am searching for more rare technical stuff that simply might not be online.

The Question Nobody is Asking

I highlighted Google's hypocritical position in judging intent with links while claiming they need an algorithmic approach to content farms. But nobody is thinking beyond the obvious question. Everyone wants to know who Google punished the most, but nobody is asking who gained the most from this update.

Demand Media put out a statement that their traffic profile did not change materially.

But what they didn't mention is that eHow's rankings are actually up! In fact, their new distribution chart looks just like their old one, only skewed a bit to the left with higher rankings. eHow's profile is 15% better than it was before the update & the only site which gained more traffic from this update than eHow did was Youtube.

How Did We Get Here?

People may have been sorta aware of garbage content & saw it ranking, but were apathetic about it. Most people are far more passive consumers of search than they believe themselves to be - when the default orders switch people still tend to click the top ranked result. It was only when eHow started branding itself as a cheap and disposable answer factory that people started to become outraged with their business model.

Demand Media further benefited from flagrant spammy guideline violations, like 301 redirecting expired domains into deep eHow pages. People I know who have done similar have seen their sites torched in Google. But eHow is different!

If you listen to Richard's interviews, you would never know him to be the type to redirect expired domains:

We really want to let Google speak for themselves. Whatever Matt Cutts and Google want to (say) about quality we totally support that because again that’s their corporate interest. What we said and would have said is we applaud Google removing duplicate content ... removing shallow, low quality content because it clogs the search results. Both we and Google are 100 percent focused on making the consumer happy. It’s the right thing to do and it’s good for our business.

If you syndicate Google's spin you can get away with things that a normal person can't. Which is why eHow renounced the content farm label even faster than they created it.

Article directories & topical hub sites have been online since before eHow was created. But eHow's marketing campaign was so caustic & so toxic that it literally destroyed the game for most of their competitors.

And now that Google has "fought content farms" (while managing to somehow miss eHow with TheAlgorithm) most of Demand Media's competitors are dead & Richard Rosenblatt gets to ride off into the sunset with another hundred million Dollars, as eHow is the chosen one. :D

Long live the content farm!

I am Long Mahalo...

...too bad Google is not!

Google just did their first content farm update & Mahalo appears to have taken a swan dive in the search results, freeing up space for higher quality websites.

Google's Amit Singhal & Matt Cutts wrote:

Many of the changes we make are so subtle that very few people notice them. But in the last day or so we launched a pretty big algorithmic improvement to our ranking—a change that noticeably impacts 11.8% of our queries—and we wanted to let people know what’s going on. This update is designed to reduce rankings for low-quality sites—sites which are low-value add for users, copy content from other websites or sites that are just not very useful. At the same time, it will provide better rankings for high-quality sites—sites with original content and information such as research, in-depth reports, thoughtful analysis and so on.

Currently this update is US only, so if you are outside of the United States you may need to get a US VPN or add &gl=us to your search string's results on Google (likeso). Recent updates have had a variety of impacts and implications outside of content mills.

But it seems other large content farms are still doing well

What sets Mahalo apart then? Perhaps it was karma. ;)

I suppose we should "pivot" this post with some featured video content

The Joys of a Scott Richter Moment

When I went to sleep last night all was well. When I woke up my inbox was exploded with angry emails about people getting dozens and dozens of emails from us...in some cases perhaps almost 100. Since we put the new design live on the site I think people are more receptive to it. And there are not many Drupal websites which have more registered users than our site does. The combination of improved usability (in some areas, still working on others), better design, and a fairly strong rate of growth in popularity have caused us to hit a bit of a breaking point.

Some of the plugins for Drupal work solidly up to a point. But everything has limits. Servers, software, etc.

When you use technology sometimes it breaks. And never at a good time!

We were getting ready to fully launch our membership site publicly, but we just had a bit of a meltdown.

I think what happened was that our autoresponder was emailing the first x people & then it would reset without ticking that the day was done and those same people would get pounded with the same exact emails again. That cycle sorta looked like this

We are still troubleshooting the autoresponder issue to fix it, but while we are troubleshooting it of course we have to turn it off. (The first step to fixing any problem is to stop digging & stop making it worse)!

But while the autoresponder is turned off, it breaks the autoresponder unsubscribe links.

So it is a pretty crappy deal no matter what we do. Even if we used something like Mailchimp going forward, it still wouldn't fix the issue from yesterday.

Lose/lose, so you get to see the rudest behavior in the world and chastised. Fun stuff. If a person is intentionally sending email spam of course they would vary the message, not do it from an account that they actually answer replies on, etc. But people assume the worst because most people get burned by scammy get-rich-quick stuff before they find their way to quality SEO information.

Anyhow, the autoresponder is off until we troubleshoot it. Sorry about the bulk emails. And I can only imagine what Scott Richter's inbox must look like! Lucky for him he doesn't actually read it. ;)

We run a fairly lean business & rely on giving away a ton of stuff to do our marketing for us to attract customers. Rather than bulking up on sales staff we decided to be lean and efficient. If a person wants a sales call to try to squeeze a free consult we say no thanks. This model has worked decently well for us, but whenever anything breaks it sucks because we don't have tons of slack built into our business.

The bright side of the issue is this: even though a minority of people who responded cursed, most people were actually surprisingly polite given how annoying that autoresponder repeat was. And while there are all sorts of food riots in the Middle East & countries collapsing, I feel a bit lucky to have myself as my biggest problem & to be able to run a site with so many great members who give us the benefit of the doubt when I shoot myself (and everyone else) in the foot. ;)

Thanks guys!

Pages