The #1 goal for any organization is self-preservation. When people feel things are fairly just & they are just getting by they are fine with squeezing out more efficiency in what they do and figuring out ways to pay the bills. But when people feel the table is tilted at some point they stop caring and do whatever it takes.

Ex Post Facto

Some longtime AdWords advertisers have recently been punished for affiliate ads they ran 8 years ago where some of the sites they promoted at some point fell out of Google's graces through an ad system which never allows you to delete your history & offers ex post facto regulations that turn a regular advertiser arbitrarily into a spammer.

What's worse is that sometimes the data Google ties together creates guilt where there is nothing but innocence.

AdSense, AdSense, AdSense

In 3 weeks it will have been 3 months since Google first launched Panda. Outside of bloggers with 50,000 RSS subscribers few (if any) reports of recovery from Panda have been seen. Some of the theories floating around what caused Panda attempt to tie it to AdSense & many of Google's AdSense case studies are now highlighting best practices to follow if you want to be just like the sites Google torched.

As if that wasn't conflicting enough, some of the webmasters that were torched by Panda received automated messages that they were missing out on revenues by not using the maximum allotted number of ad units. After the huge fall off from Panda, Google has been pushing AdSense so hard that many webmasters have been receiving unsolicited emails from Google suggesting they sign up for AdSense.

I won't run AdSense on our main sections of this site because it would be tacky and destroy perceived credibility (having a "submit your site to 2000 search engines for $29" ad next to the content doesn't inspire trust on an SEO site). I could create a content farm answers section of the site that mirrors Ask's strategy, but with a higher level of quality. I won't though, because it would be viewed as spam because I am me. Once again, SEOs should be held to a higher standard than search engines. ;)

That Which You Consume, Consumes You

Where this rubs wrong is not only the overt brand push, but also that some of Google's pushes at expansion down the search funnel have looked a lot like the spam they claim to fight.

Many UK finance comparison sites were penalized for spammy link buys, and then Google somehow managed to buy BeatThatQuote without any due diligence. Others who were penalized for sketchy links (say like Overstock.com) were whacked for a couple months. BeatThatQuote was ranking again in Google in only 2 weeks ***without*** fixing any of the actual spam link buys.

TechCrunch's April 1st article about Google Places being inadvertently classified as a content farm sounded so authentic that I saw multiple friends in-the-know pass it around as though it was true.

Bad Actors

In the Wall Street Journal there was an article about the Panda update highlighting that many small businesses were laying off their employees. The same article highlighted numerous cost extensive desperate marketing measures the firms were taking which may or may not work. Google didn't disclose much in the article other than:

The Google spokesman says the company doesn't disclose details about changes it makes to its algorithms because doing so "would give bad actors a way to game our systems."

Nobody likes bad actors, but most of the webmasters that were hit were not bad actors. Rather, most of them were naive & simply followed the Google guidelines thinking that was in their best interests and perhaps would allow them to stay competitive. Unfortunately, it wasn't.

Not only did the update allow some information-less pages to rank better than ever, but certain folks with 100% duplicate content screamed to the top of the search results.

Don't Let the Door Hit You on the Way Out!

If you adhere to guidelines, get beat down, are not told why, and are told that generally sites need to "improve their quality" that can be a pretty infuriating message. The presumption that your stuff isn't good enough when 3rd grade rewrites of your content now outrank you is both smug and obnoxious. What is worse about the update though now is that many scraper websites are outranking the original content sources, so the message is that your content is plenty good enough, but it is just not good enough when it is on your site. A large portion of those scraper sites are monetized via Google AdSense & would not even exist if it were not for AdSense.

So Google whacks your site, tells you to clean up your act (& increase your operating costs while decreasing your margins), lumps you in the bad actors group, offers no information about when the pain will (or even could) end, pays someone to steal your content, then ranks that stolen copy of your content above you in the search results.

Make Your Move

If a person has the pleasure to experience the above it doesn't take much critical thinking skills to develop a different perspective on search.

Ultimately this is going to lead to a "why not" approach to search for many folks in the search space.

• If Google already dinged your website why wouldn't you remove AdSense & replace it with competing ad programs? Why not test those affiliate programs you have been meaning to test? If you have to rework your content anyway, why not move past AdSense/webmaster welfare?
• If your AdWords budget was marginally profitable & you were buying ads to compliment your organic exposure, why wouldn't you stop buying ads with Google & test running ads on other websites? Google is fine funding an affiliate network that uses direct links, so why not use clean links on your ad buys? If you like run it through a self-hosted affiliate program so that you are just like Google.
• If your site is already whacked why wouldn't you buy links to help boost its ranking back?
• If your site earns nothing from search, why wouldn't you sell links if you have to do whatever it takes to make costs?
• If your site gets penalized & someone copying your content & wrapping it in AdSense outranks you why wouldn't you create new mirror sites? Why wouldn't you create scraper websites to pollute Google with?
• If rankings are unpredictable & one site is no longer enough, why wouldn't you create backup sites & projects of various levels of quality & effort? At this point diversity simply serves as a needed form of insurance.
• If while running these purely scientific experiments you accidentally run into something that works really well that shouldn't, why not scale it to the moon?

I am not convinced that the search results are any cleaner today than they were a few months ago. However I am fairly certain things will soon head south. I am not advocating going out of your way to be extra spammy, but am just highlighting the cost-benefit analysis which is going through the heads of thousands of webmasters who Google just torched.

Google is betting that anonymous strangers will behave more kindly than Google has, but when an animal is backed into a corner it often acts in unpredictable (and even uncontrollable) ways.

The big problem for Google is this: "when innocence itself, is brought to the bar and condemned, especially to die, the subject will exclaim, it is immaterial to me whether I behave well or ill, for virtue itself is no security." - John Adams

Changing the Cost-benefit Analysis

In the last post I mentioned how the US government tried to change the cost benefit analysis for some sleazy executives at pharmaceutical corporations which continue to operate as criminal enterprises that simply view repeated fines as a calculable cost of doing business.

If you think about what Google's Panda update did, it largely changed the cost-benefit analysis of many online publishing business models. Some will be frozen with fear, others will desperately throw money at folks who may or may not have solutions, while others who gained will buy additional marketshare for pennies on the Dollar.

"We actually came up with a classifier to say, okay, IRS or Wikipedia or New York Times is over on this side, and the low-quality sites are over on this side." - Matt Cutts

Now that Google is picking winners and losers the gap between winners & losers rapidly grows as the winners reinvest.

And that word invest is key to understanding the ecosystem.

Beware of Scrapers

To those who are not yet successful with search, the idea of spending a lot of money building on a strategy becomes a bit more risky when you see companies like Demand Media that have spent $100's of millions growing an empire only to see 40% of the market value evaporate in a couple weeks due to a single Google update. There are literally thousands of webmasters furiously filing DMCA reports to Google after Panda, because Google decided that the content quality was fine if it was on a scraper site, but the exact same content lacked quality when on the original source site.

And even some sites that were not hit by Panda (even some which have thousands of inbound links) are still getting outranked by mirroring scrapers. Geordie spent hours sharing tips on how to boost lifetime customer value. For his efforts, Google decided to rank a couple scrapers as the original source & filter out PPCBlog as duplicate content, in spite of one of the scrapers even linking to the source site.

Outstanding work Google! Killer algo :D

Even if the thinking is misguided or an out of context headline, Reuters articles like Is SEO DOA as a core marketing strategy? do nothing to build confidence to make large investments in the search channel. Which only further aids people trying to do it on the cheap. Which gets harder to do as SEO grows more complex. Which only further aids the market for lemons effect.

Market Domination

At the opposite end of the spectrum, there are currently some search results which look like this

All of the colored boxes are the same company. You need a quite large monitor to get any level of result diversity above the fold. The company that was on the right side of the classifier can keep investing to build a nearly impenetrable moat, while others who fell back will have a hard time justifying the investment. Who wants to scale up on costs while revenues are down & the odds of success are lower? Few will. But the company with the top 3 (or top 6) results is collecting the data, refining their pitch, and re-investing into locking down the market.

Much like the Gini coefficient shows increasing wealth consolidation in the United States, search results where winners and losers are chose by search engines creates a divide where doing x will be very profitable for company A, while doing the exact same thing will be a sure money loser for company B.

Thin Arbitrary Lines in the Sand

The lines between optimization & spam blur as some trusted sites are able to rank a doorway page or a recycled tweet. Once site owners know they are trusted, you can count on them green lighting endless content production.

Scraping the Scrape of the Scrape

Many mainstream media websites have topics subdomains where they use services like DayLife or Truveo to auto-generate a near endless number of "content pages." To appreciate how circular it all is consider the following

• a reporter makes a minimally informing Tweet
• Huffington Post scrapes that 3rd party Tweet and ranks it as a page
• I write a blog post about how outrageous that Huffington Post "page" was
• SFGate.com has an auto-generated "Huffington Post" topics page (topics.sfgate.com/topics/The_Huffington_Post) which highlighted my blog post
• some of the newspaper scraper pages rank in the search results for keywords
• sites like Mahalo scrape the scrape of the scrape
• etc.

At some point in some such loops I am pretty certain the loops start feeding back into themselves & create a near-infinite cycle :D

An Endless Sea of "Trustworthy" Content

The OPA mentioned a billion dollar shift in revenues which favors large newspapers. But those "pure" old-school media sites now use services like DayLife or Truveo to auto-generate content pages. And it is fine when they do it.

...but...

The newspapers call others scammy agents of piracy and copyright violators for doing far less at lower scale, all while wanting to still be ranked highly (even while putting their own original content behind a paywall), and then go out and do the exact same scraping that they complain about others doing. It is the tragedy of the commons played out on an infinite web where the cost of an additional page is under a cent & everyone is farming for attention.

And the piece of pie everyone is farming for is shrinking as:

• competition increases faster than the growth of the market
• Google mixes more verticals into the search results & replaces 3rd party content with more Google goodness
• some of the media ad Dollars shift from buying media to building branded websites

Brands Becoming the Media

Rather than subsidizing the media with ads, brands are becoming the media:

Aware that consumers spend someplace between eight and 10 hours researching cars before they contact a dealer, auto markers and dealers are vectoring ever-greater portions of their marketing budgets into intercepting consumers online.

As but one example, Ford is so keen about capturing online tire-kickers that its website gives side-by-side comparisons between its Fiesta and competing brands. While you are on the Ford site, you can price the car of your dreams, investigate financing options, estimate your payment, view local dealer inventories and request a quote from a dealer.

Search Ads Replacing the Organic Search Results

AdWords is eating up more of the value chain by pushing big brands

• comparison ads = same brands that were in AdWords appearing again
• bigger adwords ads with more extensions = less diversity above the fold
• additional adwords ad formats (like product ads) = less diversity (most of the advertisers who first tried it were big box stores, and since it is priced on a CPA profit share basis the biggest brands that typically have more pricing power with manufacturers win)

Other search services like Ask.com and Yahoo! Search are even more aggressive with nepotistic self promotion.

Small Businesses Walking a Tightrope (or, the Plank)

Not only are big brands being propped up with larger ad units (and algorithmically promoted in the organic search results) but the unstable nature of Google's results further favors big business at the expense of small businesses via the following:

• more verticals & more ad formats = show the same sources multiple times over
• less stability = more opportunities for spammers (they typically have high margins & lots of test projects in the work...when one site drops another one is ready to pop into the game...really easy for scrapers to do...just grab content & wait for the original source to be penalized, or scrape from a source which is already penalized)
• less stability = small businesses have to fire employees hard to make payroll
• less stability = lowers multiples on site sales, making it easier for folks like WebMD, Quinstreet, BankRate, and Monster.com to buy out secondary & tertiary competing sites

If you are a small business primarily driven by organic search you either need to have big brand, big ego, big balls, or a lack of common sense to stay in the market in the years to come, as the market keeps getting consolidated. ;)

You can learn a lot about how search has improved over the years by reading Matt Cutts. Recently he highlighted how search was irrelevant in the past due to a lack of diversity:

Seven of the top 10 results all came from one domain, and the urls look a little… well, let’s say fishy. In 1999 and early 2000, search engines would often return 50 results from the same domain in the search results. One nice change that Google introduced in February 2000 was “host crowding,” which only showed two results from each hostname. ... Suddenly, Google’s search results were much cleaner and more diverse! It was a really nice win–we even got email fan letters.

Thanks to those kinds of improvements, in 2011 we never have to look at search results like this.*

* And by never, I mean, unless the results are linking to fraternal Google pages, in which case, game on!

Why should Google result crowding not apply to Google.com? Sure they can say those books are from different authors, but many websites are ran by organizations with multiple authors. Some websites are even built through the partnerships of multiple different business organizations. Who knows, maybe some searchers are uncomfortable with every other listing being an out of context book highlight.

In the past I have been called cynical for highlighting stuff like the following image

I saw it as part of a trend toward home cooking promotions. And I still view it that way. The above books promotion is simply further proof of concept.

Outside of...

• Youtube
• other Google owned and operated sites
• a branded website ranking for its own brand

Can you show me *any* occurrence of a result where a site is listed 5 times in the search result? Bonus points if you can find it where the 5 times are not grouped into 1 bunch via result crowding.

Other than a home cooking override, how is it possible that this problem fixed years ago suddenly re-appears?

As a thought experiment, ask yourself if that Google ranking accident would happen if the content archive being served up was promoting media hosted on Microsoft servers.

A friend of mine summed it up nicely with:

well, it's not everyday you see that kind of power and the fact that other sites aren't afforded the same opportunity makes me think that they are being anti-competitive. Google literally wrote the book (ok scraped it) on anti-competitive practices.

Guidelines are pushed as though they are commandments from a religious tome, but they are indeed a set of arbitrary devices used to hold down those who don't have an in with Google.

When Google nuked BeatThatQuote I guessed that the slap on the wrist would last a month & give BTQ time to clean up their mess.

As it turns out, I was wrong on both accounts.

Beat That Quote is already ranking again. They rank better than ever & only after only 2 weeks!

And the spam clean up? Google did NOTHING of the sort.

Every single example (of Google spamming Google) that was highlighted is still live.

Now Google can claim they handled the spam on their end / discounted it behind the scenes, but such claims fall short when compared to the standards Google holds other companies to.

• Most sites that get manually whacked for link-based penalties are penalized for much longer than 2 weeks.
• Remember the brand damage Google did to companies like JC Penny & Overstock.com by talking to the press about those penalties? In spite of THOUSANDS of media outlets writing about Google's BTQ acquisition, The Register was the most mainstream publication discussing Google's penalization of BeatThatQuote, and there were no quotes from Google in it.
• When asking for forgiveness for such moral violations, you are supposed to grovel before Google admitting all past sins & admit to their omniscient ability to know everything. This can lead one to over-react and actually make things ever worse than the penalty was!
• In an attempt to clean up their spam penalties (or at least to show they were making an effort) JC Penny did a bulk email to sites linking to them, stating that the links were unauthorized and to remove them. So JC Penny not only had to spend effort dropping any ill gotten link equity, but also lost tons of organic links in the process.

Time to coin a new SEO phrase: token penalty.

token penalty: an arbitrary short-term editorial action by Google to deflect against public relations blowback that could ultimately lead to review of anti-competitive monopolistic behaviors from a search engine with monopoly marketshare which doesn't bother to follow its own guidelines.

Your faith in your favorite politician should be challenged after you see him out on the town snorting coke and renting hookers. The same is true for Googler's preaching their guidelines as though it is law while Google is out buying links (and the sites that buy them).

You won't read about this in the mainstream press because they are scared of Google's monopolistic business practices. Luckily there are blogs. And Cyndi Lauper. ;)

Update: after reading this blog post, Google engineers once again penalized BeatThatQuote!

Do You Really Want That Indexed?

On-demand indexing was a great value added feature for Google site search, but now it carries more risks than ever. Why? Google decides how many documents make their primary index. And if too many of your documents are arbitrarily considered "low quality" then you get hit with a sitewide penalty. You did nothing but decide to trust Google & use Google products. In response Google goes out of its way to destroy your business. Awesome!

Keep in mind that Google was directly responsible for the creation of AdSense farms. And rather than addressing them directly, Google had to roll everything through an arbitrary algorithmic approach.

< meta name="googlebot" content="noindex" />

Part of the prescribed solution to the Panda Update is to noindex content that Google deems to be of low quality. But if you are telling GoogleBot to noindex some of your content, then if you are also using them for site search, you destroy the usability of their site search feature by making your content effectively invisible to your customers. For Google Site Search customers this algorithmic change is even more value destructive than the arbitrary price jack Google Site Search recently did.

We currently use Google Site Search on our site here, but given Google's arbitrary switcheroo styled stuff, I would be the first person to dump it if they hit our site with their stupid "low quality" stuff that somehow missed eHow & sites which wrap repurposed tweets in a page. :D

Cloaking vs rel=noindex, rel=canonical, etc. etc. etc.

Google tells us that cloaking is bad & that we should build our sites for users instead of search engines, but now Google's algorithms are so complex that you literally have to break some of Google's products to be able to work with other Google products. How stupid! But a healthy reminder for those considering deeply integrating Google into your on-site customer experience. Who knows when their model will arbitrarily change again? But we do know that when it does they won't warn partners in advance. ;)

I could be wrong in the above, but if I am, it is not easy to find any helpful Google documentation. There is no site-search bot on their list of crawlers, questions about if they share the same user agent have gone unanswered, and even a blog post like this probably won't get a response.

That is a reflection of only one more layer of hypocrisy, in which Google states that if you don't provide great customer service then your business is awful, while going to the dentist is more fun than trying to get any customer service from Google. :D

I was talking to a friend about this stuff and I think he summed it up perfectly: "The layers of complexity make everyone a spammer since they ultimately conflict, giving them the ability to boot anyone at will."