Nextag has long been critical of Google's shifts on the shopping search front. Are their complaints legitimate, or are they just whiners?
Data, More Reliable Than Spin
Nothing beats data, so lets start with that.
This is what Nextag's search exposure has done over the past few years, according to SearchMetrics.
If Google did that to any large & politically connected company, you can bet regulators would have already took action against Google, rather than currently negotiating with them.
What's more telling is how some other sites in the shopping search vertical have performed.
PriceGrabber, another player in the shopping search market, has also slowly drifted downward (though at a much slower rate).
The above creates an interesting market dynamic...
the long established market leader can wither on the vine for being too focused on their niche market & not broadening out in ways that increase brand awareness
a larger site with loads of usage data can outsource the vertical and win based on the bleed of usage data across services & the ability to cross promote the site
the company investing in creating the architecture & baseline system that powers other sites continues to slide due to limited brand & a larger entity gets to displace the data source
Google then directly enters the market, further displacing some of the vertical players
The above puts Nextag's slide in perspective, but the problem is that they still have fixed costs to manage if they are going to maintain their editorial quality. Google can hand out badges for people willing to improve their product for free or give searchers a "Click any fact to locate it on the web. Click Wrong? to report a problem" but others who operated with such loose editorial standards would likely be labeled as a spammer of one stripe or another.
Scrape-N-Displace
Most businesses have to earn the right to have exposure. They have to compete in the ecosystem, built awareness & so on. But Google can come in from the top of the market with an inferior product, displace the competition, economically starve them & eventually create a competitive product over time through a combination of incremental editorial improvements and gutting the traffic & cash flow to competing sites.
"The difference between life and death is remarkably small. And it’s not until you face it directly that you realize your own mortality." - Dustin Curtis
The above quote is every bit as much true for businesses as it is for people. Nothing more than a threat of a potential entry into a market can cut off the flow of investment & paralyze businesses in fear.
If you have stuff behind a paywall or pre-roll ads you might have "poor user experience metrics" that get you hit by Panda.
If you make your information semi-accessible to Googlebot you might get hit by Panda for having too much similar content.
If you are not YouTube & you have a bunch of stolen content on your site you might get hit by a copyright penalty.
If you leave your information fully accessible publicly you get to die by scrape-n-displace.
If you are more clever about information presentation perhaps you get a hand penlty for cloaking.
None of those is a particularly desirable way to have your business die.
Editorial Integrity
In addition to having a non-comprehensive database, Google Shopping also suffers from the problem of line extension (who buys video games from Staples?).
The bigger issue is that issue of general editorial integrity.
Are products in stock? Sometimes no.
It is also worth mentioning that some sites with "no product available" like Target or Toys R Us might also carry further Google AdSense ads.
Then there are also issues with things like ads that optimize for CTR which end up promoting things like software piracy or the academic versions of software (while lowering the perceived value of the software).
Over the past couple years Google has whacked loads of small ecommerce sites & the general justification is that they don't add enough that is unique, and that they don't deserve to rank as their inventory is unneeded duplication of Amazon & eBay. Many of these small businesses carry inventory and will be driven into insolvency by the sharp shifts in traffic. And while a small store is unneeded duplication, Google still allows syndicated press releases to rank great (and once again SEOs get blamed for Google being Google - see the quote-as-headline here).
Let's presume Google's anti-small business bias is legitimate & look at Google Shopping to see how well they performed in terms of providing a value add editorial function.
A couple days ago I was looking for a product that is somewhat hard to find due to seasonal shopping. It is often available at double or triple retail on sites like eBay, but Google Shopping helped me locate a smaller site that had it available at retail price. Good deal for me & maybe I was wong about Google.
... then again ...
The site they sent me to had the following characteristics:
URL - not EMD & not a brand, broken English combination
logo - looks like I designed it AND like I was in a rush when I did it
about us page - no real information, no contact information (on an ecommerce site!!!), just some obscure stuff about "direct connection with China" & mention of business being 15 years old and having great success
age - domain is barely a year old & privacy registered
inbound links - none
product price - lower than everywhere else
product level page content - no reviews, thin scraped editorial, editorial repeats itself to fill up more space, 3 adsense blocks in the content area of the page
no reviews, thin scraped editorial, editorial repeats itself to fill up more space, 3 adsense blocks in the content area of the page
no reviews, thin scraped editorial, editorial repeats itself to fill up more space, 3 adsense blocks in the content area of the page
no reviews, thin scraped editorial, editorial repeats itself to fill up more space, 3 adsense blocks in the content area of the page
the above repetition is to point out the absurdity of the formatting of the "content" of said page
site search - yet again the adsense feed, searching for the product landing page that was in Google Shopping I get no results (so outside of paid inclusion & front/center placement, Google doesn't even feel this site is worth wasting the resources to index)
checkout - requires account registration, includes captcha that never matches, hoping you will get frustrated & go back to earlier pages and click an ad
It actually took me a few minutes to figure it out, but the site was designed to look like a phishing site, with intent that perhaps you will click on an ad rather than trying to complete a purchase. The forced registration will eat your email & who knows what they will do with it, but you can never complete your purchase, making the site a complete waste of time.
Looking at the above spam site with some help of tools like NetComber it was apparent that this "merchant" also ran all sorts of scraper sites driven on scraping content from Yahoo! Answers & similar, with sites about Spanish + finance + health + shoes + hedge funds.
It is easy to make complaints about Nextag being a less than perfect user experience. But it is hard to argue that Google is any better. And when other companies have editorial costs that Google lacks (and the other companies would be labeled as spammers if they behaved like Google) over time many competing sites will die off due to the embedded cost structure advantages. Amazon has enoug scale that people are willing to bypass Google's click circus & go directly to Amazon, but most other ecommerce players don't. The rest are largely forced to pay Google's rising rents until they can no longer afford to, then they just disappear.
Bonus Prize: Are You Up to The Google Shopping Test?
The first person who successfully solves this captcha wins a free month membership to our site.
Google launched a disavow links tool. Webmasters who want to tell Google which links they don’t want counted can now do so by uploading a list of links in Google Webmaster Tools.
If you haven’t received an “unnatural link” alert from Google, you don’t really need to use this tool. And even if you have received notification, Google are quick to point out that you may wish to pursue other avenues, such as approaching the site owner, first.
Webmasters have met with mixed success following this approach, of course. It's difficult to imagine many webmasters going to that trouble and expense when they can now upload a txt file to Google.
Careful, Now
The disavow tool is a loaded gun.
If you get the format wrong by mistake, you may end up taking out valuable links for long periods of time. Google advise that if this happens, you can still get your links back, but not immediately.
Could the use of the tool be seen as an admission of guilt? Matt gives examples of "bad" webmaster behavior, which comes across a bit like “webmasters confessing their sins!”. Is this the equivalent of putting up your hand and saying “yep, I bought links that even I think are dodgy!”? May as well paint a target on your back.
Some webmasters have been victims of negative SEO. Some webmasters have had scrapers and autogen sites that steal their content, and then link back. There are legitimate reasons to disavow links. Hopefully, Google makes an effort to make such a distinction.
Not only is it difficult working out the links that may be a problem, it can be difficult getting a view of the entire link graph. There are various third party tools, including Google’s own Webmaster Central, but they aren’t exhaustive.
Matt mentioned that the link notification emails will provide examples of problem links, however this list won't be exhaustive. He also mentioned that you should pay attention to the more recent links, presumably because if you haven't received notification up until now, then older links weren't the problem. The issue with that assumption is that links that were once good can over time become bad:
That donation where you helped a good cause & were later mortified that "online casino" and "discount cheap viagra" followed your course for purely altruistic reasons.
That clever comment on a well-linked PR7 page that is looking to cure erectile dysfunction 20 different ways in the comments.
Links from sources that were considered fine years ago & were later repositioned as spam (article banks anyone?)
Links from sites that were fine, but a number of other webmasters disavowed, turning a site that originally passed the sniff test into one that earns a second review revealing a sour stench.
This could all get rather painful if webmasters start taking out links they perceive to be a problem, but aren’t. I imagine a few feet will get blasted off in the process.
Webmasters Asked, Google Gaveth
Webmasters have been demanding such a tool since the un-natural notifications started appearing. There is no question that removing established links can be as hard, if not harder, than getting the links in the first place. Generally speaking, the cheaper the link was to get the higher the cost of removal (relative to the original purchase price). If you are renting text link ads for $50 a month you can get them removed simply by not paying. But if you did a bulk submission to 5,000 high PR SEO friendly directories...best of luck with that!
It is time consuming. Firstly, there’s the overhead in working out which links to remove, as Google doesn’t specify them. Once a webmaster has made a list of the links she thinks might be a problem, she then needs to go through the tedious task of contacting each sites and requesting that a link be taken down.
Even with the best will in the world, this is an overhead for the linking site, too. A legitimate site may wish to verify the identity of the person requesting the delink, as the delink request could come from a malicious competitor. Once identity has been established, the site owner must go to the trouble of making the change on their site.
This is not a big deal if a site owner only receives one request, but what if they receive multiple requests per day? It may not be unreasonable for a site owner to charge for the time taken to make the change, as such a change incurs a time cost. If the webmaster who has incurred a penalty has to remove many links, from multiple sites, then such costs could quickly mount. Taken to the (il)logical extremes, this link removal stuff is a big business. Not only are there a number of link removal services on the market, but one of our members was actually sued for linking to a site (when the person who was suing them paid to place the link!)
It’s hard to imagine this data not finding it’s way to the manual reviewers. If there are multiple instances of webmasters reporting paid links from a certain site, then Google have more than enough justification to take it out. This would be a cunning way around the “how do we know if a link is paid?” problem.
Webmasters will likely incorporate bad link checking into their daily activities. Monitoring inbound links wasn’t something you had to watch in the past, as links were good, and those that weren’t, didn’t matter, as they didn’t affect ranking anyway. Now, webmasters may feel compelled to avoid an unnatural links warning by meticulously monitoring their inbound links and reporting anything that looks odd. Google haven’t been clear on whether they would take such action as a result - Matt suggests they just reclassify the link & see it as a strong suggestion to treat it like the link has a nofollow attribute - but no doubt there will be clarification as the tool beds in. Google has long used a tiered index structure & enough complaints might lower the tier of a page or site, cause it's ability to pass trust to be blocked, or cause the site to be directly penalized.
This is also a way of reaffirming “the law”, as Google sees it. In many instances, it is no fault of the webmaster that rogue sites link up, yet the webmaster will feel compelled to jump through Google’s hoops. Google sets the rules of the game. If you want to play, then you play by their rules, and recognize their authority. Matt Cutts suggested:
we recommend that you contact the sites that link to you and try to get links taken off the public web first. You’re also helping to protect your site’s image, since people will no longer find spammy links and jump to conclusions about your website or business.
Left unsaid in the above is most people don't have access to aggregate link data while they surf the web, most modern systems of justice are based on the presumption of innocence rather than guilt, and most rational people don't presume that a site that is linked to is somehow shady simply for being linked to.
If the KKK links to Matt's blog tomorrow that doesn't imply anything about Matt. And when Google gets featured in an InfoWars article it doesn't mean that Google desires that link or coverage. Many sketchy sites link to Adobe (for their flash player) or sites like Disney & Google for people who are not old enough to view them or such. Those links do not indicate anything negative about the sites being linked into. However, as stated above, search is Google's monopoly to do with as they please.
On the positive side, if Google really do want sites to conform to certain patterns, and will reward them for doing so by letting them out of jail, then this is yet another way to clean up the SERPs. They get the webmaster on side and that webmaster doing link classification work for them for free.
Who, Not What
For a decade search was driven largely by meritocracy. What you did was far more important than who you were. It was much less corrupt than the physical world. But as Google chases brand ad Dollars, that view of the search landscape is no longer relevant.
Large companies can likely safely ignore much of the fear-first approach to search regulation. And when things blow up they can cast off blame on a rogue anonymous contractor of sorts. Whereas smaller webmasters walk on egg shells.
When the government wanted to regulate copyright issues Google claimed it would be too expensive and kill innovation at small start ups. Google then drafted their own copyright policy from which they themselves are exempt. And now small businesses not only need to bear that cost but also need to police their link profiles, even as competitors can use Fivver, ScrapeBox, splog link networks & various other sources to drip a constant stream of low cost sludge in their direction.
Now more than ever, status is important.
Gotchas
No doubt you’ve thought of a few. A couple thoughts - not that we advocate them, but realize they will happen:
Intentionally build spam links to yourself & then disavow them (in order to make your profile look larger than it is & to ensure that competitor who follows everything you do - but lacks access to your disavow data - walks into a penalty).
Find sites that link to competitors and leave loads of comments for the competitor on them, hoping that the competitor blocks the domain as a whole.
Find sites that link to competitors & buy links from them into a variety of other websites & then disavow from multiple accounts.
Get a competitor some link warnings & watch them push to get some of their own clean "unauthorized" links removed.
The webmaster who parts on poor terms burning the bridge behind them, or leaving a backdoor so that they may do so at anytime.
If a malicious webmaster wanted to get a target site in the bad books, they could post obvious comment spam - pointing at their site, and other sites. If this activity doesn’t result in an unnatural linking notification, then all good. It’s a test of how Google values that domain. If it does result in an unnatural link notification, the webmaster could then disavow links from that site. Other webmasters will likely do the same. Result: the target site may get taken out.
To avoid this sort of hit, pay close attention to your comment moderation.
Please add your own to the comments! :) Gotchas, that is, not rogue links.
On Feb. 25, 2011, Google released Panda to wreak havoc on the web. While it may have been designed to take out content farms, it also took out scores of quality e-commerce sites. What do content farms and e-commerce sites have in common? Lots of pages. Many with zero or very few links. And on e-commerce sites with hundreds or thousands of products, the product pages may have a low quantity of content, making them appear as duplicate, low quality, or shallow to the Panda, thus a target for massive devaluation.
My e-commerce site was hit by Panda, causing a 60% drop in traffic overnight. But I was able to escape after many months of testing content and design changes. In this post, I'll explain how we beat the Panda, and what you can do to get your site out if you've been hit.
The key to freeing your e-commerce site from Panda lies at the bottom of a post Google provided as guidance to Pandalized sites:
One other specific piece of guidance we've offered is that low-quality content on some parts of a website can impact the whole site’s rankings, and thus removing low quality pages, merging or improving the content of individual shallow pages into more useful pages, or moving low quality pages to a different domain could eventually help the rankings of your higher-quality content.
Panda doesn't like what it thinks are "low quality" pages, and that includes "shallow pages". Many larger e-commerce sites, and likely all of those that were hit by Panda, have a high number of product pages with either duplicate bits of descriptions or short descriptions, leading to the shallow pages label. In order to escape from the Panda devaluation, you'll need to do something about that. Here are a few possible solutions:
Adding Content To Product Pages
If your site has a relatively small number of products, or if each product is unique enough to support entirely different descriptions and information, you may be able to thicken up the pages with unique, useful information. Product reviews can also serve the same purpose, but if your site is already hit by Panda you may not have the customers to leave enough reviews to make a difference. Additionally, some product types are such that customers are unlikely to leave reviews.
If you can add unique and useful information to each of your product pages, you should do so both to satisfy the Panda and your customers. It's a win-win.
Using Variations To Decrease Product Pages
Some e-commerce sites have large numbers of products with slight variations. For example, if you're selling t-shirts you may have one design in 5 different sizes and 10 different colors. If you've got 20 designs, you've got 1,000 unique products. However, it would be impossible to write 1,000 unique descriptions. At best, you'll be able to write one for each design, or a total of 20. If your e-commerce site is set up so that each of the product variations has a single page, Panda isn't going to like that. You've either got near 1,000 pages that look like duplicates, or you've got near 1,000 pages that look VERY shallow.
Many shopping carts allow for products to have variations, such that in the above situation you can have 20 product pages where a user can select size and color variations for each design. Switching to such a structure will probably cause the Panda to leave you alone and make shopping easier for your customers.
Removing Poor Performing Products
If your products aren't sufficiently unique to add substantial content to each one, and they also don't lend themselves to consolidation through selectable variations, you might consider deleting any that haven't sold well historically. Panda doesn't like too many pages. So if you've got pages that have never produced income, it's time to remove them from your site.
Getting Rid of All Product Pages
This is a bold step, but the one we were forced to take in order to recover. A great many of our products are very similar. They're variations of each other. But due to the limitations of our shopping cart combined with shipping issues, where each variation had different shipping costs that couldn't be programed into the variations, it was the only viable choice we were left with.
In this option, you redesign your site so that products displayed on category pages are no longer clickable, removing links to all product pages. The information that was displayed on product pages gets moved to your category pages. Not only does this eliminate your product pages, which make up the vast majority of your site, but it also adds content to your category pages. Rather than having an "add to cart" or "buy now" button on the product page, it's integrated into the category page right next to the product.
Making this move reduced our page count by nearly 90%. Our category pages became thicker, and we no longer had any shallow pages. A side benefit of this method is that customers have to make fewer clicks to purchase a product. And if your customers tend to purchase multiple products with each order, they avoid having to go from category page to product page, back to the category page, and into another product page. They can simply purchase a number of products with single clicks.
Noindexing Product Pages
If you do get rid of all links to your product pages but your cart is still generating them, you'll want to add a "noindex, follow" tag to each of them. This can also be a solution for e-commerce sites where all traffic enters on category level pages rather than product pages. If you know your customers are searching for phrases that you target on your category pages, and not specifically searching for the products you sell, you can simply noindex all of your product pages with no loss in traffic.
If all of your products are in a specific folder, I'd recommend also disallowing that folder from Googlebot in your robots.txt file, and filing a removal request in Google Webmaster Tools, in order to make sure the pages are taken out of the index.
Other Considerations: Pagination & Search Results Pages
In addition to issues with singular product pages, your e-commerce site may have duplicate content issues or a very large number of similar pages in the index due to your on-site search and sorting features. Googlebot will fill in your search form and index your search results pages, potentially leading to thousands of similar pages in the index. Make sure your search results pages have a rel="noindex, follow" tag or a rel="canonical" tag to take care of this. Similarly, if your product pages have a variety of sorting options (price, best selling, etc.), you should make sure the rel="canonical" tag points to the default page as the canonical version. Otherwise, each product page may exist in Google's index in each variation.
Maxmoritz, a long time member of our SEO Community, has been working in SEO full time since 2005. He runs a variety of sites, including Hungry Piranha, where he blogs regularly.
My good friend Bill at SEOByTheSea has unearthed a Google patent that will likely raise eyebrows, whilst others will have their suspicions confirmed.
The patent is called Ranking Documents. When webmasters alter a page, or links to a page, the system may not respond immediately to those changes. Rather, the system may change rankings in unexpected ways.
A system determines a first rank associated with a document and determines a second rank associated with the document, where the second rank is different from the first rank. The system also changes, during a transition period that occurs during a transition from the first rank to the second rank, a transition rank associated with the document based on a rank transition function that varies the transition rank over time without any change in ranking factors associated with the document.
Further:
During the transition from the old rank to the target rank, the
transition rank might cause:
a time-based delay response,
a negative response
a random response, and/or
an unexpected response
So, Google may shift the rankings of your site, in what appears to be a random manner, before Google settles on a target rank.
Let's say that you're building links to a site, and the site moves up in the rankings. You would assume that the link building has had a positive effect. Not so if the patent code is active, as your site may have already been flagged.
Google then toys with you for a while before sending your site plummeting to the target rank. This makes it harder to determine cause and effect.
Just because a patent exists doesn't mean Google is using it, of course. This may be just be another weapon in the war-of-FUD, but it sounds plausible and it’s something to keep in mind, especially if you're seeing this type of movement.
The Search Engine As Black Box
In ancient times (1990s), SEO thrived because search engines were stupid black boxes. If you added some keywords here, added a few links there, the black box would respond in a somewhat predictable, prescribed, fashion. Your rankings would rise if you guessed what the black box liked to "see", and you plummeted if you did too much of what the black box liked to see!
Ah, the good old days.
These days, the black box isn’t quite so stupid. It’s certainly a lot more cryptic. What hasn’t changed, however, is the battle line drawn between webmasters and search engines as they compete for search visitor attention.
If there are any webmasters still under the illusion that Google is the SEOs friend, that must be a very small club, indeed. Google used to maintain a - somewhat unconvincing - line that if you just followed their ambiguous guidelines (read: behaved yourself) then they would reward you. It was you and Google on the good side, and the evil spammers on the other.
Of late, Google appear to have gotten bored of maintaining any pretense, and the battle lines have been informally redrawn. If you’re a webmaster doing anything at all that might be considered an effort to improve rank, then you're a "spammer". Google would no doubt argue this has always been the case, even if you had to read between the lines to grasp it. And they’d be right.
Unconvinced?
Look at the language on the patent:
The systems and methods may also observe spammers’ reactions to rank changes caused by the rank transition function to identify documents that are actively being manipulated. This assists in the identification of rank-modifying spammers.
“Manipulated”? “Rank modifying spammers”? So, a spammer is someone who attempts to modify their rank?
I’ve yet to meet a webmaster who didn’t wish to modify their rank.
Google As A Competitor
Google’s business model relies on people clicking ads. In their initial IPO filing, Google identified rank manipulation as a business risk.
We are susceptible to index spammers who could harm the integrity of our web search results. There is an ongoing and increasing effort by “index spammers” to develop ways to manipulate our web search results
It’s a business risk partly because the result sets need to be relevant for people to return to Google. The largely unspoken point is Google wants webmasters to pay to run advertising, not get it for “free”, or hand their search advertising budget to an SEO shop.
Why would Google make life easy for competitors?
The counter argument has been that webmasters provide free content, which the search engines need in order to attract visitors in the first place. However, now relevant content is plentiful, that argument has been weakened. Essentially, if you don't want to be in Google, then block Google. They won't lose any sleep over it.
What has happened, however, is that the incentive to produce quality content, with search engines in mind, has been significantly reduced. If content can be scraped, ripped-off, demoted and merely used as a means to distract the search engine user enough to maybe click a few search engine ads, then where is the money going to come from to produce quality content? Google may be able to find relevant content, but "relevant" (on-topic) and "quality" (worth consuming) are seldom the same thing
One content model that works in such as environment is content that is cheap to produce. Cheap content can be quality content, but like all things in life, quality tends to come with a higher price tag. Another model that works is loss-leader content, but then the really good stuff is still hidden from view, and it's still hard to do this well, unless you've established considerable credibility - which is still expensive to do.
This is the same argument the newspaper publishers have been making. The advertising doesn’t pay enough to cover the cost of production and make a profit - so naturally the winner in this game cuts production cost until the numbers do add up. What tends to be sacrificed in this process - is quality.
NFSW Corp, a new startup by ex-TechCrunch and Guardian columnist writer Paul Carr has taken the next step. They have put everything behind a paywall. There is no free content. No loss-leaders. All you see is a login screen.
Is this the future for web publishing? If so, the most valuable content will not be in Google. And if more and more valuable content lies beyond Google's reach, then will fewer people bother going to Google in the first place?
Here’s how it works. Our engineers come up with some insight or technique and implement a change to the search ranking algorithm . They hope this will improve search results, but at this point it’s just a hypothesis. So how do we know if it’s a good change? First we have a panel of real users spread around the world try out the change, comparing it side by side against our unchanged algorithm. This is a blind test — they don’t know which is which. They rate the results, and from that we get a rough sense of whether the change is better than the original. If it isn’t, we go back to the drawing board. But if it looks good, we might next take it into our usability lab — a physical room where we can invite people in to try it out in person and give us more detailed feedback. Or we might run it live for a small percentage of actual Google users, and see whether the change is improving things for them. If all those experiments have positive results, we eventually roll out the change for everyone"
Customer focus is, of course, admirable, but you’ve got to wonder about a metric that doesn’t involve the needs of publishers. If publishing on the web is not financially worthwhile, then, over time, the serps will surely degrade in terms of quality as a whole, and users will likely go elsewhere.
There is evidence this is already happening. Brett at Webmasterworld pointed out that there is a growing trend amongst consumers to skip Google altogether and just head for the Amazon, and other sites, directly. Amazon queries are up 73 percent in the last year.
There may well be a lot of very clever people at Google, but they do not appear to be clever enough to come up with a model that encourages webmasters to compete with each other in terms of information quality.
If Google doesn’t want the highest quality information increasingly locked up behind paywalls, then it needs to think of a way to nurture and incentivise the production of quality content, not just relevant content. Tell publishers exactly what content Google wants to see rank well and tell them how to achieve it. There should be enough money left on the table for publishers i.e. less competition from ads - so that everyone can win.
I’m not going to hold my breath for this publisher nirvana, however. I suspect Google's current model just needs content to be "good enough."
The best part about a growing and very quickly changing industry is the diversity of viewpoints; the worst part is the exact same thing because sometimes 1 always equals 1 and doesn't need bullshit in lieu of evidence. I try my best to stay out of the limelight and just focus on making things happen. However, occasionally a topic will bother me so much that I have to chime in. The last time was over 5 years ago so I figure I'm due to speak up again. Today's topic? Negative SEO. My issue with the topic? Deniers.
There've been several posts on how negative SEO doesn't exist (those are the worst) or that maybe it exists but only weak sites can get hit (in other words, people with opinions that didn't do any testing). I'd like to put those topics to rest as best as a guy that keeps to himself can. I really should be able to do this in one sentence, but in the event what I write as the second half of this sentence doesn't do it for you, I have a couple stories; if crappy SEO of over-optimized anchors and junky links are to blame for ranking drops, how can it be said one cannot do this to someone else, and even if you were to deny this, then why the sudden rush to denounce certain links? On to some anecdotes!
While leading a training session overseas I mentioned a site I watched get hit by some negative SEO activities. I know that it was negative SEO and not a slip up on the SEOs' part by virtue of knowing the history/team behind the site and watching it as part of my normal data routine; the site was managed by the kind of guys that get asked to speak at SEOktoberfest...the kind of people I'd go work for if my bag of tricks ever ran out. Ok, so you're asking how I know it was negative SEO. The easiest explanation is that I watched the site spike heavily with on-theme anchors from junk sites over a one week period and was filtered shortly thereafter. It stayed filtered for just under few months, but 2 days after discussing the site and explaining how I knew the site was hit it magically reappeared (yes, there were googlers in the audience).
If you are skeptical then your first response better be that I'm only loosely describing one example so let me say that in the same industry where I've shared my knowledge of the subject on some more sophisticated methods (first released in the SEObook community), I feel almost like an information arms dealer since even the larger brands have themselves or through affiliated relationships been clubbing each other over the head. You read that right; I explained how I thought negative SEO could be employed and then watched a bunch of people actually do it, repeatedly. Unfortunately, I was hit too, but that's a different issue. In this particular industry, the only people left standing now are some poorly matched local results with fake reviews, a bunch of hacked domains, and the flotsam of macroparasites that gained popularity post Penguin. The only one that came back? The one I publically shared at a conference, explaining exactly how they were a victim based on the link patterns that didn't fit with the site's history over a several year period.
I'll wrap this up with a bit of humor. As a joke a friend of mine asked me to negative SEO him for his name. Let's say his name is John Doe and his domain is johndoe.com. The negative effect was temporary, but I was able to get him filtered for a little while on his name for maybe 120 seconds of my time and less than $50. The site did come back after a few days, but our mutual feeling on the matter is that for an extra $50 double-dose I could probably get the site filtered again. Neither of us wants negative SEO to get any more prevalent than it already is, so I'll skip the details on exactly how it was performed. There are multiple forms of negative SEO significantly scarier than someone with a copy of xrumer and in some cases there is very little you can do to prevent it; if a jerk wants to take you down, it can happen. If your niches begin to look like the wasteland I described above where I shared my thoughts a little too freely, then heaven help you because it doesn't look like Google is going to.
Cygnus has been involved in search since 1997 and loves tackling new and interesting (and of course lucrative) projects. Follow @Cygnus on Twitter for his rants.
When you think of all the implications of the above video (and all the things that are going on in machine learning & search), it can be somewhat difficult to think about sustainable strategies.
When platforms are new they start off as being fairly open to win attention & maximize their growth rates. Over time as they push to monetize they shift gears & what was once true becomes misleading. Thus a lot people likely come off as sounding like quack jobs because they keep having to reinvent themselves & reassess their belief systems as the markets change.
Hello Mr. Cynic
If you write things that sound like rants & complaints a lot of people mistake it as thinking you are a crank full of gloom & nonsense. For what it is worth, in many ways I think the future of the web will still be bright, but just relatively less bright than it was in the recent past for smaller players.
The web is becoming more & more like the physical world (andis mergingwith it). For a long time search & online was largely a meritocracy, where the best person could easily win even if they came from the most humble beginnings.
In search of years gone by, large & complex organizations that were overly bloated and inefficient routinely had their asses handed to them by smaller & more efficient operations. Butthensizebecame a primary signal of relevancy & quality, and that all changed. As Larry Page & Sergey Brin warned, the relevancy algorithms inevitably follow the underlying business model of the search engine.
That is a big part of the disillusionment with Google. For many years they were a leveler which was concerned primarily with quality. That grew the importance of search & differentiated them from everyone else, but then they decided to be "the same" & so many who promoted them felt a bit betrayed.
If a person gives you something and then takes it away you likely view them worse than someone who simply never offered that in the first place. As a species we are biologically aligned with being adverse toward loss.
Vertical AND Horizontal Integration
I was chatting with a friend about the above trend & his responses were:
"you don't shoot the guy that didn't give you the job; you shoot the guy that gave you the job and then fired you"
"their public image as being a leveler becomes more grating too, given how much they no longer represent that"
"the biggest problem we have in search is that search engines don't view themselves as a medium. They want to be the cable operator + television show + in-show advertising + commercials...I'm not aware of another medium where it works that way"
If Google's internal stats show someone else is catching up to a channel they invested in, Google can...
relay this news across to drive editorial quality, content quantity, or even ad placement
preferentially promote the network they are invested in (free ads, better rankings, more "you might also like" recommendations, more post-view recommendations)
give a higher revenue share to the network they are invested in (or offer them early access to new betas and exclusives that increase monetization)
slow the growth of the competing network by using more aggressive ad placement (or lower CPM ads)
slow upload speeds for competing channels
etc.
If you are batting for the home team, such advantages are great. But they blow for everyone else in the ecosystem.
Those sorts of issues don't just appear in a few isolated incidents, but appear over and over again.
Affiliate links should be clearly labeled as such. When they are not clearly labeled & go through tracking redirects they are sneaky redirects in Google's remote rater guidelines. On YouTube the affiliate links to Amazon & iTunes are not labeled as such & add an extra layer of tracking redirects to the sequence.
Let Me See Your Backlinks!
Yesterday someone sent me an email about their reinclusion request being denied because someone else scraped their eZineArticles article & syndicated it to another 3rd party site.
They didn't create that link and yet they are somehow supposed to get a spammer (maybe one from another continent!) to remove it. In many cases spammers won't respond to anything other than cash, but if you do offer cash to get the job done then that spammer might keep adding more and more links over time, turning their mark into an easy source of subscription revenues.
What is Wrong With This Picture?
The above scenario is ridiculous.
If you look at *any* site closely enough there will be something wrong with it.
Just by the virtue of existing & ranking you will pick up dozens to hundreds of spammy links you don't even want, due to SERP scraper sites that are trying to rank on longtail keyword queries.
About 5 years ago I had a page get filtered out because it gained about 500 scraper links in a month. No matter what I did that page would not rank until it was rewrote with a fresh page title. When you could change things & have the algorithms re-evaluate them automatically there was at least a decent opportunity to get around such issues.
Now that there is a manual review process holding you responsible for the actions of third party webmasters the market is a bit more grim.
But at least a bunch of link removal services are cropping up to profit from Google's errant logic. ;)
Engineers Ad Networks Love Quality Websites Big Brands
A bigger company can always shut a site down, split off into sections, & so on. Plus if you are a bigger company you are more likely to enjoy the benefit of the doubt.
But if you are a low margin small business who has seen declining revenue AND have to jump through further hoops (rather than focusing on running your business) at some point it is easier to give up than to keep on fighting.
Eventually a lot of the displacement trends that are hitting the organic search market will hit the paid search market & Google will make many of the enterprise AdWords management tools obsolete via a combination of various free scripts & data obfuscation.
At that point in time some of the paid search folks will look like the guy to the right, but nobody will care, as those same people reminded us that this is just how business works. :D
Google appears to have a culture that condones shamelessly violating consumer privacy. How else can you explain a company that bypasses Apple's iPhone privacy settings in a reported attempt to strengthen advertising revenues?
It is hard to believe that Dave Packard or Andy Grove would ever tell a group of entrepreneurs that he did "every horrible thing in the book to just get revenues right away," or brag to trade publications that his company used behavioral psychologists to design "compulsion loops" into products to keep customers engaged. But Mark Pincus, the founder of Internet gaming giant Zynga, has done just that.
When corporate leaders pursue wealth in the winner-take-all Internet environment, companies dance on the edge of acceptable behavior. If they don't take it to the limit, a competitor will. That competitor will become the dominant supplier -- one monopoly will replace another. And when you engage in these activities you get a different set of Valley values: the values of customer exploitation.
The layout of the result looks something like this
Or if you put it in Google's browser analysis tool, it looks something like this
And with that move, if you are in ecommerce & you don't rank #1 you are essentially invisible to most searchers.
As John Andrews highlighted on Twitter: "Notice Google tells us "paid relationships improve quality" and then penalizes for paid links?"
As always, it is more profitable to follow Google's biz dev team than Google's public relations pablum.
In some cases Google might include 3 or 4 different types of monetization in a search result. In the below search result Google includes:
AdWords ads
Google Offers
Hotel Comparison ads
Hotel Price ads
And those are *in addition* to featuring promotional links to Google Maps & Google+ in the search results. Further, some of these vertical results consist exclusively of paid inclusion & then have yet another layer of PPC ads over the top.
As SEOs we focus a lot of energy on "how do I rank 1 spot higher" but when the organic results are displaced and appear below the fold why bother? The issue of the incredibly shrinking organic result set is something that can't be over-emphasized. For many SEOs the trend will absolutely be career ending.
AdWords, product listing ads, brand navigation, product search, local, etc. A result like this has a single organic listing above the fold & if Google decides to rank their local one spot higher then that turns to zero.
If you look at the new TLD announcement Google applied for .MBA & .PhD (as well as many names around entertainment, family & software). Thus it is safe to say that education will eventually be added to local, video, media, shopping & travel as verticals where Google is displacing the organic results with links to more of their fraternal listings. About the only big categories this will leave unscathed will be real estate, employment & healthcare. However those first 2 are still in contraction during our ongoing depression & Google blew a lot of their health credibility by pushing those illegal ads for steroids from a person posing as a Mexican drug lord.
In addition to these fixed vertical that cover the most profitable areas of search, Google is also building a "vertical search on the fly" styled service with their knowledge graph. Their knowledge graph extracts data from 3rd party websites & then can be used to funnel traffic and revenue to Google's various vertical services. To make it seem legit, Google will often start by sending some of the traffic onto 3rd party sites, but the end destination is no different than product search. While it is a "beta" product it is free to justify an inferior product being showcased front & center, but after Google gets enough buy in they monetize.
There is a non-subtle difference between Google's approach and Microsoft's approach to building a search ecosystem.
Let's compare that behavior against Yahoo! or Bing.
Yahoo! has long been considered out of the search game, yet when they want to have a competitive advantage they do things like license photos from Getty. They use the content with permission on agreed terms.
Google's approach is more along the lines of "scrape it now & figure out legal later." And after a long enough period has passed they will add monetization & mix it into the core of their offering, like they recently did with books:
This launch transitions the billions of pages of scanned books to a unified serving and scoring infrastructure with web search. This is an efficiency, comprehensiveness and quality change that provides significant savings in CPU usage while improving the quality of search results.
Eric Enge interviewed Stefan Weitz about the new Bing interface. As part of that interview, Stefan described Bing's editorial philosophy on building a search ecosystem
We partner with 3rd party services instead of trying to build or acquire them. There are probably something like a million apps out there today.
I talk to probably two dozen start-ups every week that are doing different cool things on the web. To think that we are ever going to be able to actually beat them, or out-execute them (when they are talking about 12 guys with half a million angel funding building some really interesting apps), it is just not likely.
Off the start forays into new categories might provide some value to publishers in order to get buy in, but eventually the "first hit free" stuff shifts to paid & Google continues to displace publishers across more and more of the ecosystem, using content scraped from said publishers.
Funding Scraping
When Google or Apple drive cars around the country or fly military-grade planes over cities to create 3D maps of cities they are creating databases & adding new information. Outside of collecting private data (like wifi payload data) there is little to complain about with that. They are adding value to the system.
However, at the same time, Google not only scrapes themselves, but they are a revenue engine that drives a lot of third party scraping. And they design penalties in a way that allows those who scrape penalized sites to outrank them. With batch penalty updates some folks can chain redirects, expired domains & so on to keep exploiting the combination of copyright violations & Google penalties to make a mint. Google also had a long history of funding Traffic Equalizer sites, sites like Mahalo that would take a copy of a search result & auto-generate a page on it, newspaper sites that would hang auto-generated stub preview articles on subdomains, & sites like eHow which integrate humans into the process.
While many sites are still penalized from the first version of Panda, downstream referrals to eHow.com from Google in the US were up over 9% last month. They know "how to create SEO content."
Recently a start up that launched a couple years ago decided to take their thousands of subdomains of scraped databases & partner with authoritative websites to syndicate that content around the web. Some of those get double listings & for some search queries there is the same page (with a different masthead logo) 5 different times. Those sites don't get hit by duplicate content filters or algorithms like Panda because they have enough domain authority that they get a free pass. Including AdSense in the set up probably makes it more palatable to Google as well.
If you have scale you can even auto-generate a bunch of "editorial" questions off the database.
More data = more pages = more questions and comparisons = more pages = SEO alchemy (especially if you don't have to worry about Panda).
The parent scraper site includes links back to itself on every syndicated page, which to some degree makes it a glorified PageRank funnel. WPMU.org got smoked for syndicating out a sponsored theme on one of his own sites, but the above industrial-scale set up is somehow reasonable because it was launched by a person who sold their first start up to Google (and will likely sell this start up to Google too). The site also includes undisclosed affiliate links & hands out "awards" badges to the best casual encounter sex dating sites, which then get syndicated around the web & get it many inbound links from "high quality" porn sites.
I won't name the site here for obvious reasons, but they are not doing the above in a cloak of darkness that one has to look hard to find & do deep research to patch together. For some search results they are half or more of the search result set & they even put out press releases when they add new syndication partners, linking to numerous new automated subdomains or sections within sites related to various categories.
When the search results look like that, if you do original in-depth reviews that are expensive there is zero incentive structure to leaving your content and ratings open to Google and these sort of scraper/syndicaters.
There is always a new spin on the mash up low end content with high trust websites and try to feed it into Google. So long as Google biases their algorithms toward big brands & looks the other way when they exploit the ecosystem that trend will not end.
The Independent Publishers Group, a principal distributor of about 500 small publishers, recently angered Amazon by refusing to accept the company’s peremptory demand for deeper discounts. Amazon promptly yanked nearly 5,000 digital titles. Small-press publishers were beside themselves. Bryce Milligan of Wings Press, based in Texas, spoke for most when, in a blistering broadside, he lambasted Amazon, complaining that its actions caused his sales to drop by 40 percent.
However, even when companies are brutal in some aspects they do amazing things in other areas, so one has to weigh the good with the bad.
At any point Google can fold one vertical into another or extend out a new model. The Android Marketplace feeds into Google Play, Google local feeds into Google+, Google search force feeds just about everything else & even free offerings on sites like YouTube will eventually become pay to play stores.
Where Google lacks marketshare & forced bundling isn't enough to compete they can buy the #2 or #3 player in the market & try to propel it to #1 using all those other forms of bundling.
Part of what made search competitive against other platforms was its openness & neutrality. But if the search results are Wal-Mart over and over again (or the same scraped info 5 times in a row, or a collection of internal listings) then the system becomes more closed off & the perception of choice becomes an illusion. John Andrews wrote a couple greatTweets expressing the shift in search:
"Google SEO is no longer worth the effort for those who are not writers, artists, speakers, trainers, or promoters. What happened to Search?"
"If you want to see what Google will look like after it locks up, look at Apple. ipad users are already "managed" very tightly."
When companies try to expand the depth of their platform with more features it is a double edged sword. At some point they capture more value than they create and are no longer worth the effort. When they get to that stage it becomes a race to the bottom with scrapers trying to outscrape one another. Then in turn the company that created the ecosystem problem uses the pollution they rewarded to further justify closing off the system, guaranteeing only more of the same. Those who actually add value move on looking for greener pastures.
A label or an interest is a vector for ad targeting. There is no need to worry about de-anonymizing data for ad targeting when it is all in-network and you monitor what someone does, control which messages they see, & track which ones they respond to. Tell someone something often enough and they may believe it is true.
The Contempt Large Companies Have for their Customers
There is a sameness to customer service from a lot of big companies. They spend loads & loads to track you and market to you, but then disappear the moment things go wrong, as they are forbidden to care.
Perhaps the only thing worse that AOL's customer support is the unmoderated comments on the YouTube page.
Denise Griffin, the person in charge of Google’s small customer-support team, asked Page for a larger staff. Instead, he told her that the whole idea of customer support was ridiculous. Rather than assuming the unscalable task of answering users one by one, Page said, Google should enable users to answer one another’s questions.
Even their official blog posts announcing that they are accepting customer feedback for your applications go unmoderated.
This sort of contempt exists at essentially all large companies.
Everything seems on the up & up, but that "private listing" was maybe for a counterfeit product.
If it isn't a counterfeit & you get too good of a price you are threatened with a lawsuit, and the branded network falls behind a "oh we are just a marketplace and can't be bothered to give a crap about our customers" public relations angle.
The Retina MacBook is the least repairable laptop we’ve ever taken apart: unlike the previous model, the display is fused to the glass—meaning replacing the LCD requires buying an expensive display assembly. The RAM is now soldered to the logic board—making future memory upgrades impossible. And the battery is glued to the case—requiring customers to mail their laptop to Apple every so often for a $200 replacement. The design may well be comprised of “highly recyclable aluminum and glass”—but my friends in the electronics recycling industry tell me they have no way of recycling aluminum that has glass glued to it like Apple did with both this machine and the recent iPad. The design pattern has serious consequences not only for consumers and the environment, but also for the tech industry as a whole.
...
Every time we buy a locked down product containing a non-replaceable battery with a finite cycle count, we’re voicing our opinion on how long our things should last. But is it an informed decision? When you buy something, how often do you really step back and ask how long it should last? If we want long-lasting products that retain their value, we have to support products that do so.
One last bit of absurdity on the YouTube front. Google recently threatened to sue a site designed to convert YouTube videos into MP3s.
How does Google's "computers deserve free speech rights" & shagging 3rd party content to fill out their own vertical search services compare against their approach when someone uses YouTube content in a way Google does not desire?
There are AdWords ads promoting free unlimited MP3 downloading & song burning bundled with shady adware.
Google's AdSense for domains funds boatloads of cybersquatting. While Google threatened to sue this particular site, they could have just took the domain due to it cybersquatting on the YouTube trademark. The fact that they chose to turn this into a press event rather than simply fix the issue shows that this is more for posturing.
Further aligned with the above point, while Google singled out a specific MP3 conversion site, there are other sites designed around doing the same exact thing which are PREMIUM ADSENSE PARTNERS, with the body of the page looking like this:
How Small Companies Are Taxed With Uncertainty
When Google decided to move away from direct marketing to brand advertising things that are often associated with size, scale & brand recognition became relevancy signals.
how much to invest in marketing, where to invest it, how to balance the need for short term cashflow with the required reinvestments to build real (or fake) brand signals
how long does the market have left before Google enters the niche and destroys the opportunity that organic SEO once represented
should you run 1 website, or many to hedge risks? and how many is optimal?
how big should your site be?
if one of your sites gets penalized, should you try to fix it up, should you start over with a new site, or should you consider SEO to be a pointless goal?
Google mentions that they want people to do what is best for the user & not worry about Google, but that advice is a recipe for pain
If you do not run a large & authoritative website there are so many landmines to trip over with the increasing complexity of SEO. And any of Google's "helpful" webmaster messages can suspend a webmaster in fear, leading them to an eventual bankruptcy.
Small companies need to do all sorts of canonicalization hoops & prune content and such to hope to avoid algorithms like Panda. Then Google changed their host crowding preferences to let some large sites get up to 8 listings in a single search result page for their LACK OF effort. Those larger sites can then partner with glorified scraper sites that syndicate databases feeding on domain authority with no risk of Panda.
Due to how Google penalizes smaller sites, those that rewrite their content will outrank them when they get hit. These horrible trends are so obvious that even non-SEOs like Tim Carter (who was a Google golden boy for years) highlights how the tables have tilted away from what is most relevant to what pays Google the most.
I am already getting fake webmaster tool notification messages using the above subject line & the following message:
Hello dear managers of http://www.seobook.com/! My name is Olivia, and the issue I’m gonna to discuss is for sure not new, but really actual and complicated, otherwise your website and therefore business wouldn’t have lost their favourable positions. Yes, I want to talk about Google Panda and Penguin. These virtual beasts become more and more freakish. Don't you think it's time to pacify them? Google intends to clean its search results from poor content websites, low quality links and hype. Are you sure your website has nothing common with this stuff?
Our team has been constantly studying Google search algorithms. We have already faced the latest freaks of Google Panda 3.4 and will be happy to win back your top positions.
We will heal your website from:
poor on page optimization;
same content submission;
low quality links to your website;
absence of website moderation;
black hat SEO applied earlier.
We will make Google be proud of you with:
high quality SEO strategy;
backlinks from relevant resources;
quality SMO;
links diversity;
unique content for every submission directory;
constatnt situation analysis and reporting.
Contact us and you will get a reliable website healer, strategy planner and safe guard of your top positions.
Looking forward to your answer!
And Gmail is letting this stuff slide through the spam filters. Along with garbage like this:
Our Web Site [the url] is definitely related to yours and by placing a link from your site to a Web page of ours, you may not only bring further value to your visitors but you may improve your search engine rankings potential as well. By NOT being what Google and other search engines refer to as a "dead-end" site or a site that does not link to other industry related and content sites, your rankings have a good chance of increasing for important keyword searches. We can explain this in further detail following a response from you.
Create FUD & some huckster will sell into your messaging with inbound spamming.
If you ever wonder where the "reputation problem" of the SEO industry comes from, wonder no more.
One company in particular does a great job of riding these trends on through to their logical conclusion, then riding them a bit longer. And that company is Google.
Google recently launched their webspam Penguin update. While they claim it only impacted about 3.1% of search queries, the 3.1% it impacted were largely in the "commercial transactional keywords worth a lot of money" category.
Based on the number of complaints online about it (there is even a petition!) this is likely every bit as large as Panda or the Florida update. A friend also mentioned that shortly after the update WickedFire & TrafficPlanet both had sluggish servers, yet another indication of the impact of the update.
Spam vs OOP
Originally leading up to the update, the update was sold as being about over-optimization. However when it was launched it was given no pet name, but rather given the name of the webspam update. Thus anyone who complained about the update was by definition a spammer.
A day after declaring that the name didn't have any name Google changed positions and called the update the Penguin update.
Why the quick turn around on the naming?
If you smoke a bunch of webmasters & then label them all as spammers, of course they are going to express outrage and look for the edge cases that make you look bad & promote those. One of the first ones out of the gate on that front was a literally blank blogspot blog that was ranking #1 for make money online.
As I joked with Eli, if it is blank then they couldn't have done anything wrong, right? :D
Another site that got nailed by the update was Viagra.com. It has since been fixed, but it is pretty hard for Google to state that the sites that got hit are spam, blend the search ads into the results so much that users can't tell them apart & force Pfizer to buy their own brand to rank. If that condition didn't get fixed quickly I am pretty certain it would lead to lawsuits.
Google also put out a form to collect feedback about the update. They only ever do that if they know they went too far and need to refine it. Or, put another way, if this was the Penguin update then this is GoogleBot:
So Worried About Manipulation That They Manipulate Themselves
When I was a kid I used to collect baseball cards. As the price of pictures from sites like iStockphoto have gone up I recently bought a few cards on eBay (in part for nostalgia & in part to have pictures for some of our blog posts). Yesterday I searched for baseball card holders for mini-cards & in the first page of search results was:
a big ecommerce site where the review on that product stated that the retail described the quantity as being 10x what you actually get (the same site had other better pages)
a user-driven aggregator site with a thin affiliate post made years ago & attributed to a site that no longer exists
a Facebook note that was auto-generated from a feed
an old blogspot splog
a broader tag page for a social site
a Yahoo! Shopping page that was completely empty
That blank Yahoo! Shopping page is also what showed up in Google's cache too. So I am not claiming that they were spamming Google in any way, rather that Google just has bad algorithms when they rank literally blank pages simply because they are on an authoritative domain name.
The SERPs lacked expert blogs, forum discussions, & niche retailers. In short, too much emphasis on domain authority yet again.
Part of the idea of the web was that it could connect supply and demand directly, but an excessive focus on domain authority leads users to have to go through another set of arbitragers. Efforts to squeeze out micro-parasites has led to the creation of macro-parasites (and micro-parasites that ride on the macro-parasite platforms).
And your business model is probably far more important than your SEO skill level is. Imagine running a consulting company for a lot of small business customers for a few hundred Dollars a month each, based on stable rankings & then dealing with a tumultuous update that hits a number of them at the same time. And then they see an older (abandoned even) competing site of lower quality with fewer links ranking and they think you are selling them a bag of smoke. These sorts of updates harm the ability to do SEO consulting for anyone who isn't consulting the big brands. Yes many people made it through this update unscathed, but how many of these sorts of updates can one manage to slide through before eventually getting clipped?
The Unknowable Future
As search evolves, invariably anyone who is doing well in the ecosystem will at some point face setbacks. Those may happen due to an algorithm update or an interface change where Google inserts itself in your market. If you never get hit, it means you were only operating at a fraction of your potential. If you consistently get hit, you might be aiming too low. Many trends can be predicted, but the future is unknowable, so set up a safety cushion when things are going well.
This year Google has moved faster than any year in their history (massive link warnings, massive link penalties, tighter integration of Panda & now Penguin) & the rate of change is only accelerating. Go back about 125 years and a candle wick adjuster was cutting edge technology marketed as brand spanking new:
Blekko has a decently competitive search service which they manage to run for only a few million a year. As computers get cheaper & Google collects more data think of all the different data points they will be able to layer into their relevancy algorithms. In some markets Chrome has more marketshare than Internet Explorer does & Android is another deep data source. And they can know what user data to trust most by tracking things like if they have a credit card or phone verified on file & how often they use various services like Gmail or YouTube. Google+ is just icing on the cake.
You have to play by their rules, which are really restrictive. The kind of environment that we developed Google in, the reason that we were able to develop a search engine, is the web was so open. Once you get too many rules, that will stifle innovation.
He was talking about Facebook, but those words are far more applicable to Google.
A Social Experiment
In the movie Dark Knight the Joker ran a social experiment where he offered 2 boats full of people the opportunity to save their own lives by blowing up the other boat. The boat full of "criminals" threw the button overboard & the other boat also decided not to push the button.
Of course taking someone's life is more extreme than taking their livelihood, but if you do the latter it might create stress and/or other issues which in effect lead to the former. Some people who see their income disappear might have a heart attack, others might have marriages that soon falls apart, leading into a spiral of depression and substance abuse & eventually suicide. Others still might have employees that get laid off & end up heading down some of the same scary paths - through no fault of their own.
Negative SEO Goes Mainstream
Anyone who outs or link bombs smaller businesses (small enough that Google punishing them destroys their livelihood rather than just giving them a bad quarter) is a _______. Anyone who advocates outing or link bombing such businesses is an even larger _______.
Why?
With all of Google's warning messages about abnormal links they have built the negative SEO industry in a big way. In some instances those who are not good enough to compete try to harm competitors. I received emails & support tickets like the following one for years and years...
...but the rate of demand increase for such "services" has been sharp this year. Every additional warning message from Google creates additional incremental demand.
And this is where outing a competitor makes one a total and complete _______ of a human being.
A Recent (& Very Public) Example of Negative SEO
Dan Thies mentioned that it was "about time" that Google started hitting some of the splog link networks.
Anyone who knows the tiniest bit about the social sciences could predict what came next.
15th March - Dan Thies posts smug tweets to Matt Cutts and pisses off the entire internet.
18th March - seofaststart.com - blog posts started - anchor text "seo" "seo service" and "seo book"
22th March - seofaststart.com - 1 million scrapebox blast started - 100% anchor text "Dan Thies"
26th March - Dan Thies posts in Twitter that he has received an unnatural links message.
Since then Dan has installed a new template & his rankings tanked. Is it the template or the spam links? Probably the spam links, given how many other sites have got hit for using too much focused anchor text.
Will the site stay tanked? If so, now Google's approach to anchor text & link spikes allows independent websites to get torched in a few weeks for a few Dollars.
Or will the site come back stronger than ever with the help of the spam links? If it does, then how long is it before people start accidentally spam blasting their own websites & posting a public case study about burning a competitor on a forum, then citing that forum thread in their reconsideration request?
If the site quickly comes back, will that be due to a manual intervention by a search engineer, or from an algorithm more advanced than some people are giving it credit for being?
When asking such questions one quickly arrives at another set of questions. Is it the web that is broken? Or is it Google's editorial approach that is broken? If the observer breaks the system they observe, then the observer is the problem.
The Bigger Issue
The bigger issue isn't the short term trends for SEO related keywords or Dan's site (he will be fine & rankings are not that important for sites about SEO), but the big issue is that if this can happen to a decade old website then this can happen to literally anybody.
Piss off a ...
competitor
SEO
web designer
web developer
business partner
blogger
blog reader
former customer
freetard
ex-friend
bitter family member
insert any classification or category you like
etc.
... and risk getting torched.
When you out someone for shady links, you can't be certain they were responsible for it. They could have had a falling out with a consultant or business partner or another competitor who wanted to hose them. Or their SEO or webmaster could have been non-transparent with them.
Then you out them & they might be toast.
White Hat, Black Hat & ________ Hat SEO
Any of the ________ who promote competitor smoking or competitor outing as somehow being "ethical" or "white hat" never bother to explain what happens to YOU when someone else does that to you.
Sketchy marketers can make just about anything look good at first glance. No matter how shiny the package in concept, it is hard to appreciate the pain until you are the one undergoing it.
Building things up is typically far more profitable than tearing things down & if SEOs go after each other then the only winner is Google. Literally every other participant in the ecosystem has higher risk, higher costs & is taxed by the additional uncertainty. Sure some of the conscripts might get a bit of revenues and some of the "white hat" hacks might gain incremental short term exposure, but as the marrow is scraped out of the bone, they too will fall hard.
Google is betting that the SEO industry is full of ________. If our trade is to worth being in, I hope Google is wrong! If not, you will soon see most of the quality professionals in our trade go underground, while only the hacks who misinform people & are an unofficial extension of Google's public relations team remain publicly visible.
That might be Google's goal.
Will they be successful at it?
That depends entirely on how intelligent members of the SEO industry are.
A bigger company can always 
