At SMX I gave a presentation on brand & how Google has biased the algorithms toward brands. having already seeing the bulk of my argument months prior, Bryson Meunier spoke after me and put together a presentation that used bogus statistics & was basically a smear of me. He was so over the top with his obnoxious behavior that when Danny Sullivan mentioned the next speaker after him he jokingly said "up next, Ron Paul."
I honestly thought the point of the discussion was to highlight how Google has (or hasn't) biased the algorithms, editorial policies & search interface toward brands. However, if a person speaks after you and uses bogus statistics to reach junk conclusions, you can't debunk their aggregate information until after you have looked into it some. An honest person can put what they know out there & share it publicly in advanced, a dishonest person hides behind junk research and the label of science to ram through poorly thought out trash, collecting whatever "data" confirms their own bias while ignorning the pieces of reality that don't.
As an example, he suggested that based on the number of employees and revenues Wikipedia is a small business. He then went on to say that since Wikipedia wasn't on Interbrand's "scientific" study that they were not a top brand. Nevermind that no countries, religions, sports, celebrities, or non-profits make the list of top "companies."
After IAC figured out that they were able to get away with running Ask.com as a thin scraper site, they outsourced "the algorithm" and fired many of their employees. Because they have fewer employees, Bryson considers Ask as "a mid-sized business" even though they are part of a multi-billion Dollar company and IAC is Google's #1 advertiser!
According to Compete's downstream traffic stats, YouTube receives about 1 in 13 search clicks from Google, but since it wasn't on Interbrand's list "who cares?" Incidentally, the folks at Interbrand do have a mention of YouTube on their top 100 brands page, but it was a suggestion that you watch their videos on YouTube. Their methodology is so suspect that Goldman Sachs and Yahoo! made the cut while YouTube didn't, even though YouTube is one of their few offsite promotional channels they promote on that very page. Their list also puts Microsoft's brand value at about double Apple's (and the list came out when Steve Jobs was still alive).
Bryson also claimed that since big brands are inefficient and slow moving they already have a big disadvantage so it makes sense for search engines to compensate for that. That is at best an illegitimate line of reasoning because those companies have plenty of solutions available to them & have the capital needed to buy out competitors. Even when the SERPs look independent, a lot of the listed sites are owned by large conglomorates. As an example, here is a random search from earlier today:
Meanwhile the same idiotic logic ignores the lack of resources at small businesses. Nowhere in his presentation was a highlight of how Google favored affiliates & direct marketers until the profit margins of the direct response marketing model started to peak & then Google transitioned to promoting brands, as they wanted to keep increasing revenues and monetize more clicks.
Bryson also shared an example of where he got a photo sharing site 40,000 unique visitors a month as a case study of the power of white hat SEO. 40,000 monthly visits to a photo sharing site might fund a light Starbucks addiction (assuming you value your time at nothing, have no employees, ignore hosting costs and the SEO is free), but not much beyond that. If that is a success case study, that shows how much harder the ecosystem is getting to operate in as a small business.
He also put out a painfully fluffy "white paper" / sales letter which stated that since Wal-Mart has a page about SEO they should outrank seobook on "SEO" related queries if my theories of brand bias are correct. That misses the point entirely. I never stated that garbage content on branded sites always outperforms quality content on niche sites, but rather that a lot of smaller websites were intentionally being squeezed out of the ecosystem. Sure some small sites manage to compete, but the odds of them succeeding today are much lower than they were 3 or 4 years ago.
At SMX near the end of our session a question was asked about the audience composition & most people were either big brands or people working for big brands. If you go back to when I first got into SEO in 2003 the audience composition was almost entirely small publishers and independent SEOs. This squeezing out of small players is not something new to search or the web. If you look at the history of any modern communications network this cycle has repeated itself in every single medium - phone, radio, television, and the web.
To be fair, I can understand why a no-name also ran SEO consultant would want to pitch himself for being up for doing SEO work for large brands. Brands generally have fatter margins, economies of scale, and large budgets. As Google tilts the algorithm toward the big brands (to where they can fall over the finish line in first place) they are the best clients to work for, since you are swimming downstream.
Why push huge boulders up the side of the mountain for crumbs when you can get paid far more to blow on a snowflake at the top of the mountain?
That is why so many SEOs fawn over trying to get brand clients. The work is high-paying, low risk, and relatively easy.
If we were ever to close up our membership site & focus primarily on SEO consulting work in more structured arrangements then absolutely we would aim at brands & help them fall over the finsh line in first place. ;)
Back when I worked with Clientside SEM we did a good number of big brand projects with some of the largest online portals & retailers. Understanding the business objectives & communicating things in a way that builds buy in from other departments is of course challenging. You need simplicity & directness without oversimplifying. But (if you work for great clients - like we did), then that is nowhere near as challenging as building a site from scratch into something that can compete for lucrative keywords. I recently stepped back from the client consulting model for a bit simply because I was pulling myself in too many directions & working too long, but Scott is still flourishing & delivering excellent results for clients.
I have nothing against the concept of branding (think of how many years I slaved building up this site & the capital I have poured into it), but I like to share the trends in the ecosystem as they are, rather than as a hack warping my view to try to pick up consulting clients. Our site would likely make far more income if we kept using the words "enterprise" "brand" "fortune 500" and then sold consulting to that target audience. In fact, a large % of our members here are fortune 500s, conglomerates, newspaper chains, magazine publishers, and so on.
It is not that brand counts for nothing (or that it should count for nothing) but anyone who claims the table isn't tilted is either ignorant, a liar, or both.
Truth has to count for something.
Disclaimer: I am not saying enterprise SEO is always easy (there are real challenges, especially with internal politics that add arbitrary constraints). And I am not saying that everyone who targets the enterprise market is a hack (there are some super talented folks out there). But the challenge of being a profitable small webmaster is much more of a struggle than ranking a site that Google is intentionally biasing their algorithms toward promoting.
Disclaimer2: I realize refuting a douchebag like Bryson Meunier is batting below my league, however as a matter of principal I won't let sleazeballs get away with taking a swipe using junk science. The word science deserves better than that.
It doesn't matter what "signals" Google chooses to use when Google also gets to score themselves however they like. And even if Google were not trying to bias the promotion of their own content then any signals they do collect on Google properties will be over-represented by regular Google users.
Google can put out something fairly average, promote it, then iterate to improve it as they collect end user data. Publishers as big as MotorTrend can't have that business model though. And smaller publishers simply get effectively removed from the web when something like Panda or a hand penalty hits them. Worse yet, upon "review" search engineers may choose to review an older version of the site rather than the current site!
With that level of uncertainty, how do you aggressively invest in improving your website?
Over a half-year after Panda launched there are few case studies of recoveries & worse yet, some of the few sites that recovered just relapsed!
If you look at search using a pragmatic & holistic view, then this year the only thing that really changed with "content" farms is you can now insert the word video for content & almost all that video is hosted on Youtube.
To highlight the absurdity, I created another XtraNormal video. :)
Compete.com's Google downstream search traffic stats are available with a premium membership to their site, & they do a good job of showing the actual traffic impact of the aggregate algorithmic changes. YouTube's growth is also well reflected in numbers from firms like SearchMetrics
“A private understanding was reached between the OPA and Google,” an office assistant with e-mail evidence told Politically Illustrated. “The organization is responsible for coordinating legal and legislative matters that impact our members, and one of the issues was applying pressure to Google to get them to adjust their search algorithm to favor our members.”
At the same time, said "premium publishers" were backfilling their websites padding them out with auto-generated junk created by companies like Daylife, where some of the pages offer Mahalo-inspired 100% recycled content.
My suspicion is that Google did not care about the auto-generated "news" garbage for a number of reasons
it helps subsidize the big media interests
they don't want to hit big media & cause a backlash
it is quite easy for Google to detect & demote whenever they want to
it gives Google more flexibility going forward when deciding how to deal with issues (if everyone is a spammer then Google has more flexibility in deciding how to handle "spam" to maximize their returns.)
It is the exact same reason that Google says link buying is bad, while tolerating "sponsored features" sections on large newspapers:
The leaders of Narrative Science emphasized that their technology would be primarily a low-cost tool for publications to expand and enrich coverage when editorial budgets are under pressure. The company, founded last year, has 20 customers so far. Several are still experimenting with the technology, and Stuart Frankel, the chief executive of Narrative Science, wouldn’t name them. They include newspaper chains seeking to offer automated summary articles for more extensive coverage of local youth sports and to generate articles about the quarterly financial results of local public companies.
Official sources using "automated journalism" is a perfect response to Google's brand-focused algorithms:
Last fall, the Big Ten Network began using Narrative Science for updates of football and basketball games. Those reports helped drive a surge in referrals to the Web site from Google’s search algorithm, which highly ranks new content on popular subjects, Mr. Calderon says. The network’s Web traffic for football games last season was 40 percent higher than in 2009.
How expensive cheap is that technology?
The above linked article states that "the cost is far less, by industry estimates, than the average cost per article of local online news ventures like AOL’s Patch or answer sites, like those run by Demand Media."
And the exposure earned by the machine-generated content will be much greater than Demand Media gets, since Demand Media was torched by the Panda update AND many of the sites using this "algorithmic journalism" were given a ranking boost by Google due to their brand strength.
The improved cost structure for firms employing "algorithmic journalism" will evoke Gresham's law. This starts off on niche market edges to legitimize the application, fund improvement of the technology & "extend journalism" but a couple years into the game a company that is about to go under bets the farm. When the strategy proves a winner for them, competing publishers either adopt the same or go under.
That is the future.
Across thousands of cities, millions of topics & billions of people.
Even More Corporate Boosts
Just because something is large does not mean it is great across the board. Businesses have strengths and weaknesses. Sure I do like love shopping on eBay for vintage video games, but does that mean I want to buy books from eBay? Nope.
Likewise, Google's friend of a friend approach to social misses the mark. Do I care that someone I exchanged emails with is a fan of an athlete who promotes his own highlight reels? No I do not.
In a world where machine generated journalism exists, I might LOVE one article from a publication while loathing auto-generated garbage published elsewhere on the same site.
Line Extension & "Merging Without Merging"
At Macworld in 2007 Eric Schmidt said "What I liked about the new device and the architecture of the Internet is you can merge without merging. Each company should do the absolutely best thing they can do every time, and I think he's shown that today."
If you don't have the ability to algorithmically generate content to test new markets then one of the best ways to "merge without merging" is to sell traffic to partners via an affiliate program.
In our free SEO tips we send new members I recommend setting up AdWords and adCenter accounts to test traffic streams, so that you have the data needed to know what keywords to target. But affiliates need not apply:
Hello Aaron Wall,
I just signed up for the Get $75 of Free AdWords with Google Adwords. After receiving an e-mail stating that I was to call an 877 number of Google Adwords, I was told in my phone call that affiliate marketing accounts were not accepted. I guess I confused by this statement. Is this in error? Or am I not understanding the Tip #3 for setting up an account for Google Adwords for promoting a website?
Thank you in advance for your time.
The same Google which allows itself to shamefully carry a "get rich quick" AdSense category considers affiliate marketing unacceptable.
Non-AdSense Affiliates Classified as Doorway Pages, Not Welcome in the Organic Search Results?
The exact same thing is happening in the organic search results right now. Maybe not on your keywords & maybe not today, but if you are an affiliate, the trend is not your friend. ;)
I have heard recently from multiple friends that some of their affiliate sites were penalized for being doorway & bridge pages. At the same time, another friend showed me some BeatThatQuote affiliates ranking thin websites.
Larger companies like BankRate can run a half-dozen credit card affiliate websites & an affiliate network. And they can create risk-adjusted yield by buying out smaller competitors, largely because Google won't penalize them based on the site being owned by a fortune 500. However the independent affiliate is forced to sell out early due to the risk that Google can arbitrarily decide they are a doorway site at anytime.
The absurd thing is that if independent webmasters don't include revenue generation in their website then they don't have the capital *required* to invest in brand & further improving their website. How do you compete against automated journalism when Google gives the automated content a ranking boost? And if you want to do higher quality than the machine generated content, how do you hire employees if you are not even allowed to monetize?
I suppose there is AdSense.
Even though AdSense publishers are Google's affiliates they are still welcome to participate in Google's ecosystem.
Risks to Small Businesses
Small businesses not only have to compete against algorithmic journalism, Google's algorithmic bias toward brands, arbitrary "doorway page" editorial judgements cast against them by engineers & significant algorithm changes, but they also have to deal with loopholes Google leaves in the system that allow them to be arbitrarily removed from the ecosystem.
The big issue Google is facing on the content quality front is the incentive structure. They have got that wrong for a long time now. They may think that these big changes are motivating people to improve quality, but realistically the lack of certainty is prohibiting investment in real quality while ramping investment in exploitation.
How can anyone invest deeply over the long term in a search ecosystem where Google...
Google would spin Performics out of DoubleClick, and sell it to holding firm Publicis.
Only one major force inside of Google hated the plan. Guess who? Larry Page.
According to our source, Larry tried to sell the rest of Google's executive team on keeping Performics.
"He wanted to see how those things work. He wanted to experiment."
The problem with that is that most honest economic innovation (eg: not just exploitation) comes from small businesses. Going into peak cheap oil where food riots are becoming more common & pensions are about to blow up, we need the kings of information to encourage innovation, rather than relying on doing whatever is easy & trusting established old leaders while retarding risk taking from (& investment in) start ups.
In some markets being successful means staying small, building deeper into a niche, and keep adding value until you have a strong position. However some ecommerce sites that were not associated with big brands were torched by the Panda update.
Betting on Brand
As Google has tilted their algorithm toward brand, some ecommerce companies that focused on winning relevant niches are now watering down their competitive advantages by betting the company on brand:
CSN Stores is today consolidating its 200+ shopping sites into a single ecommerce website under one brand: Wayfair.com.
So why the change to Wayfair.com? Primarily for obvious branding reasons: the company has long been spending a huge amount of money on marketing a lot of separate websites, and now they can focus on advertising just one.
Other reasons for the consolidation of the separate shopping site are search engine optimization – which was apparently much needed after Google’s recent Panda update – and the fresh ability to make recommendations to shoppers based on their collective purchase history.
But, as some brands abuse Google the same way the content farms did, is that a good bet? I don't think it is.
What is so Bad About Content Farms?
headline over-promises, content under-delivers
written by people who are often ignorant of what they are writing about
add nothing new to the ecosystem, just a dumbed-down reshash of what already exists
done cheaply & in bulk, in a factory-line styled format
contains frequent spelling and grammatical errors
primarily focused on pulling in traffic from search engines
exists primarily to promote something else (ads or the above-the-fold ecommerce product listings)
etc. etc. etc.
Such behavior is *not* unique to the sites that were branded as content farms & is quickly spreading across fortune 500 websites.
Big Brands Become Content Farms
A friend sent me an email which highlighted how a well-known brand was ordering thousands of pieces of "content" in bulk for their branded site.
Here is the email, with blurring to protect the guilty.
The only difference between the "content farms" and the branded sites engaging in content farming is the logo up in the top-left corner of the page. The business process from how the content is created, to who it is created by, to what they are paid to create it, to the interface it is ordered through, on to how it is published is exactly the same.
Many of the same authors who had some of their eHow "articles" deleted are now writing dozens of "articles" for fortune 500 websites.
Then I expected it would likely take a couple years to go mainstream.
But with the economy being so weak (and back in yet another undeclared recession, actually honestly never having left the last one) this shift only took 6 months to happen. At this point I expect it to spread quickly, especially as the economy gets worse. The above fortune 500 company is one that got a strong boost from Panda & as their downstream traffic from Google picks up over the next month or 2 you can expect many of their competitors to copy the strategy.
This isn't a US-only phenomena. A community member sent me the following, from another fortune 500 company.
Now that fortune 500s are doing almost everything that smaller players could do (but with more capital, more scale, more algorithmic immunity, requiring smaller players to link to them to be listed & in some cases while replacing humans with algorithms) AND get the Google brand boost the future is growing more uncertain for independent webmasters that lack brand, relationships, and community.
Big brands are basically pushed across the finish line while smaller webmasters must run uphill with a 80 pound backpack full of gear - in ice & snow, naked, while being shot at. What's worse, is that brands are now being bought, sold & licensed - just one more tool in the marketer's toolbox (presuming you have the cash).
disclaimer: I am not saying that all content farming is bad (I am fairly agnostic...if it works & people like it, then it works), but the above trend highlights the absurdity of Google's notion of whether something is spam based not on the offense, but rather who is doing it, especially as big brands just quietly turned into content farms.
Vampires have often found it advantageous to maintain a hidden presence in humanity’s most powerful institutions. In the 1600s, it was the Catholic church, and today, as you all know, it’s Google, Fox News.
Trusting a powerful authority is easy. It allows us to have a quick shorthand for how things work without having to go through the pain, effort, & expense to figure things out. But it often leads to bogus solutions.
This video does a great job of explaining how nothing replaces experience in the SEO industry.
A combination of numerous parallel projects, years of trial and error experience & a deep study of analytics data is far superior to having the God complex & feeling 100% certain you are right.
Change is the only constant in SEO.
Big plans often get subverted before they pan out & the more obvious something is the shorter its shelf life. By the time everyone notices a trend then jumping on it at that point probably isn't much of a competitive advantage. You might still be able to make some money for a limited time (or for a longer time if you apply it to new markets), but...
even the current "brand" trend that is in place now will peak out within the next 24 to 36 months. (more on that in a future post)
It is the contrarian investors who are taking (what is generally perceived to be) big risks who are allowed to ride a trend for years and years.
Options & Opportunities
When Panda happened a lot of theories were thrown out as to what happened & how to fix it. Anyone who only runs 1 website is working from a limited data set and a limited set of experience. They could of course decide to do everything, but there is an opportunity cost to doing anything.
Making things worse, if they have limited savings & no other revenue producing websites there are some risks they simply can't take. They can still sorta infer some stuff from looking at the search results, but those who have multiple sites where some were hit and others were not know intimately well the differences between the sites. They also have cashflow to fund additional trial and error campaigns & to double down on the pieces that are working to offset the losses.
Success Requires Failure
A lot of times people want to enter a market with a grand plan that they can follow without changing it once the map is made, but almost anyone who creates something that is successful is forced to change. Every year in the United States 10% of companies go under! And due to the increased level of competition online it likely separates winners from losers even faster than in the offline world. Those who stick to a grand plan are less able to keep up with innovation than those who have an allegiance to the data. Sometimes having a backup plan is far more important than having a grand plan.
Incremental Investing, Small & Large
Almost anything that I have done that has been successful has started ugly & improved over time. This site was an $8 domain & I couldn't even afford a $99 logo for it until I was a couple months into building it. Most of our other successes have been that way as well. If something works keep reinvesting until the margins drop. But when the margins do drop off, it is helpful to have another project you can invest in, such that you are not 1 and done.
The earliest Google research highlighted how ad-based search business models were bad & the now bankrupt Excite.com turned down buying Google for under $1 million. It turns out everyone was wrong there. One company adjusted & the other is bankrupt.
Overcoming the God Complex
We don't control Google. We can only influence variables that they have decided to count. As their business interests and business models change (along with the structure of the web) so must we.
When Google began speaking publicly about content farms Demand Media's Richard Rosenblatt stated that it would be silly to call their stuff a content farm & he emphasized the quality of their content & care that went into it. Of course, those who bothered looking at the content often saw something different
Panda II Hits Demand Media
When Google did the global roll out of Panda earlier this month, they also modified their approach to core Panda algorithm to include user block data:
Today we’ve rolled out this improvement globally to all English-language Google users, and we’ve also incorporated new user feedback signals to help people find better search results. In some high-confidence situations, we are beginning to incorporate data about the sites that users block into our algorithms. In addition, this change also goes deeper into the “long tail” of low-quality websites to return higher-quality results where the algorithm might not have been able to make an assessment before. The impact of these new signals is smaller in scope than the original change: about 2% of U.S. queries are affected by a reasonable amount, compared with almost 12% of U.S. queries for the original change.- Amit Singhal
While many of Demand Media's sites got dinged in the first update, the fall of content farms in general meant that any site operating in that space which was not hit ended up seeing a sharp increase in traffic (as so much of the competition fell). As sites like AnswerBag and Livestrong fell, eHow's traffic increased significantly. I believe Google didn't want to rely on end user block data because it would make it easy for people to do competitive sabotage, however I think they needed to use it in order to hit eHow with the update. eHow had a number of signals (some older quality content, nice web design, syndication partnerships, tons of media exposure, etc.) which made it hard to whack it without creating too much collateral damage unless the block data was used.
In the first two weeks of January, 0.57 percent of those who departed Google next visited a site operated by Demand Media ... by mid-April, with the full suite of Panda updates in place, Demand was feeling the pain. As of April 16, it accounted for only 0.34 percent of Google’s downstream, a 40 percent decline from the start of 2011.
Demand Media's Stock Falls 40%
Incidentally, over the past couple weeks Demand Media's stock is off roughly 40%
That article (which claimed eHow to be profitable as hell, a fuzzy claim depending on how one accounts for content depreciation) was aimed at trying to position Demand for an IPO and to try to pull in more media syndication partnerships.
What it did was inflame the web community & encourage others to play the same game & create content farms based on the blueprint Demand gave away. When a piece of marketing either pisses off almost everyone & encourages many of the people who are not pissed off to compete directly against you & cut your margins it is not a successful marketing approach.
A reason it was so easy for journalists to claim bad things about Demand Media was that the wages were so low that they didn't practically allow for any in-depth research to be done (unless a person was willing to work far below minimum wage). Thus when journalists started to dig into eHow's business model they got eHow writers to state things like:
"I was completely aware that I was writing crap," she said. "I was like, 'I hope to God people don't read my advice on how to make gin at home because they'll probably poison themselves.'
"Never trust anything you read on eHow.com," she said, referring to one of Demand Media's high-traffic websites, on which most of her clips appeared
The larger your scale is the easier it is to find something wrong with what you are doing. 1% of a really big number is much greater than 10% of a rather small number. If you are cutting corners & operating at scale & create a lot of enemies then I wish you the best of luck, because you are going to need it!
In spite of letting a few things fall through the cracks, to this day there are some OUTRAGEOUS eHow titles. A friend showed me a couple and after 5 minutes of searching I found:
How to Pick Your Nose The Proper Way ehow.com/how_5722363_pick-nose-proper-way.html
How to Pick Your Nose or Scratch Surreptitiously ehow.com/how_2181862_pick-nose-scratch-surreptitiously.html
How to Effectively Pick Your Nose ehow.com/how_5067366_effectively-pick-nose.html
Exploring Other Orifaces
How to Fart ehow.com/how_2151823_fart.html
How to Stop Farting ehow.com/how_4785860_stop-farting.html
How to Muffle a Fart ehow.com/how_2320127_muffle-fart.html
How to Poop in the Woods ehow.com/how_2179463_poop-woods.html
How to Not Get an Ehow Article Erased ehow.com/how_5570908_not-ehow-article-erased.html
How to Slack at Work (and not get caught) ehow.com/how_4522164_slack-work-not-caught.html
How to Slack Off at Work and Not Get Caught ehow.com/how_4837878_slack-off-work-not-caught.html
How to Do Nothing at Work and Still Get Paid ehow.com/how_4430256_do-nothing-work-still-paid.html
Honing Your Social Graces & Charm School
How to Manipulate People to do Your Bidding ehow.com/how_2167832_manipulate-people-do-bidding.html
How to Get a DUI ehow.com/how_4825159_get-a-dui.html "You might think getting a DUI is as easy as getting behind the wheel of a car after drinking alcohol. But that's only half the battle. You also need to get pulled over by law enforcement and cited for it."
How to Not Be a Husband Caught Cheating ehow.com/how_5528899_not-husband-caught-cheating.html
"Don't leave trails which can and will turn into signs you're cheating. First point to remember is to not use any computer your partner has access to when you communicate via email or IM to the cohort."
AdSense Click Fraud
How to get banned from Google Adsense ehow.com/how_5740892_banned-google-adsense.html
How to Increase Your Click Through Rate with Google AdSense ehow.com/how_5203081_increase-through-rate-google-adsense.html
How to not get Caught With Google Adsense Click Fraud ehow.com/how_5979999_not-caught-google-adsense-fraud.html
Leveraging Expired Domains
Demand Media bought out a leading domain registrar named eNom & leveraged some of the expired domains with links to prop up eHow, by 301 redirecting those domains into eHow's deep pages.
eHow was not only churning out loads of shallow content, but Demand Media was also using the data gleaned from eHow to make sister sites which included auto-generated pages and feeding search engines their own results.
They Made Google Look Stupid
Doing one thing and claiming another can provide cover for some finite period of time, but ultimately when you create such a spectacle out of Google that your exploitative ways become the core marketing message for Google's competitors you know your days are numbered. And given that the Wired piece made the media hate Demand Media, there was nobody left to defend them other than folks who would also seem in some way conflicted.
Ultimately this goes back to the core issue that hurt Demand Media: branding.
Don't make Google look stupid. That is the #1 rule of SEO.
Much noise was made recently about Google taking a whack at so-called content farms -- sites which apply industrial production techniques to the creation of content targeting the long-tail of the query distribution. This is a subject of huge interest to many Internet businesses, either because they advertise on the AdWords Content Network (and, by extension, on content farms), because they compete with content farms on particular searches, or merely because they hate seeing content farms in their search results. As luck has it, I am three for three. It pains me to say it, but content farming is here to stay. It is an economic inevitability.
The Attention Economy
Much of the Internet currently operates in an attention economy, a level or two removed from direct monetization. Facebook is worth in excess of 50 billion not just because they're making money hand over fist -- though they are -- but because they have achieved a dominant position in the attention economy, and they command such huge rivers of attention that they can trade trickles of it to people for actual money.
Google is the dominant player in the attention economy -- they harvest vast amounts of attention via controlling navigation on the Internet (via a commanding lead in search), they sell attention in the form of AdWords ads, and they provide a marketplace for attention with their AdSense product.
Individual publishers -- from the New York Times down to the smallest hobbyist site on the Internet -- are also largely in the attention economy. For a mega-brand like the New York Times, attention can be generated -- they can literally make news. Disney has a repeatable industrial process which takes as input one female teenager and produces as output a cultural phenomenon with hundreds of thousands of rabid fans.
Smaller players -- Google back in the dorm room days or hobbyist sites today -- largely cannot create attention on these scales, they can only harvest attention which already exists. Attention exists in the world for things independent of their own existence. People play golf. People bake cookies. People read Dan Brown novels. People receive massages. For all these things and more, people demand content: they want to improve their golf swing, they want new cookie recipes, they want new Dan Brown novels, they want massage how-to videos. And they are willing to pay with attention, a scarce commodity which can be converted into cash.
The Economics of Content Creation
Consider a hypothetical Internet with no efficient way of converting attention into money. This is not difficult to imagine: it was essentially the Internet of the dot-com bubble, where everyone wanted "eyeballs" but "eyeballs" plus banner advertising resulted in economically non-viable businesses. In this hypothetical Internet, content is mostly produced by people who have intrinsic reasons for creating it: hobbyists who want to share their passion, law professors who want to increase their professional reputation, governments who need to employ somebody and might as well employ a webmaster, and the like. This is widely viewed as a Garden of Eden scenario: the Internet, without the corrupting influence of money.
We had this Internet, and the average user experience was miserable.
Ability to publish content on the Internet was once dominated by presence of arcane technical skills (being a "webmaster", a title which thankfully has fallen out of fashion). Webmasters were, by and large, very geeky people. They largely scratched their own itches, which (predictably) resulted in an Internet chock-full of Dungeons and Dragons character sheets, trivia about Matter-Eater Lad, and fansubbed anime episodes.
Less well-represented on the Garden of Eden Internet was content appealing to demographics which don't intersect with geeks that often. Women, the very young, the elderly, non-English speakers, etc etc, were across a very real digital divide from the D&D players. You could still find advice on how to make an apple pie online, but if you did, it was because you got lucky and had a CS professor with quirky interests (for a CS professor, at any rate).
This started to change with the widespread adoption of content management systems, which took the level of computer skill for content creation down from "close to programming" to "close to using a word processor." The first very popular CMSes were blogs, and there was much triumphantalist backslapping among bloggers that blogging was democratizing the Internet. You could be blogging in your pajamas and still take on the New York Times, or so the argument went.
Ability to use a word processor is more widely spread among the population than webmastering skills, but it is still a far cry from universal. Blogging caught on primarily with professional communicators: professors, journalists, and other folks who had long been using skill with the printed word and perceived authority with pre-existing audiences. Concurrent with this, there was an explosion of content creation aimed at the concerns of well-educated, middle-class American white urban professionals. Politics, financial advice, education, religion, international news: covered, covered, covered, both by established media and publishing interests moving online and by the new media (rather like the old media, except with orders of magnitude lower capital requirements). Content was now a democracy, in the same sense that America after the Revolution was a democracy: white property owners could be reasonably assured of having their interests represented.
There still existed massive demand -- unharvested attention -- for content outside the early adopters of the Internet. Larger scale online publishers began to go after the head of the demand distribution, and hobbyist sites continued to publish things like apple pie recipes, often with a quantum leap in presentational quality from just a few years previously. Google AdWords was one of the primary lubricants for making this happen -- a hobbyist site dominating a niche like e.g. apple pies could suddenly generate non-trivial amounts of money for the site owner, largely by taking transaction costs about negotiating advertising sales out of the equation. This also allowed Google to monetize its own attention surplus better, because sending a searcher to a site with AdSense on it gives them a second chance at getting paid for a click. AdSense has generated roughly a third of Google's revenue for the last several years.
The Industrialization Of Content Production
With technology continuing to bring down barriers to creating content and business model optimization like AdWords improving the opportunity to monetize attention, it was virtually inevitable that eventually the supply and demand curves would cross. They long since had for high-value verticals like e.g. mortgages, where huge transaction volumes, high margins, gigantic advertising spends, and liquid affiliate/lead gen markets have long subsidized huge volumes of content creation. Many quite savvy Internet users were simply unaware this had happened, since one does not search for mortgages or poker every day. The Internet is a virtually uncountable multitude of attention markets, and in many of them it was more expensive to create content than the harvestable attention could justify. Those niches continued to be underserved, in the capitalist sense of the word: people would have consumed more content for them, but that content did not exist.
Then disruptive innovation happened: basically, a number of firms figured out that the combination of algorithmically predicting attention plus outsourcing content creation could let them exploit relatively small amounts of attention, in parallel, at massive scales. This innovation caused the supply and demand curves to cross for a huge number of attention markets which had not crossed before. The result: content farming at massive, massive scale.
Consider bingo cards for elementary schoolteachers, a very niche subject that happens to pay my rent. Attention exists for it: bingo has long been used in American classrooms to review vocabulary across a variety of subjects. As teachers and parents gradually started using the Internet and using Google, their attention about bingo -- a tiny, tiny sliver of the massive river of attention Google controls -- became up for grabs. Some flowed to hobbyist sites like my own, some flowed to larger publishers like the NYT's About.com unit, and some was simply poorly served. Teachers typed queries into Google and got garbage results which were not responsive.
I have advertised on Google's AdWords Content Network for years, and for the last four years I've been essentially willing to buy as much traffic as Google cares to sell me for a range of quality below a given price. This makes my AdWords stats a proxy for who is getting traffic for bingo-related searches. (Google controls navigation on the Internet, so the presence of traffic for near-term desires like bingo cards strongly suggests that it was searched for. Check your Analytics if you don't believe me.)
My market has massive seasonal changes in attention, so let's look at consistent month-long slices of it, compared year-to-year. Here's a tale of four Februaries.
In 2008, my AdWords spend was dominated by legacy Internet publishers like About.com, niche publishers in education, and hobbyist sites. Total spend was about $370, of which About captured almost $70 (~19%).
In 2009, hobbyist sites and niche publishers decline with the ascendancy of a new publisher called Kaboose, an early iteration of a content farm, focused on topics of interest to women (including, e.g., bingo). Total spend was about $560, of which Kaboose captured almost $160 (a whopping 29%), more than quintupling their performance from 2008. Or, to put it another way, more than half of increase in the size of this small attention market can be attributed to one publisher. 2009 also sees a new site in my top 10: a minor player called eHow run by an obscure firm Demand Media.
In 2010, spend again increases (to $640), and the top positions are dominated by content farms and ezinearticles, a legacy crowdsourced content farm. Kaboose loses share to new content farm entrants, and eHow has comparatively modest 50% year over year growth. Content farms now control over a third of this attention market.
In 2011, spend again increases (to $920 -- nearly 50% year over year growth), and content farms dominate the attention market. eHow has improve its execution again, to the point where they singlehandedly capture $150 in ads, quintupling performance from a year before. (Yep, their revenue is now ten times what it was in 2009.)
The Microeconomics Of Content Farming
Why did content farming capture so much of the attention economy so quickly? Basically, once the process for creating content very responsive to a single search term was repeatable, it could be replicated down the long-tail very, very quickly, in response to market signals such as e.g. successful pages in related searches. My business has long had a page about Valentine's Day bingo cards because I know, being a publisher in the niche, that they're very valuable -- there exists a substantial amount of attention which will be paid to Valentine's Day bingo every February. Do you think Valentine's Day bingo cards is a tiny niche, on Internet scales? eHow has over thirty pages targeting variants on this top -- thirty slices of a fraction of a tiny niche which were worth individualized effort to target. Some representative titles:
Church Valentine's Party Games
Make Valentine Bingo Cards
Classroom Valentine's Day Party Games
Valentine's Math Games
Christian Valentine's Games
Christian Adult Valentine's Games
Valentine's Party Games For Older Kids
etc, etc, etc, etc
Zooming in on the performance of just one of these pages, about Valentine's bingo for churches, I paid $9 for ads on it in February 2011. If we make the unreasonably pessimistic assumption that it never makes money except in February, and that the remnant image advertising is basically a wash (not true, given the amount that Groupon and online games throw around at monetizing it), this suggests that the four text ads on the page probably generated on the order of $30 in revenue. Google's 68% revenue share means that Demand Media got about $20 in revenue from this page... in 2011 alone.
Content farms are targeting evergreen content, though: Valentine's Day is going to happen in 2012, and there will still exist churches who want to play bingo on it. Will revenue from this page go to zero? That is highly unlikely, because this page wasn't written in 2011 -- it was written in 2010, when I paid $1 for ads in it (implying about $2 in revenue). Due to changes in the search environment and Demand Media's increasing sophistication with leveraging internal traffic, it got nine times more valuable at no marginal cost in the course of a single year.
Content farms operate on a portfolio strategy: the pieces of content which succeed, like that page, subsidize the pieces of content which don't. As long as the average revenue portfolio-wide exceeds cost of content production, one should expect the content farms to pour capital into content production and scale it to the moon. The portfolio strategy appears to be winning, judging by eHow's meteoric rise in revenue and the demonstrated ability for content farms to choke out non-farming content sources. eHow alone showed my ads on five times as many pages in 2011 as in 2010.
And why wouldn't they? The unit economics of content farming are stunningly attractive. Demand Media pays on the order of $10 to have the 312 words on that page written and edited. If Wall Street could design an equity which cost $10 and paid $2 per share in 2010, $20 per share in 2011, and an unknown but positive amount thereafter, all other investment classes would be virtually obsolete. The only problem is systemic risks.
The only thing that can reverse this is content getting more expensive to create or attention getting scarcer (or harder to monetize) for these markets.
There is more attention to monetize: It is possible that Internet use will decline in the future, but I will offer excellent odds to anyone who wishes to bet that: Kansas schoolmarms in the elementary bingo market have quite a ways to go before they catch up to the average reader of this blog in online consumption, which predicts a large aggregate increase in attention harvestable on the Internet and even larger proportional increases to the attention markets they care about.
Google and advertisers increasing cost of attention: Ignoring huge sources of attention of dubious worth, like ads displayed next to Facebook games, an AdSense ad displayed to someone 2 seconds after they type in a query into Google is, essentially, a search ad.
Read that again, because it is important.
This means that content farms are essentially in the search ad monetization business -- i.e. the most profitable business in the history of the Internet. Search ads monetize extraordinarily well because in addition to capturing user attention they come with user intent. This makes them orders of magnitude more valuable than the old banner display networks (which users quickly become blind to), sidebar ads next to Farmville or pictures of that cute girl from chemistry class, and the like. Content farms preserve search intent because the laser targetting combination of their one-topic pages and AdSense means that the AdSense ads are guaranteed to be responsive content to the search and, give that everything else on the page was written by a content farm, the ads are the best content on the page.
Sure, farms cede a large portion of the reach of search to actual search engines, since they can't rank for head queries, but even 5% of Google's market cap would be nothing to sneeze at. Google has incentives to help them rather than competing with them. Meanwhile, any market with competitors will tend to drive the cost of ads up until they have expended all of their margin on the sale. For a high-margin category like software, if my competitor is willing to pay 51% of his sale price to generate one marginal sale (when you back it out to cost-per-click prices), I'm willing to bid 51%. The equilibrium outcome is that my advertising costs increase over time while my ROI decreases, but it remains profitable and I'd be a fool not to do it. Google and their publishing partners win and win big.
This Is Old News. Google Fixed Content Farming... Right?
Back in late February 2011, Google rolled out the Panda update, which was widely perceived to be aimed at content farms. What actually happened was that it separated Content Farming 1.0 from Content Farming 2.0 -- earlier entrants like ezinearticles and Mahalo (and a raft of sites you've never heard about) lost out to better executing farms, including eHow.
For example, instead of comparing Februaries like we did earlier, let's see the progression of Marches in the bingo niche. Largely due to the absence of Valentine's Day, March consistently has less attention available than February: aside from an anomalous 2008 (long story with short moral: don't bork your AdWords code), spends fell 28% in 2009 and 17% in 2010. The decline was much more pronounced in 2011, possibly attributable to Panda reshaping the attention economy landscape: it jumped to 37%.
However, the performance of individual publishers was mixed:
eHow (Demand Media) declined only 18%. This looks virtually in line with historical seasonal trends (growth in 2010 was so fast they were actually flat over the interval, i.e. growing much faster than market). Their performance in March 2011 (historically a "bad" month for bingo attention) crushed their performance in February 2009 (historically a "great" month for bingo attention). One could be excused for believing eHow was not net-affected by Panda
LoveToKnow got annihilated -- spend decreased 71%. (The comparable decrease in 2010 was only 25%.)
ezinearticles got annihilated -- spend decreased 68%. (The comparable decrease in 2010 was only ~10%.)
About.com was severely affected -- spend decreased about 46%. (The comparable decrease in 2010 was, again, lower -- only 21%.)
Summed over all the content farmers, Panda appears to have picked a winner with regards to this slice of the attention economy: eHow.
I had been wistfully hoping that when the content farms got crushed that my site, which competes with them for many queries, would pick up some of the redistributed attention. If this happened, it has been too minor to notice in my Analytics stats -- my organic searches from Google look roughly in line with where I would expect them to be absent Panda. The big winner and the big losers appear to be concentrated among farmers, with fairly minor spillover to the rest of this sliver of the attention economy. This makes sense to me, in a way -- I simply don't have a page which is more responsive to the need for church Valentine's bingo activities than eHow's does. I believe my pages are far and away better than eHow's -- my pages about making bingo cards will actually let you make bingo cards -- but reasonable people could disagree on whether that is more important than capturing all parts of the user's intent, including the "specifically for churches" bit of it. Outside of my narrow slice of the online experience, a Big Publisher advocacy group estimates that the Panda update redistributed $1 billion in advertising revenue, which is nothing to sneeze at. However, with the Content Network generating over $20 billion in annual ad sales, $1 billion looks less like a fundamental shift and more like repartitioning scraps left to the losers.
Did Panda Kill Farming?
Only the economics can kill farming, and it does not appear that Panda meaningfully changes the microeconomics of content farms. If you can sell $40 of ads against a $10 page prior to Panda, and after Panda you can only sell $20 of ads, well, farm on. The model scales to the moon as long as the portfolio is even marginally profitable. The losing farms will also be incentivized to go back to the drawing board and reevaluate where they place their content bets: perhaps it is no longer lucrative enough for them to go after certain micro-markets, like elementary school bingo cards (or like the bottom half of elementary school bingo cards), but their more valuable markets are probably still stupidly profitable. Those will get more competitive as they redeploy their content creation resources going forward, assuming they're capable at executing on that.
Demand Media, on the other hand, is grinning like the cat that just caught the canary. Not only is their core business proposition virtually unaffected, in spite of the worst nightmare of their business model (Google coming down on it like the fist of an angry god) coming true, their unit economics down the tail just got radically better. Going forward, they can expect less competition in the less lucrative markets, allowing them to capture larger fragments of the attention available in those markets, and proportionally higher revenues.
The Future: Outfarming The Farmers?
A frequent theme of dystopian science fiction is that man-machine hybrids outcompete the human race. Algorithmic/freelancer hybrids, like content farms, are pretty much there, for a large and increasing portion of the content tail. This is going to get exacerbated by changes in content production and consumption, such as the rise of video (which has orders of magnitude higher production costs than text) and decline of hobbyist content creation. In 2006, my business had significant competition for keywords from individual teachers' sites, where Mrs. Smith decided (back in 1996) to put up a web page to try out this new Internet thing on her computer. In 2021, there will be many less sites created by Mrs. Smiths, because Mrs. Smith in 2011 now has an iPad to watch her Khan Academy videos on, and the iPad is virtually useless for creating websites. I'm already seeing anecdotal behavioral changes in my customers ("Say, how do I hook a printer up to an iPad so I can make my cards from there? I hate turning on the computer -- I think it has a virus or something, and it is slow."), and ordinarily they're quite behind the curve.
Additionally, while Mrs. Smith had sufficient dedication to the niche to target the most common activities, she never made more than 5 or so pages about bingo. I have about a thousand bingo activities, created with focused application of custom software by freelancers. The content farms are making me look practically lazy with their scale of publication.
This suggests an obvious route for improvement for me: if it is stupidly profitable for me to pay $200 to Google so that it can pay $130 to eHow so that it can pay $50 to freelance writers to create 5 pages, why don't I just take the $200 and pay freelance writers to write those same 5 pages... and then 15 more? The only thing which has stopped me from doing it so far is concern about polluting the Internet. But the economic attraction of doing it is undeniable. If the choice is a user getting their bingo content from an anonymous freelancer at eHow working through their queue of 400 articles for the week or getting it from someone who is only employed to write bingo articles, shouldn't they get it from me?
This dilemma, repeated a thousand times across a thousand markets, is going to create the Internet of 2020. Break out your straw hats, folks: we are all going to be farming or, at best, a step removed from farming by paying intermediaries (Google and the farms) to do our farming for us. The main distinction is going to be between successful execution of farming strategies (like eHow) and poor execution of farming strategies (like their competitors who recently got whacked).
Demand Media is already shopping out their business model as a service: since newspapers and other legacy publishers are a) dying but b) scare Google (because they can cause Google to have bad PR, which might result in government regulation, which is Google's sole competitive risk), Demand Media would love newspapers to be the front man for their farmed content. That, or parallel arrangements, is going to be almost irresistible to anyone with sufficient signals of trust to rank for arbitrary longtail content in their niche. I mean, "Create a repeatable process to create content of a known level of quality, throw money at the process to scale it, then sell ads against the result" is the entire newspaper business model! Content farming just takes out the sucky bits like "own a multi-billion-dollar distribution network for dead trees" and "write articles which are relevant to a few hundred people at an amortized cost of over $1,000 per article." (It is an open secret that the most lucrative ads in a newspaper aren't around the news, but are in sections like Style and Travel. It does not take Pulitzer Prize-winning journalism to write articles on this season's hottest shade of fuchsia or compelling reasons to go to Cancun. "Real" news has always been a loss leader to sell advertising against their other content. If they can create ten times as much fluff at a tenth of the cost, why not? And if they can... do they need the "real" news in the first place?)
Can Google just tighten the screws with another Son of Panda update? That is unlikely to work unless they repeal the laws of economics: farming happens on every topic for which the supply and demand curve crosses. Slashing content farm's ability to rank across the board by 40% just makes a fraction of the content space monetarily unattractive to them, but the content space is virtually infinite and the ability to monetize attention is increasing all the time. If Groupon will pay for remnant inventory on a page about How To Pick Your Nose, who will compete for that attention except a content farm?
Is that the Internet I want? Probably not. But then again, I'm privileged -- as a geek, my interests in content will always be well represented on the Internet, even without monetary incentives to create it. People will go to StackOverflow to answer my questions before I've even asked them, they'll create Starcraft XII walk-through videos, they'll even write software, all without seeing a penny for it. The experience for less privileged folks, though, demonstrably sucked at the dawn of the Internet, and it is not obvious to me that removing most of the growth in content responsive to their needs is a net win for them. We might see an Internet where the content-rich win and everybody else gets farming.
I was looking for information about the nuclear reactor issue in Japan and am glad it did not turn out as bad as it first looked!
But in that process of searching for information I kept stumbling into garbage hollow websites. I was cautious not to click on the malware results, but of the mainstream sites covering the issue, one of the most flagrant efforts was from the Huffington Post.
AOL recently announced that they were firing15% to 20% of their staff. No need for original stories or even staff writers when you can literally grab a third party tweet, wrap it in your site design, and rank it in Google. Inline with that spirit, I took a screenshot. Rather than calling it the Huffington Post I decided a more fitting title would be plundering host. :D
We were told that the content farm update was to get rid of low quality web pages & yet that information-less page was ranking at the top of their search results, when it was nothing but a 3rd party tweet wrapped in brand and ads.
You can imagine in a hyperspace a bunch of points, some points are red, some points are green, and in others there’s some mixture. Your job is to find a plane which says that most things on this side of the place are red, and most of the things on that side of the plane are the opposite of red. - Google's Amit Singhal
If you make it past Google's arbitrary line in the sand there is no limit to how much spamming and jamming you can do.
we actually came up with a classifier to say, okay, IRS or Wikipedia or New York Times is over on this side, and the low-quality sites are over on this side. - Matt Cutts
(G)arbitrage never really goes away, it just becomes more corporate.
As bad as that sounds, it is actually even worse than that. Today Google Alerts showed our brand being mentioned on a group-piracy website built around a subscription model of selling 3rd party content without permission! As annoying as that feels, of course there are going to be some dirtbags on the way that you have to deal with from time to time. But now that the content farm update has went through, some of the original content producers are no longer ranking for their own titles, whereas piracy sites that stole their content are now the canonical top ranked sources!
Google never used to put piracy sites on the first page of results for my books, this is a new feature on their part, and I think it goes a long way to show that their problem is cultural rather than technical. Google seems to have reached the conclusion that since many of their users are looking for pirated eBooks, quality search results means providing them with the best directory of copyright infringements available. And since Google streamlined their DMCA process with online forms, I couldn’t discover a method of telling them to remove a result like this from their search results, though I tried anyway.
... I feel like the guy who was walking across the street when Google dropped a 1000 pound bomb to take out a cockroach - MorrisRosenthal
Media is all about profit margins. eHow was originally founded in 1998 & had $36 million in venture capital behind it. But the original cost structure was flawed due to high content costs. The site failed so badly that it was sold in 2004 for $100,000. The original site owners had GoogleBot blocked. Simply by unblocking GoogleBot and doing basic SEO to existing content the site had a revenue run rate of $4 million Dollars within 2 years, which allowed the site to be flipped for a 400-fold profit.
People are arguing if Demand Media is overvalued at its current valuation, and at one point the NYT was debating buying Demand Media by rolling About.com into Demand & own 49% of the combined company. But the salient point to me is that we are talking about something that was bought for only $100,000 7 years ago. Sure the opportunities today may be smaller scale and different, but if you see value where others do not then recycling something that has been discarded can be quite valuable.
The key, he said, was to keep costs low. If possible, don't pay for the intellectual content. Look for material, he urged, on which the copyright has expired. Any book published in the U.S. before 1923 was available.
He said he was in the process of turning vanloads of old books into websites. With a few hours of labor, for example, you could take a turn-of-the-century Creole cookbook and transform it into the definitive site for vintage Creole recipes. Google's AdSense program would then load the thing up with ads for shrimp and cooking pots and spices and direct people looking for shrimp recipes to your website.
How did ArticlesBase grow to that size? It and Ezinearticles were a couple of the "selected few" which lasted through the last major burn down of article directories about 3 or 4 years ago. But it seems their model has peaked after this last Google update.
Search is Political
Content farms are proving to be a political issue in search. They are beginning to replace "the evil SEO" in the eye of the press as "what creates spam." Rich Skrenta created a spam clock which stated that a million spam pages are created every hour. He then followed up by banning 20 content farms from the Blekko search results & burning spam man. ;)
Microsoft also got in on the bashing, with Harry Shum highlighting that Google was funding web pollution. When Blekko's model is based on claiming Google polluted the web with crap, Microsoft says the same thing, and there is a rash of end user complaints, there are few independent experts the media can call upon to talk about Google - unless they decide to talk to SEOs, who tend to be quite happy to highlight Google's embarrassing hypocrisy. Freelance writers may claim that marketing is what screwed up the web, but ultimately Google has nobody credible and well known to defend them at this point. The only people who can defend Google's approach are those who have a taste in the revenue stream. Hence why Google had to act against content farms.
Google stepping up their public relations smear campaigns against Bing and others is leaving Google looking either hypocritical or ignorant in many instances, like when a Google engineer lambasted an ad network without realizing Google was serving the scam ad.
At first glance StackExchange's growth looks exciting, but it has basically gone nowhere outside of the programming niche. In my opinion they are going to need to find subject matter experts to lead some of their niche sites & either pay those experts or give them equity in the sites if they want to lead in other markets. Worse yet, few people are as well educated about online schemes as programmers, so the other sites will not only lack leadership, but will also be much harder to police. Just look at the junk on Yahoo! Answers! There are Wordpress themes and open source CMS tools for QnA sites, but I would pick a tight niche early if I was going to build one as the broader sites seem to be full of spam and the niche sites struggle to cross the chasm. As of writing this, fewer than 50 Mahalo answers pages currently indexed in Google have over 100 views. It flat out failed, even with financial bribes and a PR spin man behind it.
At the organizational level, Google is essentially chaos. In search quality in particular, once you've demonstrated that you can do useful stuff on your own, you're pretty much free to work on whatever you think is important. I don't think there's even a mechanism for shifting priorities like that.
We've been working on this issue for a long time, and made some progress. These efforts started long before the recent spat of news articles. I've personally been working on it for over a year. The central issue is that it's very difficult to make changes that sacrifice "on-topic-ness" for "good-ness" that don't make the results in general worse. You can expect some big changes here very shortly though.
A good example of the importance of padding out results with junk on-topic content to aid perceived relevancy can be seen by looking at the last screenshot of a search result here. Blekko banned the farms, but without them there is not much relevant content that is precisely on-topic. (In other words, content farms may be junk, but it is hard to have the same level of perceived relevancy without them).
Google created a Chrome plugin to solicit end user feedback on content mills, but that will likely only reach tech savvy folks & the feedback is private. Google can claim to use any justification for removing sites they do not like though, just like they do with select link buying engagements. Look the other way where it is beneficial, and remove those which you personally dislike.
In a recent WSJ article Amit Singhal was quoted as saying new signals have been added to Google's relevancy algorithms:
Singhal did say that the company added numerous “signals,” or factors it would incorporate into its algorithm for ranking sites. Among those signals are “how users interact with” a site. Google has said previously that, among other things, it often measures whether users click the “back” button quickly after visiting a search result, which might indicate a lack of satisfaction with the site.
In addition, Google got feedback from the hundreds of people outside the company that it hires to regularly evaluate changes. These “human raters” are asked to look at results for certain search queries and questions such as, “Would you give your credit card number to this site?” and “Would you take medical advice for your children from those sites,” Singhal said.
Evolving the Model
One interesting way to evolve the content farm model is through the use of tight editorial focus, a core handful of strong editors, and wiki software. WikiHow was launched by a former eHow owner, and when you consider how limited their relative resources are, their traffic level and perceived editorial quality are quite high. Jack Herrick has struck how-to gold twice!
Notice any content farms missing from the above list? Maybe the biggest one? Here is a list of some of eHow's closest competing sites (based on keywords, from SEM Rush). The ones in red got pummeled, the ones in yellow dropped as well & were fellow Demand Media sites, and the ones in green gained traffic.
Jason "will do anything for ink" Calacanis recently gave an about face speech claiming people need to step away from the content farm business model, and in doing so admitted that roughly everything he said about Mahalo over the past couple years was a complete lie. Surprise, surprise. The interesting bit is that the start up community - which used to fawn over his huckster PR driven ploys - no longer buys them. Jason claimed to have "pivoted" his business model again, but once again we see more garbage content. His credibility has been spent. And so have his rankings! Sistrix shows that not only is he ranking for fewer keywords, but that the graph has skewed downward to worse average positions.
After the Crash, What is Next?
The biggest content farms like Ask & eHow will still do well in the short run. Over the long run I see Google bringing the results of content farms to the attention of book publishers & then working to slowly rotate out from farmed content to published book content. Most readers do not know that most book writers are lucky if they earn $10,000 writing a 300 or 400 page tome. Publishers tell book authors that with the additional exposure they can often sell lots more other things, but unless the content is highly targeted that might not back out well for the author. But that cheap content is far better structured and far more vetted than the mill stuff is.
Over the past week I have been seeing more ebooks in the search results, though I am not entirely sure if that is just because I am searching for more rare technical stuff that simply might not be online.
But what they didn't mention is that eHow's rankings are actually up! In fact, their new distribution chart looks just like their old one, only skewed a bit to the left with higher rankings. eHow's profile is 15% better than it was before the update & the only site which gained more traffic from this update than eHow did was Youtube.
If you listen to Richard's interviews, you would never know him to be the type to redirect expired domains:
We really want to let Google speak for themselves. Whatever Matt Cutts and Google want to (say) about quality we totally support that because again that’s their corporate interest. What we said and would have said is we applaud Google removing duplicate content ... removing shallow, low quality content because it clogs the search results. Both we and Google are 100 percent focused on making the consumer happy. It’s the right thing to do and it’s good for our business.
If you syndicate Google's spin you can get away with things that a normal person can't. Which is why eHow renounced the content farm label even faster than they created it.
Article directories & topical hub sites have been online since before eHow was created. But eHow's marketing campaign was so caustic & so toxic that it literally destroyed the game for most of their competitors.
And now that Google has "fought content farms" (while managing to somehow miss eHow with TheAlgorithm) most of Demand Media's competitors are dead & Richard Rosenblatt gets to ride off into the sunset with another hundred million Dollars, as eHow is the chosen one. :D
Many of the changes we make are so subtle that very few people notice them. But in the last day or so we launched a pretty big algorithmic improvement to our ranking—a change that noticeably impacts 11.8% of our queries—and we wanted to let people know what’s going on. This update is designed to reduce rankings for low-quality sites—sites which are low-value add for users, copy content from other websites or sites that are just not very useful. At the same time, it will provide better rankings for high-quality sites—sites with original content and information such as research, in-depth reports, thoughtful analysis and so on.
Currently this update is US only, so if you are outside of the United States you may need to get a US VPN or add &gl=us to your search string's results on Google (likeso). Recent updates have had a variety of impacts and implications outside of content mills.