It was hard to spot, at first.

It started with one store on the outskirts of town. It was big. Monolithic. It amalgamated a lot of cheap, largely imported stuff and sold the stuff on. The workers were paid very little. The suppliers were squeezed tight on their margins.

And so it grew.

And as it grew, it hollowed out the high street. The high street could not compete with the monoliths sheer power. They couldn’t compete with the monoliths influence on markets. They couldn’t compete with the monoliths unique insights gained from clever number crunching of big data sets.

I’m talking about Wal Mart, of course.

Love ‘em or loathe ‘em, Walmart gave people what they wanted, but in so doing, hollowed out a chunk of America's middle class. It displaced a lot of shop keepers. It displaced small business owners on Main Street. It displaced the small family retail chain that provided a nice little middle class steady earner.

Where did all those people go?

It was not only the small, independent retail businesses and local manufacturers who were fewer in number. Their closure triggered flow-on effects. There was less demand for the services they used, such as local small business accountants, the local lawyer, small advertising companies, local finance companies, and the host of service providers that make up the middle class ecosystem.

Where did they all go?

Some would have taken up jobs at WalMart, of course. Some would become unemployed. Some would close their doors and take early retirement. Some would change occupations and some would move away to where prospects were better.

What does any of this have to do with the internet?

The same thing is happening on the internet.

And if you’re a small business owner, located on the web-equivalent of the high street, or your business relies on those same small business owners, then this post is for you.

Is Technology Gutting The Middle Class?

I’ve just read “Who Owns The Future”, by Jaron Lanier. Anyone who has anything to do with the internet - and anyone who is even remotely middle class - will find it asks some pretty compelling questions about our present and future.

Consider this.

At the height of it’s power, the photography company Kodak employed more than 140,000 people and wa worth $28 billion. They even invented the first digital camera. But today Kodak is bankrupt, and the new face of digital photography has become Instagram. When it was sold to Facebook for a billion dollars in 2012, Instagram only employed 13 people

Great for Instagram. Bad for Kodak. And bad for the people who worked for Kodak. But, hey. That’s progress, right? Kodak had an outdated business model. Technology overtook them.

That’s true. It is progress. It’s also true that all actions have consequences. The consequence of transformative technology is that, according to Lanier, it may well end up destroying the middle class if too much of the value is retained in the large technology companies.

Lanier suggests that the advance of technology is not replacing as many jobs as it destroys, and those jobs that are destroyed are increasingly middle class.

Not Political (Kinda)

I don’t wish to make this post political, although all change is inherently political. I’m not taking political sides. This issue cuts across political boundaries. I have a lot of sympathy for technological utopian ideas and the benefits technology brings, and have little time for luddism.

However, it’s interesting to focus on the the consequences of this shift in wealth and power brought about by technology and whether enough people in the internet value chain receive adequate value for their efforts.

If the value doesn't flow through, as capitalism requires in order to function well, then few people win. Are children living at home longer than they used to? Are people working longer hours than they used to in order to have the same amount of stuff? Has the value chain been broken, Lanier asks? And, if so, what can be done to fix it?

What Made Instagram Worth One Billion Dollars?

Lanier points out that Instagram wasn’t worth a billion dollars because it had extraordinary employees doing amazing things.

The value of Instagram came from network effects.

Millions of people using Instagram gave the Instagram network value. Without the user base, Instagram is just another photo app.

Who got paid in the end? Not the people who gave the network value. The people who got paid were the small group at the top who organized the network. The owners of the "Siren Servers":

The power rests in what Lanier calls the “Siren Servers”: giant corporate repositories of information about our lives that we have given freely and often without consent, now being used for huge financial benefit by a super-rich few

The value is created by all the people who make up the network, but they only receive a small slither of that value in the form of a digital processing tool. To really benefit, you have to own, or get close to, a Siren Server.

Likewise, most of Google’s value resides in the network of users. These users feed value into Google simply by using it and thereby provide Google with a constant stream of data. This makes Google valuable. There isn’t much difference between Google and Bing in terms of service offering, but one is infinitely more valuable than the other purely by virtue of the size of the audience. Same goes for Facebook over Orkut.

You Provide Value

Google are provided raw materials by people. Web publishers allow Google to take their work, at no charge, and for Google to use that work and add value to Google’s network. Google then charges advertisers to place their advertising next to the aggregated information.

Why do web publishers do this?

Publishers create and give away their work in the hope they’ll get traffic back, from which they may derive benefit. Some publishers make money, so they can then pay real-world expenses, like housing, food and clothing. The majority of internet publishers make little, or nothing, from this informal deal. A few publishers make a lot. The long tail, when it comes to internet publishing, is pretty long. The majority of wealth, and power, is centralized at the head.

Similarly, Google’s users are giving away their personal information.

Every time someone uses Google, they are giving Google personal information of value. Their search queries. They browsing patterns. Their email conversations. Their personal network of contacts. Aggregate that information together, and it becomes valuable information, indeed. Google records this information, crunches it looking for patterns, then packages it up and sells it to advertisers.

What does Google give back in return?

Web services.

Is it a fair exchange of value?

Lanier argues it isn’t. What’s more, it’s an exchange of value so one-sided that it’s likely to destroy the very ecosystem on which companies like Google are based - the work output, and the spending choices, of the middle class. If few of the people who publish can make a reasonable living doing so, then the quality of what gets published must decrease, or cease to exist.

People could make their money in other ways, including offline. However, consider that the web is affecting a lot of offline business, already. The music industry is a faint shadow of what it once was, even as recent as one decade ago. There are a lot fewer middle class careers in the music industry now. Small retailers are losing out to the web. Fewer jobs there. The news industry is barely making any money. Same goes for book publishers. All these industries are struggling as online aggregators carve up their value chains.

Now, factor in all the support industries of these verticals. Then think about all the industries likely to be affected in the near future - like health, or libraries, or education, for example. Many businesses that used to hire predominantly middle class people are going out of business, downsizing their operations, or soon to have chunks of their value displaced.

It’s not Google’s aim to gut the middle class, of course. This post is not an anti-Google rant, either, simply a look at action and consequence. What is the effect of technology and, in particular, the effect of big technology companies on the web, most of whom seem obsessed with keeping you in their private, proprietary environments for as long as possible?

Google’s aim is index all the worlds information and make it available. That’s a good aim. It’s a useful, free service. But Lanier argues that gutting the middle class is a side-effect of re-contextualising, and thereby devaluing, information. Information may want to be free, but the consequence of free information is that those creating the information may not get paid. Many of those who do get paid may be weaker organizations more willing to sacrifice editorial quality in able to stay in business. We already see major news sites with MFA-styled formatting on unvetted syndicated press releases. What next?

You may notice that everyone is encouraged to “share” - meaning “give away” - but sharing doesn't seem to extend to the big tech companies, themselves.

They charge per click.

Robots.txt

One argument is that if someone doesn’t like Google, or any search engine, they should simply block that search engine via robots.txt. The problem with that argument is it’s like saying if you don’t like aspects of your city, you should move to the middle of the wilderness. You could, but really you’d just like to make the city a better place to be, and to see it thrive and prosper, and be able to thrive within it.

Google provides useful things. I use Google, just like I use my iPhone. I know the deal. I get the utility in exchange for information, and this exchange is largely on their terms. What Lanier proposes is a solution that not only benefits the individual, and the little guy, but ultimately the big information companies, themselves.

Money Go Round

Technology improvements have created much prosperity and the development of a strong middle class. But the big difference today is that what is being commoditized is information itself. In a world increasingly controlled by software that acts as our interface to information, if we commoditize information then we commoditize everything else.

If those creating the information don’t get paid, quality must decrease, or become less available than it otherwise would be. They can buy less stuff in the real world. If they can’t buy as much stuff in the real world, then Google and Facebook’s advertisers have fewer people to talk to that they otherwise would.

It was all a social construct to begin with, so what changed, to get to your question, is that at the turn of the [21st] century it was really Sergey Brin at Google who just had the thought of, well, if we give away all the information services, but we make money from advertising, we can make information free and still have capitalism. But the problem with that is it reneges on the social contract where people still participate in the formal economy. And it’s a kind of capitalism that’s totally self-defeating because it’s so narrow. It’s a winner-take-all capitalism that’s not sustaining

That isn’t a sustainable situation long-term. A winner-takes-all system centralizes wealth and power at the top, whilst everyone else occupies the long tail. Google has deals in place with large publishers, such as AP, AFP and various European agencies, but this doesn't extend to smaller publishers. It’s the same in sports. The very top get paid ridiculous amounts of money whilst those only a few levels down are unlikely to make rent on their earnings.

But doesn’t technology create new jobs? People who were employed at Kodak just go do something else?

The latest waves of high tech innovation have not created jobs like the old ones did. Iconic new ventures like Facebook employ vastly fewer people than big older companies like, say, General Motors. Put another way, the new schemes.....channel much of the productivity of ordinary people into an informal economy of barter and reputation, while concentrating the extracted old -fashioned wealth for themselves. All activity that takes place over digital networks becomes subject to arbitrage, in the sense that risk is routed to whoever suffers lesser computation resources

The people who will do well in such an environment will likely be employees of those who own the big data networks, like Google. Or they will be the entrepreneurial and adaptable types who manage to get close to them - the companies that serve WalMart or Google, or Facebook, or large financial institutions, or leverage off them - but Lanier argues there simply aren't enough of those roles to sustain society in a way that gave rise to these companies in the first place.

He argues this situation disenfranchises too many people, too quickly. And when that happens, the costs spread to everyone, including the successful owners of the networks. They become poorer than they would otherwise be by not returning enough of the value that enables the very information they need to thrive. Or another way of looking at it - who’s going to buy all the stuff if only a few people have the money?

The network, whether it be a search engine, a social network, an insurance company, or an investment fund uses information to concentrate power. Lanier argues they are all they same as they operate in pretty much the same way. The use network effects to mine and crunch big data, and this, in turn, grows their position at the expense of smaller competitors, and the ecosystem that surrounds them.

It doesn’t really matter what the intent was. The result is that the technology can prevent the middle class from prospering and when that happens, everyone ultimately loses.

So What Does He Propose Can Be Done?

A few days ago, Matt Cutts released a video about what site owners can expect from the next round of Google changes.

Google have announced a web spam change, called Penguin 2.0. They’ll be “looking at” advertorials, and native advertising. They’ll be taking a “stronger line” on this form of publishing. They’ll also be “going upstream” to make link spammers less effective.

Of course, whenever Google release these videos, the webmaster community goes nuts. Google will be making changes, and these changes may either make your day, or send you to the wall.

The most interesting aspect of this, I think, is the power relationship. If you want to do well in Google’s search results then there is no room for negotiation. You either do what they want or you lose out. Or you may do what they want and still lose out. Does the wealth and power sit with the publisher?

Nope.

In other news, Google just zapped another link network.

Cutts warns they’ll be going after a lot of this happening. Does wealth and power sit with the link buyer or seller?

Nope.

Now, Google are right to eliminate or devalue sites that they feel devalues their search engine. Google have made search work. Search was all but dead twelve years ago due to the ease with which publishers could manipulate the results, typically with off-topic junk. The spoils of solving this problem have flowed to Google.

The question is has too much wealth flowed to companies like Google, and is this situation going to kill off large chunks of the ecosystem on which it was built? Google isn’t just a player in this game, they’re so pervasive they may as well be the central planner. Cutts is running product quality control. The customers aren’t the publishers, they’re the advertisers.

It’s also interesting to note what these videos do not say. Cutts video was not about how your business could be more prosperous. It was all about your business doing what Google wants in order for Google to be more prosperous. It’s irrelevant if you disagree or not, as you don’t get to dictate terms to Google.

That’s the deal.

Google’s concern lies not with webmasters just as WalMarts concern lies not with small town retailers. Their concern is to meet company goals and enhance shareholder value. The effects aren’t Google or WalMarts fault. They are just that - effects.

The effect of Google pursuing those objectives might be to gouge out the value of publishing, and in so doing, gouge out a lot of the value of the middle class. The Google self-drive cars project is fascinating from a technical point of view - the view Google tends to focus on - but perhaps even more fascinating when looked at from a position they seldom seem to consider, at least, not in public, namely what happens to all those taxi drivers, and delivery drivers, who get their first break in society doing this work? Typically, these people are immigrants. Typically, they are poor but upwardly mobile.

That societal effect doesn't appear to be Google’s concern.

So who’s concern should it be?

Well, perhaps it really should be Google’s concern, as it’s in their own long-term best interest:

Today, a guitar manufacturer might advertise through Google. But when guitars are someday spun out of 3D printers, there will be no one to buy an ad if guitar designs are “free”. Yet Google’s lifeblood is information put online for free. That is what Google’s servers organize. Thus Google’s current business model is a trap in the longterm

Laniers suggestion is everyone gets paid, via micro-payments, linked back to the value they helped create. These payments continue so long as people are using their stuff, be it a line of code, a photograph, a piece of music, or an article.

For example, if you wrote a blog post, and someone quoted a paragraph of it, you would receive a tiny payment. The more often you’re quoted, the more relevant you are, therefore the more payment you receive. If a search engine indexes your pages, then you receive a micro-payment in return. If people view your pages, you receive a micro-payment. Likewise, when you consume, you pay. If you conduct a search, then you run Google’s code, and Google gets paid. The payments are tiny, as far as the individual is concerned, but they all add up.

Mammoth technical issues of doing this aside, the effect would be to take money from the head and pump it back into the tail. It would be harder to build empires off the previously free content others produce. It would send money back to producers.

It also eliminates the piracy question. Producers would want people to copy, remix and redistribute their content, as the more people that use it, the more money they make. Also, with the integration of two-way linking, the mechanism Lanier proposes to keep track of ownership and credit, you’d always know who is using your content.

Information would no longer be free. It would be affordable, in the broadest sense of the word. There would also be a mechanism to reward the production, and a mechanism to reward the most relevant information the most. The more you contribute to the net, and the more people use it, the more you make. Tiny payments. Incremental. Ongoing.

Interesting Questions

So, if these questions are of interest to you, I’d encourage you to read “Who Owns The Future” by Jaron Lanier. It’s often rambling - in a good way - and heads off on wild tangents - in a good way, and you can tell there is a very intelligent and thoughtful guy behind it all. He’s asking some pretty big, relevant questions. His answers are sketches that should be challenged, argued, debated and enlarged.

And if big tech companies want a challenge that really will change the world, perhaps they could direct all that intellect, wealth and power towards enriching the ecosystem at a pace faster than they potentially gouge it.

What are the incentives to publish high-value content to the web?

Search engines, like Google, say they want to index quality content, but provide little incentive to create and publish it. The reality is that the publishing environment is risky, relatively poorly paid in most instances, and is constantly being undermined.

The Pact

There is little point publishing web content if the cost of publishing outweighs any profit that can be derived from it.

Many publishers, who have search engines in mind, work on an assumption that if they provide content to everyone, including Google, for free, then Google should provide traffic in return. It’s not an official deal, of course. It’s unspoken.

Rightly or wrongly, that’s the “deal” as many webmasters perceive it.

What Actually Happens

Search engines take your information and, if your information is judged sufficiently worthy that day, as the result of an ever-changing, obscure digital editorial mechanism known only to themselves, they will rank you highly, and you’ll receive traffic in return for your efforts.

That may all change tomorrow, of course.

What might also happen is that they could grab your information, amalgamate it, rank you further down the page, and use your information to keep visitors on their own properties.

Look at the case of Trip Advisor. Trip Advisor, frustrated with Google’s use of its travel and review data, filed a competition complaint against Google in 2012.

The company said: "We hope that the commission takes prompt corrective action to ensure a healthy and competitive online environment that will foster innovation across the internet."

The commission has been investigating more than a dozen complaints against Google from rivals, including Microsoft, since November 2010, looking at claims that it discriminates against other services in its search results and manipulates them to promote its own products.

TripAdvisor's hotel and restaurants review site competes with Google Places, which provides reviews and listings of local businesses."We continue to see them putting Google Places results higher in the search results – higher on the page than other natural search results," said Adam Medros, TripAdvisor's vice president for product, in February. "What we are constantly vigilant about is that Google treats relevant content fairly."

Similarly, newspapers have taken aim at Google and other search engines for aggregating their content, and deriving value from that aggregation, but the newspapers claim they aren’t making enough to cover the cost of producing that content in the first place:

In 2009 Rupert Murdoch called Google and other search engines “content kleptomaniacs”. Now cash-strapped newspapers want to put legal pressure on what they see as parasitical news aggregators.”

Of course, it’s not entirely the fault of search engines that newspapers are in decline. Their own aggregation model - bundling news, sport, lifestyle, classifieds topics - into one “place” has been surpassed.

Search engines often change their stance without warning, or can be cryptic about their intentions, often to the determent of content creators. For example, Google has stated they see ads as helpful, useful and informative:

In his argument, Cutts said, “We actually think our ads can be as helpful as the search results in some cases. And no, that’s not a new attitude.”

And again:

we firmly believe that ads can provide useful information

And again:

In entering the advertising market, Google tested our belief that highly relevant advertising can be as useful as search results or other forms of content

However, business models built around the ads as content idea, such as Suite101.com, got hammered. Google could argue these sites went too far, and that they are asserting editorial control, and that may be true, but such cases highlight the flaky and precarious nature of the search ecosystem as far as publishers are concerned. One day, what you're doing is seemingly "good", the next day it is "evil". Punishment is swift and without trial.

Thom Yorke sums it up well:

In the days before we meet, he has been watching a box set of Adam Curtis's BBC series, All Watched Over by Machines of Loving Grace, about the implications of our digitised future, so the arguments are fresh in his head. "We were so into the net around the time of Kid A," he says. "Really thought it might be an amazing way of connecting and communicating. And then very quickly we started having meetings where people started talking about what we did as 'content'. They would show us letters from big media companies offering us millions in some mobile phone deal or whatever it was, and they would say all they need is some content. I was like, what is this 'content' which you describe? Just a filling of time and space with stuff, emotion, so you can sell it?"

Having thought they were subverting the corporate music industry with In Rainbows, he now fears they were inadvertently playing into the hands of Apple and Google and the rest. "They have to keep commodifying things to keep the share price up, but in doing so they have made all content, including music and newspapers, worthless, in order to make their billions. And this is what we want? I still think it will be undermined in some way. It doesn't make sense to me. Anyway, All Watched Over by Machines of Loving Grace. The commodification of human relationships through social networks. Amazing!

There is no question the value of content is being deprecated by big aggregation companies. The overhead of creating well-researched, thoughtful content is the same whether search engines value it or not. And if they do value it, a lot of the value of that content has shifted to the networks, distributors and aggregators and away from the creators.

Facebook’s value is based entirely on the network itself. Almost all of Google’s value is based on scraping and aggregating free content and placing advertising next to it. Little of this value gets distributed back to the creator, unless they take further, deliberate steps to try and capture some back.

In such a precarious environment, what incentive does the publisher have to invest and publish to the “free” web?

Content Deals

Google lives or dies on the relevancy of the information they provide to visitors. Without a steady supply of “free” information from third parties, they don’t have a business.

Of course, this information isn’t free to create. So if search engines do not provide you profitable traffic, then why allow search engines to crawl your pages? They cost you money in terms of bandwidth and may extract, and then re-purpose, the value you created to suit their own objectives.

Google has done content-related deals in the past. They did one in France in February whereby Google agreed to help publishers develop their digital units:

Under the deal, Google agreed to set up a fund, worth 60 million euroes, or $80 million, over three years, to help publishers develop their digital units. The two sides also pledged to deepen business ties, using Google’s online tools, in an effort to generate more online revenue for the publishers, who have struggled to counteract dwindling print revenue.

This seems to fit with Google’s algorithmic emphasis on major web properties, seemingly as a means to sift the "noise in the channel". Such positioning favors big, established content providers.

It may have also been a forced move as Google would have wanted to avoid a protracted battle with European regulators. Whatever the case, Google doesn’t do content deals with small publishers and it could be said they are increasingly marginalizing them due to algorithm shifts that appear to favor larger web publishers over small players.

Don't Be Evil To Whom?

Google’s infamous catch-phrase is “Don’t Be Evil”. In the documentary Inside Google", Eric Schmidt initially thought the phrase was a joke. Soon after, he realized they took it seriously.

The problem with such a phrase is that it implies Google is a benevolent moral actor that cares about......what? You - the webmaster?

Sure.

“Don’t Be Evil” is typically used by Google in reference to users, not webmasters. In practice, it’s not even a question of morality, it’s a question of who to favor. Someone is going to lose, and if you’re a small webmaster with little clout, it’s likely to be you.

For example, Google appear to be kicking a lot of people out of Adsense, and as many webmasters are reporting, Google often act as judge, jury and executioner, without recourse. That’s a very strange way of treating business “partners”, unless partnership has some new definition of which I'm unaware.

It's getting pretty poor when their own previously supportive ex-employees switch to damning their behavior:

But I think Google as an organization has moved on; they're focussed now on market position, not making the world better. Which makes me sad. Google is too powerful, too arrogant, too entrenched to be worth our love. Let them defend themselves, I'd rather devote my emotional energy to the upstarts and startups. They deserve our passion.

Some may call such behavior a long way from “good” on the “good” vs “evil” spectrum.

How To Protect Value

Bottom line: if your business model involves creating valuable content, you’re going to need a strategy to protect it and claw value back from aggregators and networks in order for a content model to be sustainable.

Some argue that if you don’t like Google, then block them using robots.txt. This is one option, but there’s no doubt Google still provides some value - it’s just a matter of deciding where to draw the line on how much value to give away.

What Google offers is potential visitor attention. We need to acquire and hold enough visitor attention before we switch the visitors to desired action. An obvious way to do this, of course, is to provide free, attention grabbing content that offers some value, then lock the high value content away behind a paywall. Be careful about page length. As HubPages CEO Paul Edmonds points out:

Longer, richer pages are more expensive to create, but our data shows that as the quality of a page increases, its effective revenue decreases. There will have to be a pretty significant shift in traffic to higher quality pages to make them financially viable to create"

You should also consider giving the search engines summaries or the first section of an article, but block them from the rest.

Even if you decide to block search engines from indexing your content they still might pay others to re-purpose it:

I know a little bit about this because in January I was invited to a meeting at the A.P.’s headquarters with about two dozen other publishers, most of them from the print world, to discuss the formation of the consortium. TechCrunch has not joined at this time. Ironically, neither has the A.P., which has apparently decided to go its own way and fight the encroachments of the Web more aggressively (although, to my knowledge, it still uses Attributor’s technology). But at that meeting, which was organized by Attributor, a couple slides were shown that really brought home the point to everyone in the room. One showed a series of bar graphs estimating how much ad revenues splogs were making simply from the feeds of everyone in the room. (Note that this was just for sites taking extensive copies of articles, not simply quoting). The numbers ranged from $13 million (assuming a $.25 effective CPM) to $51 million (assuming a $1.00 eCPM)

You still end up facing the cost of policing "content re-purposing" - just one of the many costs publishers face when publishing on the web, and just one more area where the network is sucking out value.

Use multiple channels so you’re not reliant on one traffic provider. You might segment your approach by providing some value to one channel, and some value to another, but not all of it to both. This is not to say models entirely reliant on Google won’t work, but if you do rely on a constant supply of new visitors via Google, and if you don’t have the luxury of having sufficient brand reputation, then consider running multiple sites that use different optimization strategies so that the inevitable algorithm changes won’t take you out entirely. It’s a mistake to think Google cares deeply about your business.

Treat every new visitor as gold. Look for ways to lock visitors in so you aren’t reliant on Google in future for a constant stream of new traffic. Encourage bookmarking, email sign-ups, memberships, rewards - whatever it takes to keep them. Encourage people to talk about you across other media, such as social media. Look for ways to turn visitors into broadcasters.

Adopt a business model that leverages off your content. Many consultants write business books. They make some money from the books, but the books mainly serve as advertisements for their services or speaking engagements. Similarly, would you be better creating a book and publishing it on Amazon than publishing too much content to the web?

Business models focused on getting Google traffic and then monetarizing that attention using advertising only works if the advertising revenue covers production cost. Some sites make a lot of money this way, but big money content sites are in the minority. Given the low return of a lot of web advertising, other webmasters opt for cheap content production. But cheap content isn’t likely to get the attention required these days, unless you happen to be Wikipedia.

Perhaps a better approach for those starting out is to focus on building brand / engagement / awarenesss / publicity / non-search distribution. As Aaron points out:

...the sorts of things that PR folks & brand managers focus on. The reason being is that if you have those things...

• the incremental distribution helps subsidize the content creation & marketing costs
• many of the links happen automatically (such that you don't need to spend as much on links & if/when you massage some other stuff in, it is mixed against a broader base of stuff)
• that incremental distribution provides leverage in terms of upstream product suppliers (eg: pricing leverage) or who you are able to partner with & how (think about Mint.com co-marketing with someone or the WhiteHouse doing a presentation with CreditCards.com ... in addition to celebrity stuff & such ... or think of all the ways Amazon can sell things: rentals, digital, physical, discounts via sites like Woot, higher margin high fashion on sites like Zappos, etc etc etc)
• as Google folds usage data & new signals in, you win
• as Google tracks users more aggressively (Android + Chrome + Kansas City ISP), you win
• if/when/as Google eventually puts some weight on social you win
• people are more likely to buy since they already know/trust you
• if anyone in your industry has a mobile app that is widely used & you are the lead site in the category you could either buy them out or be that app maker to gain further distribution
• Google engineers are less likely to curb you knowing that you have an audience of rabid fans & they are more likely to consider your view if you can mobilize that audience against "unjust editorial actions"

A lot of the most valuable content on this site is locked-up. We’d love to open this content up, but there is currently no model that sufficiently rewards publishers for doing so. This is the case across the web, and it's the reason the most valuable content is not in Google.

It’s not in Google because Google, and the other search engines, don’t pay.

Fair? Unfair? Is there a better way? How can content providers - particularly newcomers - grow and prosper in such an environment?

Growing search marketshare is hard work. At a recent investor conference Marissa Mayer stated that: "The key pieces are around the underpinnings of the alliance themselves. The point is, we collectively want to grow share, rather than trading share with each other."

Part of the reason Yahoo! & Bing struggle to gain marketshare is Google's default search placement payments to Mozilla and Apple. If the associated browsers have nearly 1/3 the market & Chrome is another 1/3 of the market then it requires Yahoo! or Bing to be vastly better than Google to break the Google habit + default search placement purchases.

Danny reported some interesting comments from Nikesh Arora:

• half of those billions of queries it handles comes from Google partners, rather than searches at Google directly.
• Arora also said that he expects about 50% of advertising to move online in the next three to five years.
• he just said ad team looks at ways to make ads not look like ads. I think he meant that positively, like content you want.

A friend sent me a screenshot where he was surprised how similar the results looked between Bing & Google.

If Bing looks too different it feels out of place, if it looks to similar it doesn't feel memorable. And if Google is optimized for revenue generation then Bing is going to have a fairly similar look & feel to their results if they want to earn enough to bid on partnerships.

Another factor helping Google maintain their dominance in search marketshare is the shift of search query mix to mobile, where Google has a 95.8% marketshare.

Mobile search has a significantly higher CTR than desktop search, due in large part to there being less screen real estate. By the end of this year tablets will likely account for 20% of Google's search ad clicks & drive $5 billion in ad revenues. Add in mobile phones with tablets & mobile search will drive 1/3 of paid search clicks by the end of this year.

With mobile becoming such a huge share of search clicks Google is forcing advertisers into buying all platforms with their ad purchase via their enhanced AdWords campaigns. Google builds off that sort of dominance & Yahoo! is only making about $125 million a year in total revenue from Yahoo!'s mobile traffic.

In spite of losing share on browser defaults & mobile, Yahoo! managed to grow their search ad clicks 11% year over year. How was Yahoo! able to do that? In part by quietly dialing up on search arbitrage. They have long had a "trending now" box on their homepage, but over the past year they have dialed up on ads in their news, finance & sports sections that are linked to search queries. Some of these ad units are in the sidebar & some are inline with the articles.

Yahoo! also buys ads on some smaller ad networks & sends those through to a search result with almost no organic results.

Yahoo! has had a long history of search arbitrage, but they typically did it through a partner network which lowered click value. That was part of what lowered their click prices & made them sign the deal with Microsoft (you couldn't even opt out of Yahoo!'s partner syndication until after they signed the deal with Microsoft).

I recently saw the above ad for Bing which highlighted how they want to work with brands, but Bing still has a number of issues they are dealing with on the monetization front: tighter broad matching, smaller ad ecosystem, regional issues with ad targeting, and no serious effort to develop a contextual ad program open to the long tail of publishers. In spite of those issues, the Yahoo! / Bing ad network was finally starting to build a critical mass & Yahoo! responded by signing a deal to carry Google's contextual AdSense ads.

As Google continues to layer contextual search layers into mobile devices, launch their own physical stores, layer their social network into the search ecosystem, expand their venture investments, inserts themselves at an ISP level, shape the news, control a greater share of ad budget with programmatic bidding, control measurements of success, redefine words, scrape-n-displace publishers with the knowledge graph, de-fund competitors, & hyper-target ads at users, their leverage & market dominance will only grow.

Google is great at growing the search pie.

Yahoo!, not so much. ;)

Due to heavy lobbying, the FTC's investigation into Google's business practices has ended with few marks or bruises on Google's behalf. If the EU has similar results, you can count on Google growing more anti-competitive in their practices:

Google is flat-out lying. They’ve modified their code to break Google Maps on Windows Phones. It worked before, but with the ‘redirect,’ it no longer works.

We are only a couple days into the new year, but there have already been numerous absurdities highlighted, in addition to the FTC decision & Google blocking Windows Phones.

When is Cloaking, Cloaking?

Don't ask Larry Page:

Mr. Page, the CEO, about a year ago pushed the idea of requiring Google users to sign on to their Google+ accounts simply to view reviews of businesses, the people say. Google executives persuaded him not to pursue that strategy, fearing it would irritate Google search users, the people say.
...
Links to Google+ also appear in Google search-engine results involving people and brands that have set up a Google+ account.

Other websites can't hardcode their own listings into the search results. But anyone who widely attempted showing things to Googlebot while cloaking them to users would stand a good chance of being penalized for their spam. They would risk both a manual intervention & being hit by Panda based on poor engagement metrics.

Recall that a big portion of the complaint about Google's business practices was their scrape-n-displace modus operandi. As part of the FTC agreement, companies are able to opt out of being scraped into some of Google's vertical offerings, but that still doesn't prevent their content from making its way into the knowledge graph.

Now that Google is no longer free to scrape-n-displace competitors, apparently the parallel Google version of that type of content that should be "free and open to all to improve user experience" (when owned by a 3rd party) is a premium feature locked behind a registration wall (when owned by Google). There is a teaser for the cloaked information in the SERPs, & you are officially invited to sign into Google & join Google+ if you would like to view more.

Information wants to be free.

Unless it is Google's.

Then users want to be tracked and monetized.

Trademark Violations & Copyright Spam

A few years back Google gave themselves a pat on the back for ending relationships with "approximately 50,000 AdWords accounts for attempting to advertise counterfeit goods."

How the problem grew to that scale before being addressed went unasked.

Last year Google announced a relevancy signal based on DMCA complaints (while exempting YouTube) & even nuked an AdSense publisher for linking to a torrent of his own ebook. Google sees a stray link, makes a presumption. If they are wrong and you have access to media channels then the issue might get fixed. But if you lack the ability to get coverage, you're toast.

Years ago a study highlighted how Google's AdSense & DoubleClick were the monetization engine for stolen content. Recently some USC researchers came to the same conclusion by looking at Google's list of domains that saw the most DMCA requests against them. Upon hearing of the recent study, Google's shady public relations team stated:

"To the extent [the study] suggests that Google ads are a major source of funds for major pirate sites, we believe it is mistaken," a Google spokesperson said. "Over the past several years, we've taken a leadership role in this fight. The complexity of online advertising has led some to conclude, incorrectly, that the mere presence of any Google code on a site means financial support from Google."

So Google intentionally avails their infrastructure to people they believe are engaged in criminal conduct (based on their own 50,000,000+ "valid" DMCA findings) and yet Google claims to have zero responsibility for those actions because Google may, in some cases, not get a direct taste in the revenues (only benefiting indirectly through increasing the operating costs of running a publishing business that is not partnered with Google).

A smaller company engaged in a similar operation might end up getting charged for the conduct of their partners. However, when Google's ad code is in the page you are wrong to assume any relationship.

The above linked LA Times article also had the following quote in it:

"When our ads were running unbeknownst to us on these pirate sites, we had a serious problem with that," said Gareth Hornberger, senior manager of global digital marketing for Levi's. "We reached out to our global ad agency of record, OMD, and immediately had them remove them.... We made a point, moving forward, that we really need to take steps to avoid having these problems again."

Through Google's reality warping efforts the ad network, the ad agency, the publisher, and the advertiser are all entirely unaccountable for their own efforts & revenue streams. And it is not like Google or the large ad agencies lack the resources to deal with these issues, as there is some serious cash in these types of deals: "WPP, Google's largest customer, increased its spending on Google by 25% in 2012, to about $2 billion."

These multi-billion Dollar budgets are insufficient funds to police the associated activities. Whenever anything is mentioned in the media, mention system complexity & other forms of plausible deniability. When that falls short, outsource the blame onto a contractor, service provider, or rogue partner. Contrasting that behavior, the common peasant webmaster must proactively monitor the rest of the web to ensure he stays in the graces of his Lord Google.

DMCA Spam

You have to police your user generated content, or you risk your site being scored as spam. With that in mind, many big companies are now filing false DMCA takedown requests. Sites that receive DMCA complaints need to address them or risk being penalized. Businesses that send out bogus DMCA requests have no repercussions (until they are eventually hit with a class action lawsuit).

Remember how a while back Google mentioned their sophisticated duplication detection technology in YouTube?

There are over a million full movies on YouTube, according to YouTube!

The other thing that is outrageous is that if someone takes a video that is already on YouTube & re-uploads it again, Google will sometimes outrank the original video with the spam shag-n-republish.

In the below search result you can see that our video (the one with the Excel spreadsheet open) is listed in the SERPs 3 times.

The version we uploaded has over a quarter million views, but ranks below the spam syndication version with under 100 views.

There are only 3 ways to describe how the above can happen:

• a negative ranking factor against our account
• horrible relevancy algorithms
• idiocy

I realize I could DMCA them, but why should I have to bear that additional cost when Google allegedly automatically solved this problem years ago?

Link Spam

Unlike sacrosanct ad code, if someone points spam links at your site, you are responsible for cleaning it up. The absurdity of this contrast is only further highlighted by the post Google did about cleaning up spam links, where one of the examples they highlighted publicly as link spam was not a person's spam efforts, but rather a competitor's sabotage efforts that worked so well that they were even publicly cited as being outrageous link spam.

It has been less than 3 months since Google launched their disavow tool, but since it's launch some webmasters are already engaging in pre-negative SEO. That post had an interesting comment on it:

Well Mr Cutts, you have created a monster in Google now im afraid. Your video here http://www.youtube.com/watch?v=HWJUU-g5U_I says that with the new disavow tool makes negative SEO a mere nuisance.
Yet in your previous video about the diavow tool you say it can take months for links to be disavowed as google waits to crawl them???
In the meantime, the time lag makes it a little more than a "nuisance" don't you think?

Where Does This Leave Us?

As Google keeps adding more advanced filters to their search engines & folding more usage data into their relevancy algorithms, they are essentially gutting small online businesses. As Google guts them, it was important to offer a counter message of inclusion. A WSJ articles mentioned that Google's "get your business online" initiative was more effective at manipulating governmental officials than their other lobbying efforts. And that opinion was sourced from Google's lobbyists:

Some Washington lobbyists, including those who have done work for Google, said that the Get Your Business Online effort has perhaps had more impact on federal lawmakers than any lobbying done on Capitol Hill.

Each of the additional junk time wasting tasks (eg: monitoring backlinks and proactively filtering them, managing inventory & cashflow while waiting for penalties tied to competitive sabotage to clear, filing DMCAs against Google properties when Google claims to have fixed the issue years ago, merging Google Places listings into Google+, etc.) Google foists onto webmasters who run small operations guarantees that a greater share of them will eventually get tripped up.

Not only will the algorithms be out of their reach, but so will consulting.

That algorithmic approach will also only feed into further "market for lemons" aspects as consultants skip the low margin, small budget, heavy lifting jobs and focus exclusively on servicing the companies which Google is biasing their "relevancy" algorithms to promote in order to taste a larger share of their ad budgets.

While chatting with a friend earlier today he had this to say:

Business is arbitrage. Any exchange not based in fraud is legitimate regardless of volume or medium. The mediums choose to delegitimize smaller players as a way to consolidate power.

Sadly most journalists are willfully ignorant of the above biases & literally nobody is comparing the above sorts of behaviors against each other. Most people inside the SEO industry also avoid the topic, because it is easier (& more profitable) to work with the elephants & attribute their success to your own efforts than it is highlight the holes in the official propaganda.

I mean, just look at all the great work David Naylor did for a smaller client here & Google still gave him the ole "screw you" in spite of doing just about everything possible within his control.

The linkbuilding tactics used by the SEO company on datalabel.co.uk were low quality, but the links were completely removed before a Reconsideration Request was filed. The MD’s commenting and directory submissions were done in good faith as ways to spread the word about his business. Despite a lengthy explanation to Google, a well-documented clean-up process, and eventually disavowing every link to the site, the domain has never recovered and still violates Google’s guidelines.

If you’ve removed or disavowed every link, and even rebuilt the site itself, where do you go from there?

Sharing is caring!

Please share :)

Embed code is here.

If you find the following a bit hard to read due to font size, a wider version is located here.