Back in 2009 Google executives were scared of not being able to retain talent with stock options after Google's stock price cratered with the rest of the market & Google's ad revenue growth rate slid to zero. That led them to reprice employee stock options. That is as close as Google has come to a "near death" experience since their IPO. They've consistently grown & become more dominant.
In 2012 a Googler named Jon Rockway was more candid than Googlers are typically known for being: "SEO isn't good for users or the Internet at large. ... It's a bug that you could rank highly in Google without buying ads, and Google is trying to fix the bug."
"If you want to stop spam, the most straight forward way to do it is to deny people money because they care about the money and that should be their end goal. But if you really want to stop spam, it is a little bit mean, but what you want to do, is break their spirits." - Matt Cutts
Through a constant ex-post-facto redefinition of "what is spam" to include most anything which is profitable, predictable & accessible, Google engineers work hard to "deny people money."
Over time SEO became harder & less predictable. The exception being Google investments like Thumbtack, in which case other's headwind became your tailwind & a list of techniques declared off limits became a strategy guidebook.
"We also like that it means alpha‑bet (Alpha is investment return above benchmark), which we strive for!" - Larry Page
From Do/Know/Go to Scrape/Displace/Monetize
It takes a lot of effort & most people are probably too lazy to do it, but if you look at the arch of Google's patents related to search quality, many of the early ones revolved around links. Then many focused on engagement related signals. Chrome & Android changed the pool of signals Google had access to. Things like Project Fi, Gogle Fiber, Nest, and Google's new OnHub router give them more of that juicy user data. Many of their recently approved patents revolve around expanding the knowledge graph so that they may outright displace the idea of having a neutral third party result set for an increasing share of the overall search pie.
Searchers can instead get bits of "knowledge" dressed in various flavors of ads.
This sort of displacement is having a significant impact on a variety of sites. But for most it is a slow bleed rather than an overnight sudden shift. In that sort of environment, even volunteer run sites will eventually atrophy. They will have fewer new users, and as some of the senior people leave, eventually fewer will rise through the ranks. Or perhaps a greater share of the overall ranks will be driven by money.
Jimmy Wales stated: “It is also false that ‘Wikipedia thrives on clicks,’ at least as compared to ad-revenue driven sites… The relationship between ‘clicks’ and the things we care about: community health and encyclopedia quality is not nothing, but it’s not as direct as some think.”
Most likely the relationship *is* quite direct, but there is a lagging impact. Today's major editors didn't join the site yesterday & take time to rise through the ranks.
If Google works hard enough at prioritizing "deny people money" as a primary goal, then they will eventually get an index quality that reflects that lack of payment. Plenty of good looking & well-formatted content, but a mix of content which:
is monetized indirectly & in ways which are not clearly disclosed
has interstitial ads and slideshows where the ads look like the "next" button & the "next" button is colored the same color as the site's background
There has been a general pattern in search innovation. Google introduces a new feature, pitches it as being the next big thing, gets people to adopt it, collects data on the impact of the feature, clamps down on selectively allowing it, perhaps removes the feature outright from organic search results, permanently adds the feature to their ad units.
This sort of pattern has happened so many times it is hard to count.
Google puts faces in search results for authorship & to promote Google+, Google realizes Google+ is a total loser & disconnects it, new ad units for local services show faces in the search results. What was distracting noise was removed, then it was re-introduced as part of an ad unit.
Some sites which bundle software got penalized in organic search and are not even allowed to buy AdWords ads. At an extreme degree, sites which bundled no software, but simply didn't link to an End User Agreement (EULA) from the download page were penalized. Which leads to uncomfortable conversations like this one:
Google Support: I looked through this, and it seemed that one of the issues was a lack of an End User Agreement (EULA)
Google Support: Hmm, They do want it on the download page itself
Simtec: How come there isn’t one here? google.co.uk/chrome/browser/desktop/
Google Support: LOL
Simtec: No really?
Google Support: That’s a great question
Of course, it goes without saying that much of the Google Chrome install base came from negative option software bundling on Adobe Flash security updates.
Google claimed helpful hotel affiliate sites should be rated as spam, then they put their own affiliate ads in hotel search results & even recommended hotel searches in the knowledge graph on city name searches.
Google search engineers have recently started complaining about interstitial ads & suggested they might create a "relevancy" signal based on users not liking those. At the same time, an increasing number of YouTube videos have unskippable pre-roll ads. And the volume of YouTube ad views is so large that it is heavily driving down Google's aggregate ad click price. On top of this, Google also offers a survey tool which publishers can lock content behind & requires users to answer a question before they can see the full article they just saw ranking in the search results.
"Everything is possible, but nothing is real." - Living Colour
Blue Ocean Opportunity
Amid the growing ecosystem instability & increasing hypocrisy, there have perhaps been only a couple "blue ocean" areas left in organic search: local search & brand.
And it appears Google might be well on their way in trying to take those away.
For years brand has been the solution to almost any SEO problem.
I wonder how many SEOs working for big brands have done absolutely nothing of value since 2012 yet still look like geniuses to executives.— Ross Hudgens (@RossHudgens) August 7, 2015
Now that the mobile search interface is literally nothing but ads above the fold, early data shows a significant increase in mobile ad clicks. Of course it doesn't matter if there are 2 or 3 ads, if Google shows ad extensions on SERPs with only 2 ads to ensure they drive the organic results "out of sight, out of mind."
Earlier this month it was also noticed Google replaced 7-pack local results with 3-pack local results for many more search queries, even on desktop search results. On some of these results they only show a call button, on others they show links to sites. It is a stark contrast to the vast array of arbitrary (and even automated) ad extensions in AdWords.
Why would they determine users want to see links to the websites & the phone numbers, then decide overnight users don't want those?
Why would Google determine for many years that 7 is a good number of results to show, and then overnight shift to showing 3?
If Google listed 7 ads in a row people might notice the absurdity of it and complain. But if Google only shows 3 results, then they can quickly convert it into an ad unit with little blowback.
You don't have to be a country music fan to know the Austin SEO limits in a search result where the local results are now payola.
Try not to hurt your back while looking down for the organic search results!
Here are two tips to ensure any SEO success isn't ethereal: don't be nearby, and don't be a business. :D