Starting next week, we will begin taking into account a new signal in our rankings: the number of valid copyright removal notices we receive for any given site. Sites with high numbers of removal notices may appear lower in our results.
This data presents information specified in requests we received from copyright owners through our web form to remove search results that link to allegedly infringing content. It is a partial historical record that includes more than 95% of the volume of copyright removal requests that we have received for Search since July 2011. It does not include:
requests submitted by means other than our web form, such as fax or written letter
requests for products other than Google Search (e.g, requests directed at YouTube or Blogger)
requests sent to Google Search for content appearing in other Google products (e.g., requests for Search, but specifying YouTube or Blogger URLs).
Google does not state where the thresholds will be set & grants blanket immunity for themselves, yet they (illegitimately) emphasize that they are being transparent.
Only copyright holders know if something is authorized, and only courts can decide if a copyright has been infringed; Google cannot determine whether a particular webpage does or does not violate copyright law. So while this new signal will influence the ranking of some search results, we won’t be removing any pages from search results unless we receive a valid copyright removal notice from the rights owner. And we’ll continue to provide "counter-notice" tools so that those who believe their content has been wrongly removed can get it reinstated. We’ll also continue to be transparent about copyright removals.
Kim DotCom suffered a militant raid on his house & had his assets frozen for running MegaUpload, which was a tiny spec of dirt compared to the size of YouTube.
"In a July 19, 2005 e-mail to YouTube co-founders Chad Hurley and Jawed Karim, YouTube co-founder Steve Chen wrote: 'jawed, please stop putting stolen videos on the site. We’re going to have a tough time defending the fact that we’re not liable for the copyrighted material on the site because we didn’t put it up when one of the co-founders is blatantly stealing content from other sites and trying to get everyone to see it.'"
"Chen twice wrote that 80 percent of user traffic depended on pirated videos. He opposed removing infringing videos on the ground that 'if you remove the potential copyright infringements... site traffic and virality will drop to maybe 20 percent of what it is.' Karim proposed they 'just remove the obviously copyright infringing stuff.' But Chen again insisted that even if they removed only such obviously infringing clips, site traffic would drop at least 80 percent. ('if [we] remove all that content[,] we go from 100,000 views a day down to about 20,000 views or maybe even lower')."
"In response to YouTube co-founder Chad Hurley’s August 9, 2005 e-mail, YouTube co-founder Steve Chen stated: 'but we should just keep that stuff on the site. I really don’t see what will happen. what? someone from cnn sees it? he happens to be someone with power? he happens to want to take it down right away. he get in touch with cnn legal. 2 weeks later, we get a cease & desist letter. we take the video down.'"
"A true smoking gun is a memorandum personally distributed by founder Karim to YouTube’s entire board of directors at a March 22, 2006 board meeting. Its words are pointed, powerful, and unambiguous. Karim told the YouTube board point-blank:
'As of today episodes and clips of the following well-known shows can still be found: Family Guy, South Park, MTV Cribs, Daily Show, Reno 911, Dave Chapelle. This content is an easy target for critics who claim that copyrighted content is entirely responsible for YouTube’s popularity. Although YouTube is not legally required to monitor content (as we have explained in the press) and complies with DMCA takedown requests, we would benefit from preemptively removing content that is blatantly illegal and likely to attract criticism.'"
"A month later, [YouTube manager Maryrose] Dunton told another senior YouTube employee in an instant message that 'the truth of the matter is probably 75-80 percent of our views come from copyrighted material.' She agreed with the other employee that YouTube has some 'good original content' but 'it’s just such a small percentage.'"
"In a September 1, 2005 email to YouTube co-founder Steve Chen and all YouTube employees, YouTube co-founder Jawed Karim stated, 'well, we SHOULD take down any: 1) movies 2) TV shows. we should KEEP: 1) news clips 2) comedy clips (Conan, Leno, etc) 3) music videos. In the future, I’d also reject these last three but not yet.'"
Broader Copyright Questions
There still are a lot of murky questions in Google's "transparency."
If a person embeds an image from Imgur, ImageShack, TinyPic, PhotoBucket or elsewhere & the page that has a hotlink gets a DMCA how does that count?
If a brand is large enough does it take many DMCAs to get hit?
Is there any analysis of the underlying business model of the site? What happens to document storage sites like DocStoc & Scribd, or even image sites like Pinterest?
What happens to sites that link at penalized sites too frequently?
Has anyone registered DMCASEO.com & DMCA-SEO.com yet? ;)
In terms of impact on the web for publishers, this change is every bit as big as Florida, Panda & Penguin. It may not seem so at first (as it will take time for market participants to consider the uses) but this is a huge deal. Consider some of the following scenarios...
You try to create something like YouTube for another form of content (Pinterest?) and it gets hit as spam for following Google's lead.
You offer a free blogging platform that competes with Blogspot, but it gets hit as spam for following Google's lead.
You decide to create a project like Google's book scanning project & you get hit as spam for following Google's lead.
You create a new social network & are struggling to compete with Google's preferential ranking & hard coded placements of their own network. You make your network more open to encourage growth & you get hit as spam.
If You are Amazon or eBay you can afford premium featured content to pull up your other listings. But if you can't afford their cost structure & hire freelance writers or work with outsourced workers to create some of your content & they use some copyright work without you knowing. But does Amazon now have to vigilantly review their reviews for plagiarism?
A competitor licenses some of their content as Creative Commons for years & doesn't mind wide use of it. Then you use it & one day they see you as a competitive threat and remove their Creative Commons license & bulk DMCA you. Or you have a lifetime syndication deal with a company, they later change the policy & claim that your documents are forged.
Getty images presumes you didn't license an image that you did & files a DMCA. At some point there is no purpose in targeting the webmaster or host...just go direct to Google knowing that you can create the equivalent of a "patent trolling" styled business model where you create a business model where it is cheaper for people to pay to have the issue resolved the quick way before they lodge a formal complaint. Some organizations might even have a subscription service set up where you pre-pay for immunity.
A former employee who wrote content for you claims you used it without permission. Or that same former employee used pirated images & longish quotes from other sources that they didn't disclose to you that they now highlight via DMCA.
You license data from a source & they do a mid-contract change leveraging the small print & have a bot lined up to send 40,000 DMCAs against you if you do not agree to the higher pricepoint.
Google is considering making an investment in your site & you want too much money. As an edge case near the threshold of this copyright limit you know you have immunity if you join the borg, but lack it if you don't work with them.
Your leading competitor realizes that your blog publishes comments by default with editorial review (and that even later has lax review) and then they file DMCA reports against you. Or they could just grab chunks of content from Google's leaderboard of complainers and post them into your web forum, knowing that those companies will file a DMCA report against you.
A site has some content public & some behind a paywall. With a page partially indexed, how does Google respond to DMCA requests when the alleged infraction is behind a registration wall or paywall?
A competitor (inspired by Google no doubt) hires off shore "contractors" to copy your site & then file DMCA reports against you in bulk. How long until people start uploading their own content to file their own DMCAs against certain sites with user generated content?
Even if your site is 100% legal, a combination of ignorance & crowd-driven vigilante justice can still take you down.
Any site that offers interactive features & has user generated content is at risk of being labeled as spam unless they have tight editorial control over user generated content. And at the same time, Google can enter vertical after vertical with scrape & displace garbage knowing that they don't have those editorial costs due to their self-granted blanket immunity.
If you do not register your sites with Google & counter claims (even bogus ones) then you are seen as being a spammer. And if you register with Google then when they don't like something one site does they can hit other sites all at the same time. No point going to the host or registrar, go direct to Google & start building up negative karma.
Why did Google feel the need to grant themselves blanket immunity from the policy?
That question was largely missing among the fanboi blogs & journalists who were encouraged by Google's "transparency."
24 Karat Pyrite On Sale for Only $100 an Ounce
If YouTube is going to win big, then that's a great place to invest, right?
Google is also investing in select channels (like Machinima). It is quite hard to outperform Google in returns while investing into a platform that they control & thus have better data on than you ever could.
As YouTube's dominance increases (and it will now that competing platforms with a similar business model will be smeared as spam), you can count on them offering premium partners crappier revenue share deals in years to come. They will offer nice deals to Warner Bros. & such, but the independent smaller players will get cut out of the ecosystem in much the same way as they did in Google's organic search results.
Google, prince of transparency (for everyone but Google), requires that premium publishers *not* disclose the terms of their deals: "The Partner Program forbids participants to reveal specifics about their ad-share revenue. Rates can vary depending on the size and demographics of the partner’s audience and an array of other metrics."
Note that I don't claim YouTube is a bad host for your own content, but that I am skeptical in applying the VC model to it with a belief that you can out-invest Google on their own site; particularly when they own the dominant platform, control the non-public revenue share rates, invest in competing channels & can offer free promotion + higher rates to anyone they invest into in order to dominate the category.
And the issue isn't just video either. The same dynamic can apply to just about any other infrastructural layer. For instance, Google could buy out a torrent site (say like uTorrent) and have that site gain immediately immunity for being part of the borg, while other sites that compete now absorb both greater editorial filtering costs & greater risks that destroy their ROI.
As Google continues to lock down search, you can expect more smart publishers to hedge investments in search and YouTube with investments in proprietary non-search applications that Google can't take away.
The Devil is in the Details
"We are optimistic that Google’s actions will help steer consumers to the myriad legitimate ways for them to access movies and TV shows online, and away from the rogue cyberlockers, peer-to-peer sites, and other outlaw enterprises that steal the hard work of creators across the globe. We will be watching this development closely — the devil is always in the details — and look forward to Google taking further steps to ensure that its services favor legitimate businesses and creators, not thieves." - Michael O’Leary, Senior Executive Vice President for Global Policy and External Affairs of the Motion Picture Association of America, Inc.
The concerned with Google pitching themselves as the preeminent authority on copyright is they have consistently played both sides of the fence.
Their image search offers thumbnails
they offer a "quick view" version of PDFs in the SERPs
their search results offer cached versions of pages
for years they included piracy-related searches in Google Instant (along with the above mentioned ads) to try to force brands to need to buy their own brand on broad match to try to mask over the problem
they scraped content from Yelp & TripAdvisor to populate their places page while giving those businesses the ultimatum that if they didn't like it, they should block Googlebot (and it took government scrutiny to curtail the practice)
Now that Google wants to sell premium content they (sort of) respect copyright (& are willing to hold the rest of the web to a higher standard than themselves to create this impression).
I have long believed that relevancy signals were often politically driven & that internal business development goals often lead or create various signals. Certainly that was obvious when Google+ was hardcoded in the search results. It was equally true when Knol outranked the original content sources. Google frequently pretends to be (belligerently) unaware of externalities, but when the issues impact their own business they gain an elevated sense of importance.
We saw it again when universal search was rolled out only *after* Google bought YouTube.
We saw once more when their hard stance on (against?) property rights for a decade quickly flipped like a switch after Android Marketplace & Google Play stunk up the joint from not making enough sales to be significant. Google needs a complete media catalog to become a default purchase hub & they can't get that until they display that they respect property rights.
And these business objectives not only influence the relevancy algorithms, but also the editorial guidelines.
And even while Google is rolling out this "copyright violators are spammers" algorithm (which they are exempt from) they still chug on with their ebook offering:
They posted several of my 41 books up as free downloads (some were missing a few pages at most a single chapter) It took several e-mails from me pointing out that they were infringing copyright before they took them down. During the time my books were free on Google my sales of e-books fell dramatically. " - K C Watkins
When Google started scanning books an internal document stated: “[we want web searchers interested in book content to come to Google not Amazon” ... or, as put another way, in that same document, “[e]verything else is secondary … but make money.”
Worstream recently put out an infographic where they suggested that 64.6% of search result clicks on highly commercial keywords are clicks on AdWords ads. Shortly before Google's quarterly announcement RKG put out their digital marketing report. In it they highlight how search ad CTR differs by device.
What causes a higher CTR on cell phones & tablets? A smaller search interface, which allows ads to dominate a larger portion of the screen real estate.
Screen Real Estate
Vertical iPhone = 1/3 of an organic listing above the fold.
Horizontal iPhone = all ads above the fold.
Vertical iPad is about 2/3 ads above the fold.
Horizontal iPad has about half of a single organic listing above the fold.
Google offers sitelinks when they think a search query is navigational in nature. In spite of that, for some brands they will still show 3 AdWords ads above the organic search results, in an attempt to force the brand to re-buy their own brand equity.
If you control what is above the fold (and can get away with serving nothing but ads above the fold) you can make a lot of money.
Gain a Competitive Advantage Today
Your top competitors have been investing into their marketing strategy for years.
Now you can know exactly where they rank, pick off their best keywords, and track new opportunities as they emerge.
Explore the ranking profile of your competitors in Google and Bing today using SEMrush.
Enter a competing URL below to quickly gain access to their organic & paid search performance history - for free.
See where they rank & beat them!
Comprehensive competitive data: research performance across organic search, AdWords, Bing ads, video, display ads, and more.
Compare Across Channels: use someone's AdWords strategy to drive your SEO growth, or use their SEO strategy to invest in paid search.
Global footprint: Tracks Google results for 120+ million keywords in many languages across 28 markets
Historical data: since 2009, before Panda and Penguin existed, so you can look for historical penalties and other potential ranking issues.
When you think of all the implications of the above video (and all the things that are going on in machine learning & search), it can be somewhat difficult to think about sustainable strategies.
When platforms are new they start off as being fairly open to win attention & maximize their growth rates. Over time as they push to monetize they shift gears & what was once true becomes misleading. Thus a lot people likely come off as sounding like quack jobs because they keep having to reinvent themselves & reassess their belief systems as the markets change.
Hello Mr. Cynic
If you write things that sound like rants & complaints a lot of people mistake it as thinking you are a crank full of gloom & nonsense. For what it is worth, in many ways I think the future of the web will still be bright, but just relatively less bright than it was in the recent past for smaller players.
The web is becoming more & more like the physical world (andis mergingwith it). For a long time search & online was largely a meritocracy, where the best person could easily win even if they came from the most humble beginnings.
In search of years gone by, large & complex organizations that were overly bloated and inefficient routinely had their asses handed to them by smaller & more efficient operations. Butthensizebecame a primary signal of relevancy & quality, and that all changed. As Larry Page & Sergey Brin warned, the relevancy algorithms inevitably follow the underlying business model of the search engine.
That is a big part of the disillusionment with Google. For many years they were a leveler which was concerned primarily with quality. That grew the importance of search & differentiated them from everyone else, but then they decided to be "the same" & so many who promoted them felt a bit betrayed.
If a person gives you something and then takes it away you likely view them worse than someone who simply never offered that in the first place. As a species we are biologically aligned with being adverse toward loss.
Vertical AND Horizontal Integration
I was chatting with a friend about the above trend & his responses were:
"you don't shoot the guy that didn't give you the job; you shoot the guy that gave you the job and then fired you"
"their public image as being a leveler becomes more grating too, given how much they no longer represent that"
"the biggest problem we have in search is that search engines don't view themselves as a medium. They want to be the cable operator + television show + in-show advertising + commercials...I'm not aware of another medium where it works that way"
If Google's internal stats show someone else is catching up to a channel they invested in, Google can...
relay this news across to drive editorial quality, content quantity, or even ad placement
preferentially promote the network they are invested in (free ads, better rankings, more "you might also like" recommendations, more post-view recommendations)
give a higher revenue share to the network they are invested in (or offer them early access to new betas and exclusives that increase monetization)
slow the growth of the competing network by using more aggressive ad placement (or lower CPM ads)
slow upload speeds for competing channels
etc.
If you are batting for the home team, such advantages are great. But they blow for everyone else in the ecosystem.
Those sorts of issues don't just appear in a few isolated incidents, but appear over and over again.
Affiliate links should be clearly labeled as such. When they are not clearly labeled & go through tracking redirects they are sneaky redirects in Google's remote rater guidelines. On YouTube the affiliate links to Amazon & iTunes are not labeled as such & add an extra layer of tracking redirects to the sequence.
Let Me See Your Backlinks!
Yesterday someone sent me an email about their reinclusion request being denied because someone else scraped their eZineArticles article & syndicated it to another 3rd party site.
They didn't create that link and yet they are somehow supposed to get a spammer (maybe one from another continent!) to remove it. In many cases spammers won't respond to anything other than cash, but if you do offer cash to get the job done then that spammer might keep adding more and more links over time, turning their mark into an easy source of subscription revenues.
What is Wrong With This Picture?
The above scenario is ridiculous.
If you look at *any* site closely enough there will be something wrong with it.
Just by the virtue of existing & ranking you will pick up dozens to hundreds of spammy links you don't even want, due to SERP scraper sites that are trying to rank on longtail keyword queries.
About 5 years ago I had a page get filtered out because it gained about 500 scraper links in a month. No matter what I did that page would not rank until it was rewrote with a fresh page title. When you could change things & have the algorithms re-evaluate them automatically there was at least a decent opportunity to get around such issues.
Now that there is a manual review process holding you responsible for the actions of third party webmasters the market is a bit more grim.
But at least a bunch of link removal services are cropping up to profit from Google's errant logic. ;)
Engineers Ad Networks Love Quality Websites Big Brands
A bigger company can always shut a site down, split off into sections, & so on. Plus if you are a bigger company you are more likely to enjoy the benefit of the doubt.
But if you are a low margin small business who has seen declining revenue AND have to jump through further hoops (rather than focusing on running your business) at some point it is easier to give up than to keep on fighting.
Eventually a lot of the displacement trends that are hitting the organic search market will hit the paid search market & Google will make many of the enterprise AdWords management tools obsolete via a combination of various free scripts & data obfuscation.
At that point in time some of the paid search folks will look like the guy to the right, but nobody will care, as those same people reminded us that this is just how business works. :D
Google appears to have a culture that condones shamelessly violating consumer privacy. How else can you explain a company that bypasses Apple's iPhone privacy settings in a reported attempt to strengthen advertising revenues?
It is hard to believe that Dave Packard or Andy Grove would ever tell a group of entrepreneurs that he did "every horrible thing in the book to just get revenues right away," or brag to trade publications that his company used behavioral psychologists to design "compulsion loops" into products to keep customers engaged. But Mark Pincus, the founder of Internet gaming giant Zynga, has done just that.
When corporate leaders pursue wealth in the winner-take-all Internet environment, companies dance on the edge of acceptable behavior. If they don't take it to the limit, a competitor will. That competitor will become the dominant supplier -- one monopoly will replace another. And when you engage in these activities you get a different set of Valley values: the values of customer exploitation.
The layout of the result looks something like this
Or if you put it in Google's browser analysis tool, it looks something like this
And with that move, if you are in ecommerce & you don't rank #1 you are essentially invisible to most searchers.
As John Andrews highlighted on Twitter: "Notice Google tells us "paid relationships improve quality" and then penalizes for paid links?"
As always, it is more profitable to follow Google's biz dev team than Google's public relations pablum.
In some cases Google might include 3 or 4 different types of monetization in a search result. In the below search result Google includes:
AdWords ads
Google Offers
Hotel Comparison ads
Hotel Price ads
And those are *in addition* to featuring promotional links to Google Maps & Google+ in the search results. Further, some of these vertical results consist exclusively of paid inclusion & then have yet another layer of PPC ads over the top.
As SEOs we focus a lot of energy on "how do I rank 1 spot higher" but when the organic results are displaced and appear below the fold why bother? The issue of the incredibly shrinking organic result set is something that can't be over-emphasized. For many SEOs the trend will absolutely be career ending.
AdWords, product listing ads, brand navigation, product search, local, etc. A result like this has a single organic listing above the fold & if Google decides to rank their local one spot higher then that turns to zero.
If you look at the new TLD announcement Google applied for .MBA & .PhD (as well as many names around entertainment, family & software). Thus it is safe to say that education will eventually be added to local, video, media, shopping & travel as verticals where Google is displacing the organic results with links to more of their fraternal listings. About the only big categories this will leave unscathed will be real estate, employment & healthcare. However those first 2 are still in contraction during our ongoing depression & Google blew a lot of their health credibility by pushing those illegal ads for steroids from a person posing as a Mexican drug lord.
In addition to these fixed vertical that cover the most profitable areas of search, Google is also building a "vertical search on the fly" styled service with their knowledge graph. Their knowledge graph extracts data from 3rd party websites & then can be used to funnel traffic and revenue to Google's various vertical services. To make it seem legit, Google will often start by sending some of the traffic onto 3rd party sites, but the end destination is no different than product search. While it is a "beta" product it is free to justify an inferior product being showcased front & center, but after Google gets enough buy in they monetize.
There is a non-subtle difference between Google's approach and Microsoft's approach to building a search ecosystem.
Let's compare that behavior against Yahoo! or Bing.
Yahoo! has long been considered out of the search game, yet when they want to have a competitive advantage they do things like license photos from Getty. They use the content with permission on agreed terms.
Google's approach is more along the lines of "scrape it now & figure out legal later." And after a long enough period has passed they will add monetization & mix it into the core of their offering, like they recently did with books:
This launch transitions the billions of pages of scanned books to a unified serving and scoring infrastructure with web search. This is an efficiency, comprehensiveness and quality change that provides significant savings in CPU usage while improving the quality of search results.
Eric Enge interviewed Stefan Weitz about the new Bing interface. As part of that interview, Stefan described Bing's editorial philosophy on building a search ecosystem
We partner with 3rd party services instead of trying to build or acquire them. There are probably something like a million apps out there today.
I talk to probably two dozen start-ups every week that are doing different cool things on the web. To think that we are ever going to be able to actually beat them, or out-execute them (when they are talking about 12 guys with half a million angel funding building some really interesting apps), it is just not likely.
Off the start forays into new categories might provide some value to publishers in order to get buy in, but eventually the "first hit free" stuff shifts to paid & Google continues to displace publishers across more and more of the ecosystem, using content scraped from said publishers.
Funding Scraping
When Google or Apple drive cars around the country or fly military-grade planes over cities to create 3D maps of cities they are creating databases & adding new information. Outside of collecting private data (like wifi payload data) there is little to complain about with that. They are adding value to the system.
However, at the same time, Google not only scrapes themselves, but they are a revenue engine that drives a lot of third party scraping. And they design penalties in a way that allows those who scrape penalized sites to outrank them. With batch penalty updates some folks can chain redirects, expired domains & so on to keep exploiting the combination of copyright violations & Google penalties to make a mint. Google also had a long history of funding Traffic Equalizer sites, sites like Mahalo that would take a copy of a search result & auto-generate a page on it, newspaper sites that would hang auto-generated stub preview articles on subdomains, & sites like eHow which integrate humans into the process.
While many sites are still penalized from the first version of Panda, downstream referrals to eHow.com from Google in the US were up over 9% last month. They know "how to create SEO content."
Recently a start up that launched a couple years ago decided to take their thousands of subdomains of scraped databases & partner with authoritative websites to syndicate that content around the web. Some of those get double listings & for some search queries there is the same page (with a different masthead logo) 5 different times. Those sites don't get hit by duplicate content filters or algorithms like Panda because they have enough domain authority that they get a free pass. Including AdSense in the set up probably makes it more palatable to Google as well.
If you have scale you can even auto-generate a bunch of "editorial" questions off the database.
More data = more pages = more questions and comparisons = more pages = SEO alchemy (especially if you don't have to worry about Panda).
The parent scraper site includes links back to itself on every syndicated page, which to some degree makes it a glorified PageRank funnel. WPMU.org got smoked for syndicating out a sponsored theme on one of his own sites, but the above industrial-scale set up is somehow reasonable because it was launched by a person who sold their first start up to Google (and will likely sell this start up to Google too). The site also includes undisclosed affiliate links & hands out "awards" badges to the best casual encounter sex dating sites, which then get syndicated around the web & get it many inbound links from "high quality" porn sites.
I won't name the site here for obvious reasons, but they are not doing the above in a cloak of darkness that one has to look hard to find & do deep research to patch together. For some search results they are half or more of the search result set & they even put out press releases when they add new syndication partners, linking to numerous new automated subdomains or sections within sites related to various categories.
When the search results look like that, if you do original in-depth reviews that are expensive there is zero incentive structure to leaving your content and ratings open to Google and these sort of scraper/syndicaters.
There is always a new spin on the mash up low end content with high trust websites and try to feed it into Google. So long as Google biases their algorithms toward big brands & looks the other way when they exploit the ecosystem that trend will not end.
The Independent Publishers Group, a principal distributor of about 500 small publishers, recently angered Amazon by refusing to accept the company’s peremptory demand for deeper discounts. Amazon promptly yanked nearly 5,000 digital titles. Small-press publishers were beside themselves. Bryce Milligan of Wings Press, based in Texas, spoke for most when, in a blistering broadside, he lambasted Amazon, complaining that its actions caused his sales to drop by 40 percent.
However, even when companies are brutal in some aspects they do amazing things in other areas, so one has to weigh the good with the bad.
At any point Google can fold one vertical into another or extend out a new model. The Android Marketplace feeds into Google Play, Google local feeds into Google+, Google search force feeds just about everything else & even free offerings on sites like YouTube will eventually become pay to play stores.
Where Google lacks marketshare & forced bundling isn't enough to compete they can buy the #2 or #3 player in the market & try to propel it to #1 using all those other forms of bundling.
Part of what made search competitive against other platforms was its openness & neutrality. But if the search results are Wal-Mart over and over again (or the same scraped info 5 times in a row, or a collection of internal listings) then the system becomes more closed off & the perception of choice becomes an illusion. John Andrews wrote a couple greatTweets expressing the shift in search:
"Google SEO is no longer worth the effort for those who are not writers, artists, speakers, trainers, or promoters. What happened to Search?"
"If you want to see what Google will look like after it locks up, look at Apple. ipad users are already "managed" very tightly."
When companies try to expand the depth of their platform with more features it is a double edged sword. At some point they capture more value than they create and are no longer worth the effort. When they get to that stage it becomes a race to the bottom with scrapers trying to outscrape one another. Then in turn the company that created the ecosystem problem uses the pollution they rewarded to further justify closing off the system, guaranteeing only more of the same. Those who actually add value move on looking for greener pastures.
A label or an interest is a vector for ad targeting. There is no need to worry about de-anonymizing data for ad targeting when it is all in-network and you monitor what someone does, control which messages they see, & track which ones they respond to. Tell someone something often enough and they may believe it is true.
The Contempt Large Companies Have for their Customers
There is a sameness to customer service from a lot of big companies. They spend loads & loads to track you and market to you, but then disappear the moment things go wrong, as they are forbidden to care.
Perhaps the only thing worse that AOL's customer support is the unmoderated comments on the YouTube page.
Denise Griffin, the person in charge of Google’s small customer-support team, asked Page for a larger staff. Instead, he told her that the whole idea of customer support was ridiculous. Rather than assuming the unscalable task of answering users one by one, Page said, Google should enable users to answer one another’s questions.
Even their official blog posts announcing that they are accepting customer feedback for your applications go unmoderated.
This sort of contempt exists at essentially all large companies.
Everything seems on the up & up, but that "private listing" was maybe for a counterfeit product.
If it isn't a counterfeit & you get too good of a price you are threatened with a lawsuit, and the branded network falls behind a "oh we are just a marketplace and can't be bothered to give a crap about our customers" public relations angle.
The Retina MacBook is the least repairable laptop we’ve ever taken apart: unlike the previous model, the display is fused to the glass—meaning replacing the LCD requires buying an expensive display assembly. The RAM is now soldered to the logic board—making future memory upgrades impossible. And the battery is glued to the case—requiring customers to mail their laptop to Apple every so often for a $200 replacement. The design may well be comprised of “highly recyclable aluminum and glass”—but my friends in the electronics recycling industry tell me they have no way of recycling aluminum that has glass glued to it like Apple did with both this machine and the recent iPad. The design pattern has serious consequences not only for consumers and the environment, but also for the tech industry as a whole.
...
Every time we buy a locked down product containing a non-replaceable battery with a finite cycle count, we’re voicing our opinion on how long our things should last. But is it an informed decision? When you buy something, how often do you really step back and ask how long it should last? If we want long-lasting products that retain their value, we have to support products that do so.
One last bit of absurdity on the YouTube front. Google recently threatened to sue a site designed to convert YouTube videos into MP3s.
How does Google's "computers deserve free speech rights" & shagging 3rd party content to fill out their own vertical search services compare against their approach when someone uses YouTube content in a way Google does not desire?
There are AdWords ads promoting free unlimited MP3 downloading & song burning bundled with shady adware.
Google's AdSense for domains funds boatloads of cybersquatting. While Google threatened to sue this particular site, they could have just took the domain due to it cybersquatting on the YouTube trademark. The fact that they chose to turn this into a press event rather than simply fix the issue shows that this is more for posturing.
Further aligned with the above point, while Google singled out a specific MP3 conversion site, there are other sites designed around doing the same exact thing which are PREMIUM ADSENSE PARTNERS, with the body of the page looking like this:
How Small Companies Are Taxed With Uncertainty
When Google decided to move away from direct marketing to brand advertising things that are often associated with size, scale & brand recognition became relevancy signals.
how much to invest in marketing, where to invest it, how to balance the need for short term cashflow with the required reinvestments to build real (or fake) brand signals
how long does the market have left before Google enters the niche and destroys the opportunity that organic SEO once represented
should you run 1 website, or many to hedge risks? and how many is optimal?
how big should your site be?
if one of your sites gets penalized, should you try to fix it up, should you start over with a new site, or should you consider SEO to be a pointless goal?
Google mentions that they want people to do what is best for the user & not worry about Google, but that advice is a recipe for pain
If you do not run a large & authoritative website there are so many landmines to trip over with the increasing complexity of SEO. And any of Google's "helpful" webmaster messages can suspend a webmaster in fear, leading them to an eventual bankruptcy.
Small companies need to do all sorts of canonicalization hoops & prune content and such to hope to avoid algorithms like Panda. Then Google changed their host crowding preferences to let some large sites get up to 8 listings in a single search result page for their LACK OF effort. Those larger sites can then partner with glorified scraper sites that syndicate databases feeding on domain authority with no risk of Panda.
Due to how Google penalizes smaller sites, those that rewrite their content will outrank them when they get hit. These horrible trends are so obvious that even non-SEOs like Tim Carter (who was a Google golden boy for years) highlights how the tables have tilted away from what is most relevant to what pays Google the most.
For a long time Yahoo! provided great link data, but most other search engines were more reserved with sharing link data for competing sites. What were some of the driving forces behind Bing opening up on this front?
Bing values the power of strong partnerships as one way to spur innovation and deliver compelling experiences for our users. For any partnership to be effective, remaining as transparent as possible is critical, including those we forge with agency and publisher partners. Sharing link information was something very clearly asked for by tool users, so after doing the internal work to see if we could provide the information, it was an easy decision to build this tool when the answer came back positive. You wanted it, we had it and could share it. Done.
As a search engine your web index is much much larger than most SEO tools. On Twitter Rand mentioned that the index size of Bing's new Link Explorer was fairly comparable to Open Site Explorer. Is the link data offered in the tool a select slice of the index? Were you trying to highlight the highest quality link sources for each site?
We see the entire index, or at least "can" see the entire index and link ecosystem. We’re limited to the actual number we can show at any given time, however.
Currently it appears as though the tool lists link source URLs & page titles. Will the tool also add anchor text listings at some point?
On the list – sometimes we run into data sourcing issues, so when we hit those walls, it takes us longer to add features. Bing WMT pulls data from all the sources available within Bing Search, and sometimes those have limits imposed for other reasons. In those cases, we must abide by those rules or seek to influence changes to increase our own access/capacities. A search engine is a complex thing it turns out… J
There are filters for "anchor text" and "additional query." What are the differences between these filters?
Anchor Text is pretty clear to most SEOs. "Additional Query" allows you to look for, as an example, a page with "N" text appearing on it. So text not just as "anchor text", but simply appearing on the page.
Currently if I search for "car" I believe it will match pages that have something like "carson" on it. In the future will there be a way to search for an exact word without extra characters?
I’m going to split this answer. Users can enable “Strict” filtering to only see “cars” data by selecting the “Strict” box. To your point, however, this is what some of our tools are Beta. We will continually refine them as time goes on, adding features folks find useful.
Will you guys also offer TLD-based filters at some point?
First time anyone's mentioned it, so I’ll add this to our list for consideration.
A few years ago my wife was at a PPC seminar where a Bing representative stated that the keyword search data provided in the tools matched your internal data. Is this still the case?
Bing Advertising is completely separate from Webmaster Tools. I’m not sure if that rep was meaning data within the adCenter tools matches data or what. Bing WMT does import CPC data to showcase alongside keywords which sent traffic to your site. That data matches as we pull direct from adCenter. The data we show through our tools comes direct from Bing Search, so that’s a match if this is what you’re referring to.
Bing's Webmaster tools offers an API with keyword research & link data. Bing's Ad Intelligence is easily one of my 3 favorite SEO tools. Will Bing eventually offer a similar SEO-oriented plugin for Excel?
No plans on the roadmap for an Excel plugin.
At SMX Derrick Connell suggested that there was a relevancy perception gap perhaps due to branding. What are some of the features people should try or things they should search for that really highlight where Bing is much stronger than competing services?
Without doubt people should be logging in and using the Facebook integration when searching. This feature is tremendously helpful when you’re researching something, for example, as you can reach out directly to friends for input during your research process. While searching, keep your eyes open for the caret that indicates there is more data about a specific result. Hovering over that activates the “snapshot” showing the richer experience we have for that result. Businesses need to make sure they focus on social and managing it properly. It’s not going away and those who lag will find themselves facing stiff, new competition from those getting social right. Businesses also need to get moving adopting rich snippets on their sites. This data helps us provide the deeper experiences the new consumer interface is capable of in some cases.
You have wrote a couple books & done a significant amount of offline marketing. One big trend that has been highlighted for years and years is everything moving online, but as search advances do you see offline marketing as becoming an important point of differentiation for many online plays?
In a way yes. In fact, with the simplification of SEO via tools like our own and many others, more and more businesses can get things done to a level on their own. SEO will eventually become a common marketing tactic, and when that hits, we’re right back to a more traditional view of marketing: where all tactics are brought to bear to sell a product or service. Think of this…email marketing is still one of the single best converting forms of marketing in existence. Yet so many businesses focus on SEO (drive new traffic!) instead of email (work with current, proven shoppers!).
In fact, neither alone is the "best" strategy for most online businesses. It’s a blend of everything. Social happens either with you or without you. You can influence it, and by participating, the signals the engines see change. We can see those changes and it helps us understand if a searcher might or might not have a good experience with you. That can influence (when combined with a ton of other factors, obviously) how we rank you. Everything is connected today. Complex? Sure, but back in the day marketers faced similar complexity with their own programs. Just a new "complex" for us today. More in the mix to manage.
What is the best part about being an SEO who also works for a search engine?
On Wednesday, June 6th at 10AM PST, I was part of the team that brought a new level of tools forward, resetting expectations around what Webmaster Tools should deliver to users. Easily one of the proudest moments of my life was that release. While I’m an SEO and I work for the engine, the PM and Lead Engineer on the WMT product are also SEOs. ;) To say Bing is investing in building the partnership with SEOs is no mere boast. Great tools like this happen because the people building them live the life of the user.
What is the hardest part about being an SEO who also works for a search engine?
Still so few people around me that speak this language. The main difficulty is in trying to understand the sheer scope of search. Because everything you thought you knew as an SEO take son an entirely different dimension when you’re inside the engine. Imagine taking every SEO conversation and viewing it through a prism. So many more things to consider.
And, finally, nothing against Matt here, but why are dogs so much better than cats?
1 – they listen to you and execute commands like a soldier
2 – generally, they don’t crap in your house
3 – you can have a genuine conversation with a dog
4 – one of my dogs drives
5 – when was the last time your cat fetched anything for you?
6 – your dog might look at you funny, but won’t hiss at you
7 – guard cat? Hardly… you’d be better off with peacocks in the yard.
8 – dogs make great alarm clocks
9 – even YOU know you look strange walking your cat on a leash…
10 – dogs inspire you to be a better person
Brett Tabke is one of the most well known names in the SEO vertical, playing a crucial roll in building up Webmaster World & the Pubcon conference. We recently interviewed him about some of the latest changes in search & where online publishing is heading.
One of the more famous pieces of content on WMW (or really the whole of the SEO industry) over the years has been Successful Site in 12 Months with Google Alone: 26 steps to 15k a day. That predates my entry into the SEO niche & yet seems surprisingly relevant over a decade later. Are you surprised how well that has held up given how much search has changed? If you were to write it today, what would you add or change?
The surprise was how well received it was at the time. I wrote that piece in about an hour start-to-finish. It has held up well because it is a article about content production first and SEO second. If I had to rewrite it, I would drop the references to specific timings and numbers. Those numbers have been washed away with progress over the years.
At a recent SEO conference a few months back someone asked who was a big brand or worked for big brands & something like 80%+ of hands in the audience went up. When I got into search it was sort of the other way around, where most attendees were small publishers & affiliates. Originally when I got into search I felt that some conferences were a bit more corporate & Pubcon had far more "in the know" independent affiliate types who test lots of things out & such. How has Pubcon done such a good job of staying relevant throughout the shift in landscape of the industry?
Part of it is our 10 year game plan. I didn't realize I had a 10 year game plan in 2001, but I did. That plan was to add just "one thing more" to each conference. One thing more awesome that the last. We have done that for 10 years running, and each time we do, the conference grows a little bit in that direction. We make sure that as the audience grows in their knowledge, then we do too.
The other part of it, is our roots. We have been committed to sticking with the successful 'independent' website owners. By making sure we support them, we know that we have tapped into people who are experts in their particular field. We are not a 'vertical niche' conference. We are 8 to 10 'vertical niche' conferences combined. I always felt the niche conferences were great (blogging, affiliates, webhosting, domaining), but people have needs that cross over from discipline to discipline. By stretching across those genres, we attract the corporates that need to reach them, and keep the experts that are "in the know".
A lot of people who speak on authority about search & the SEO niche claim that the shift to big business is natural, legitimate & the way it should be. Do you agree with that, or feel that is mostly self-serving advice?
I don't believe SEO has become "big business". I do believe that SEO is no longer a question of SEO, but rather general marketing. It is a tricky and potentially inflammatory topic. I've come to realize, that you can't go on gut instinct alone in this business. We have been in-the-trenches with SEO since 95. Our whole world outlook has been through SEO. To stick our heads in the sand and not see the winds of change would be both foolish and detrimental to our bottom lines.
Yes, SEO has changed remarkably over the years. We started by just trying to get listed in the directory or search engine at all. Then we went on to on-the-page optimization. Google came along and turned it into "all links all the time" era. Finally today, we are into an obtuse era, where the the genres of optimization are so diverse from maps, to local, to authorship, to what friends you have on social networks. It has left alot of SEO's wondering where they should focus their energies at.
I've talked with many website owners that are starting to wonder if Google has become a losing value proposition for them. They are getting more traffic now from other engines and social sites. They wonder if Google actually wants to send them traffic. That the value of their website to Google, is nothing more than the data it can provide to train the machine for other SERP's and not theirs.
Facebook has so far failed to disrupt search and BingHoo has failed to disrupt Google. Do you see anything on the horizon that will change the search landscape?
I disagree. I think $100 billion IPO disproves that theory. Facebook just did an IPO for 100 x Gross income. Wow. Lets think about that for awhile. If you or I go to sell our businesses, we are going to be lucky to get 3x to 4x in this climate and Facebook just took on investors that were tagged to 100x their yearly gross!? Fantasy land. That is the same fantasy that a handful of guys at Instagram built into $1bilion dollar sale.
Facebook has totally disrupted Google and BingHoo. We just need to adjust our thinking. People know how to find what they want on the web. The great days of random web exploration are over. The grand days of social interaction are here.
Look at all those changes that Google has done the last 4 years: Google local push, Google Instant, Google Plus Network, and even partially Android. Those changes are a result of social media - of Facebook and Twitter. Those changes were not part of the 'grand plan' - they were short term and reactionary.
In SEO, in many cases those who lie are often heralded as being "white hat" while being granted greater access, whereas those who have knowledge and openly share it are often branded as being "black hat." Does this trend at some point reverse course? How many people can be labeled as black hat before the label lacks meaning?
As long as people focus on their own sites, and work to promoting them, there are no hats - only marketing. What you do on the privacy of your own site, is your business. As long as no laws are broken, there are no hats.
What is the most trite & least correct SEO tip that is shared publicly relentlessly?
Great question. About 2 years ago I heard a well known SEO suggest that you should remove all outbound links from your site in order to 'sculpt' page rank on your site. This spring, I heard that same SEO tell people they need to link out to make their links and site "look natural". The 'sculpt pagerank SILO'ing articles were based out of my work on Theme Pyramid from 2001. While the outbound linking was directly from previously mentioned 26 Steps article. I took alot of heat in "26 Steps to 15k a Day" for recommending that people link out to other sites. The theory then was that clearly SE's were going to look at outbound links to determine a sites theme. There are alot of metrics that outbound linking can be used as a quality (or lack their of) signal for search algos.
I think I saw you mention something about getting links that drive traffic. Why are links that drive traffic more valuable than those which do not?
Google is using every signal it can. They have:
Google Analytics
Google Adsense
Google Toolbar
Google Chrome browser.
They can track traffic over most of the web. They know what sites people spend time on and what links get clicked most. Links that drive traffic tend to drive higher SERP rankings.
If enough independent SEOs get driven out of the ecosystem, won't that at some point have a knock on effect on the wages of in-house SEOs at larger companies?
I don't think so. It is rare to run into "single task" SEO's at all but the largest corporations. Even the big media sites with well known SEO's require those people to work in other marketing areas as well. The knowledge of the general SEO today is vastly further along that we were just a few years ago.
The financial fall out from 2008 was that advertising and marketing agency budgets were cut to the bone in late '08. As those agencies were jettisoned, in house marketing people were given the task of SEO. In 08, we saw our InHouse panel attendance go from a few dozen to a few hundred as corporations sent their people to be trained. It was a breath taking switch. Those corporations learned the value of a good SEO fairly quickly in 2009. That trend has continued today with InHouse divisions paying more than ever before for quality SEO's.
As the table keeps getting tilted, at some point do smaller independent players decide to opt out of search en mass, sort of like you did with the robots.txt blog? Does more and more featured content eventually end up behind paywalls as ad-based models are seen as less reliable among algorithmic uncertainties?
Well, paywalls are also finding where their wall is at. There has to be some huge value add to having a pay wall. I think we have seen the failures of many pay wall sites as well as some huge successes. The big problem with pay walls, is that it is hard to keep it behind the wall.
I do think we are seeing people playing with more revenue opportunities than ever before. Ads, affiliates, and links are all being explored as viable alternatives. There is also a second (or is this the third?) wave of ecom sites being built. The big change this time is the reliance on conversion. People are getting pretty savvy about building sites that convert.
A lot of the biggest algorithmic holes & flaws that have been created over the years were created by attempts to over-compensate for other issues. Do you see brand (or brand-type) signals & usage signals dominating search for years to come?
Yes, yes I do. There is no other way around it. People want the authority of brand. Google can only give them so much before they wander back to big brands as the authority in various spaces.
Is authority bias / brand bias / etc. a crutch until enough vertical search technology has been deployed to where the regular organic results are largely irrelevant and below the fold for any query with a sniff of commercial intent? Are we moving from open ecosystems to closed ones? If so, is it a trend that reverses at some point?
I do think we are transitioning from a 100% search based traffic economy to a greater percentage being driven by other traffic sources. Social media will continue to grab more and more of the traffic pie.
The new tablet and phone traffic economies are more suited for social than search. Search is a proactive, brain-fully-engaged activity. Social is much more of a click and 'let it come to me' experience. Search is about products, and research, and finding the right bits of information. Social is about chilling with the tablet in the easy chair while watching reruns on your Netflix TV and surfing your buddies Facebook pictures. There is also the coming wave of new intelligent TV's that will lean them selves to more of a passive social experience than an active search experience. Those new technologies are disruptive to the old search economy.
If companies like Amazon.com create large distributed ad networks won't that eventually push Google to back off their authority bias?
I don't think anyone but Facebook could dislodge Google's ad dominance.
One can find free consumer reviews by the boatload on authority sites & vertical marketplaces, but are in-depth expert reviews being driven out of the marketplace due to shifts in algorithmic preferences?
I agree to a degree. The one push back on that, is Quora. Other than that, the Bizarre Voice driven review model continues to grow without any major competitors.
What is something people will see at Pubcon that they can't find elsewhere?
Expertise. 55% of Pubcon attendees consider themselves to be experts in their field, while another 30% consider themselves to be advanced. That leads us to ask speakers to bring their Grade A1+ game to their presentations and they do. I feel our level of content is the highest in the entire educational conference space. That, and we keep the conference reasonably priced for the average techhead.
----
Thanks Brett. This year's Pubcon is once again in Las Vegas, Nevada & takes place the week of October 15 through 18.
Is outing & writing polarizing drivel hate baiting or a service to the community?
It is all a matter of perspective, isn't it?
Some people would like to claim that it is one thing when they do it & something else when somebody else does it.
Unfortunately for those who want to have their cake & eat it too, consistency matters.
Even these guys know that.
If you brand those who fall outside the guidelines or get hit by updates as scammers to be avoided, then when your company gets caught working an angle & "scamming" (based on your own past sermons) your own judgement gets cast against yourself.
Is that fair?
In a word: yes.
Any belief system that is imposed onto others, but unacceptable when imposed upon the person who states it, isn't a belief system at all. It's duplicitous hackery at best - possibly much worse.
If your own company doesn't follow your own advice, then what does that say about your value systems? How many people have had their potential held back by listening to your misinformation & making the unfortunate mistake of trusting you? What does that sort of behavior do to the reputation of the industry? Now everyone else is suspect because you pitched bogus pablum at newbies.
To speak publicly about the pitfalls of doing "blackhat" techniques and then turn around and be caught red-handed for the same just gives credibility to the naysayers claiming our industry is filled with slime balls.
If you want to be a polarizing asshat, then don't be surprised when you eat your own cooking. To expect anything less is an open expression of ignorance of the field of inbound marketing marketing.
A bigger company can always 
