Years ago Google introduced rel=nofollow, claiming it as a cure-all for comment spam. Once in place, it was quickly promoted as a tool to use on any paid link. Google scared webmasters about selling links so much that many webmasters simply became afraid to link out to anyone for fear of falling out of favor with Google.
Our affiliate program on this site stopped passing link juice after a fellow SEO blogger outed it quite publicly. Other affiliate programs continue to pass PageRank. Highlighting Google's double standards invites more scrutiny and more selective arbitrary enforcement. Whereas promoting Google products earns free links. ;)
No Disclosure Required: WOOT!
Reading the news today I found out that VigLink bought out DrivingTraffic. Both are networks to help publishers monetize their outbound links. The claim about VigLink is the one of no-effort money:
"Quite simply, if you're a Web publisher who hasn't recognized the value of your outbound traffic, you are leaving money on the table," said Raymond Lyle, CEO and Co-Founder of Driving Revenue. "Dozens of our publishers make six figure incomes for a one-time investment of one minute of work. Who isn't interested in that?"
The page loads fast. And your site looks exactly the same. Even your links look and behave the same way. The only difference is that now when your visitors buy products or services you'll earn a commission. ... Once you have set up viglink you can sign in to view reports about your site. You can see how much money you are making every day and compare that with last week. You can see which merchants are the most profitable, and make decisions on who to link to in the future.
So basically Viglink is suggesting controlling who you link to based on whatever makes you the most money, and not providing any disclosure of the financial relationship.
AKA: paid links.
Here is where it really gets screwed up: Google is an investor in VigLink.
Selectively allowing some links to pass link juice while arbitrarily blocking others indeed controls the shape of the web graph. It gives anyone who works with Google a strong competitive advantage in the organic search results over those who are not using Google endorsed technology.
As Google reached the limits of returns in direct marketing they started pushing the value of branding (because, hey, if you can chalk it up to latent branding value there is no cap on your max bid). Surprisingly, they even got many big brands to buy their own brands AND buy sitelinks on the AdWords ads. Some went so far as providing case studies for how much of their own brand traffic they were now willing to pay for, which they previously got free. :D
Sure that can make sense for seasonal promotions, but you could do the same thing by having subdomains and sister websites. Dell.com can be the main site, Dell.net (or deals.dell.com) can be the deals & promotions website, and Dell.org can be the good karma charity site. No paying someone else for brand you already spent to build. Beautiful. But I digress...
Companies with a high page rank are in a strong position to move into new markets. By “pointing” to this new information from their existing sites they can pass on some of their existing search engine aura, guaranteeing them more prominence.
Google’s Mr Singhal calls this the problem of “brand recognition”: where companies whose standing is based on their success in one area use this to “venture out into another class of information which they may not be as rich at”. Google uses human raters to assess the quality of individual sites in order to counter this effect, he adds.
Those are all irrelevant details, just beyond Google's omniscient view. :D
The other thing which is absurd, is that if you listen to Google's SEO tips, they will tell you to dominate a small niche then expand. Quoting Matt Cutts: "In general, I’ve found that starting with a small niche and building your way up is great practice."
And now brand extension is somehow a big deal worth another layer of arbitrary manual inspection and intervention?
If sites which expand in scope deserve more scrutiny then why is there so much scrape & mash flotsam in the search results? What makes remixed chunks of content better than the original source? A premium AdSense feed? Brand?
This image might need updated in the years to come, but it does a great job laying out how Google works when you type a query into their search engine. Search is so easy to do that it is hard to appreciate how complex it is unless you take a look under the hood. Which is exactly what this graphic does :D
Click the image to get the full sized beefy image :D
A side benefit of this graphic is that it should help prospective clients realize how complex SEO & PPC campaigns can be. So if anyone is trying to be an el cheapo with their budget you can use this to remind them how complex search is, and thus how time consuming and expensive a proper search marketing campaign is.
In the past when I claimed that the Google Maps insertion in organic search results wasn't more organic search but rather a Google promotion, I was met with skepticism by some, who argued that Google Maps was just another flavor of organic search and visitors would still be able to go to the end ranked website.
If you search for something on Google and click on one of the end URLs you can still visit them, but Google made one step in the opposite direction today. If you click on the map now the Google Maps section lists a bunch of places on the maps, rather than giving you the URLs. You then have to click onto one of the locations to see it on the map and open a pop up area which contains information including the URL. More clicks to do the same thing.
How long until Google replaces the URL listings in the search results with links to locations on the Google Maps or links to Google Places pages? It is the next obvious step in this transition.
Originally Google wanted to send you to the destination as quickly as possible, hoping that in doing so they would encourage you to come back again. This year Google's strategy has changed to one that wants you to stay for a while. There is no better example of that shift than Youtube Leanback:
Jamie Davidson, a YouTube product manager, says that the 15 minutes of daily viewing by a user typically involves six videos, with the conclusion of each presenting "a decision point, and every decision point is an opportunity to leave. We’re looking at how to push users into passive-consumption mode, a lean-back experience."
Generally I have not been a huge fan of registering all your websites with Google (profiling risks, etc.), but they keep using the carrot nicely to lead me astray. :D ... So much so that I want to find a Googler and give them a hug.
Google recently decided to share some more data in their webmaster tools. And for many webmasters the data is enough to make it worth registering (at least 1 website)!
AOL Click Data
When speaking of keyword search volume beakdown data people have typically shared information from the leaked AOL search data.
The big problem with that data is it is in aggregate. It is a nice free tool, and a good starting point, but it is fuzzy.
In general, for navigational searches people click the top result more often than they would on an informational search.
In general, for informational searches people tend to click throughout the full set of search results at a more even distribution than they would for navigational or transactional searches.
The only solid recently-shared publicly data on those breakdowns is from Dogpile [PDF], a meta search engine. But given how polluted meta search services tend to be (with ads mixed in their search results) those numbers were quite a bit off from what one might expect. And once more, they are aggregate numbers.
Pretty solid looking estimates can get pretty rough pretty fast. ;)
The Value of Data
If there is one critical piece of marketing worth learning above all others it is that context is important.
My suggestions as to what works, another person's opinions or advice on what you should do, and empirical truth collected by a marketer who likes to use numbers to prove his point ... well all 3 data sets fall flat on their face when compared against the data and insights and interactions that come from running your own business. As teachers and marketers we try to share tips to guide people toward success, but your data is one of the most valuable things you own.
A Hack to Collect Search Volume Data & Estimated CTR Data
In their Excel plug-in Microsoft shares the same search data they use internally, but its not certain that when they integrate the Yahoo! Search deal that Microsoft will keep sharing as much data as they do now.
Google offers numerous keywordresearchtools, but getting them to agree with each other can be quite a challenge.
There have been some hacks to collect organic search clickthrough rate data on Google. One of the more popular strategies was to run an AdWords ad for the exact match version of a keyword and bid low onto the first page of results. Keep the ad running for a while and then run an AdWords impression share report. With that data in hand you can estimate how many actual searches there were, and then compare your organic search clicks against that to get an effective clickthrough rate.
The New Solution
Given search personalization and localization and the ever-changing result sets with all the test Google runs, even the above can be rough. So what is a webmaster to do?
Well Google upgraded the data they share inside their webmaster tools, which includes (on a per keyword level)
keyword clickthrough rank
clickthrough rate at various ranking positions
URL that was clicked onto
Trophy Keywords vs Brand Keywords
Even if your site is rather well known going after some of the big keywords can be a bit self-defeating in terms of the value delivered. Imagine ranking #6 or #7 for SEO. Wouldn't that send a lot of search traffic? Nope.
When you back away the ego searches, the rank checkers, etc. it turns out that there isn't a ton of search volume to be had ranking on page 1 of Google for SEO.
With only a 2% CTR the core keyword SEO is driving less than 1/2 the traffic driven by our 2 most common brand search keywords. Our brand might not seem like it is getting lots of traffic with only a few thousand searches a month, but when you have a > 70% CTR that can still add up to a lot of traffic. More importantly, that is the kind of traffic which is more likely to buy from you than someone searching for a broad discovery or curiosity type of keyword.
The lessons for SEOs in that data?
Core keywords & raw mechanical SEO are both quite frequently heavily over-rated in terms of value.
Rather than sweating trying to rank well for the hardest keywords first focus on more niche keywords that are easy to rank for.
Search is becoming the default navigational tool for the web. People go to Google and then type in "yahoo." If you don't have a branded keyword as one of your top keywords that might indicate long-term risk to your business. If a competitor can clone most of what you are doing and then bake in a viral component you are toast.
Going After the Wrong Brand Keywords
Arbitraging 3rd party brands is an easy way to build up distribution quickly. This is why there are 4,982 Britney Spears fan blogs (well 2 people are actually fans, but the other 4,980 are marketers).
But if you want to pull in traffic you have to go after a keyword that is an extension of the brand. Ranking for "eBay" probably won't send you much traffic (as their clickthrough rate on their first result is probably even higher than the 70% I had above). Though if you have tips on how to buy or sell on eBay those kinds of keywords might pull in a much higher clickthrough rate for you.
To confirm the above I grabbed data for a couple SEO tool brands we rank well for. A number 3 ranking (behind a double listing) and virtually no traffic!
Different keyword, same result
Link building is still a bit of a discovery keyword, but I think it is perhaps a bit later staged than just the acronym "SEO." Here the click volume distribution is much flatter / less consolidated than it was on the above brand-oriented examples.
If when Google lowers your rank you still pull in a fairly high CTR that might be a signal to them that your site should rank a bit higher.
When the internal Google remote quality rater guidelines leaked online there was a core quote inside it that defined the essence of spam:
Final Notes on Spam
When trying to decide if a page is Spam, it is helpful to ask yourself this question: if I remove the scraped (copied) content, the ads, and the links to other pages, is there anything of value left? if the answer is no, the page is probably Spam.
With the above quote in mind please review the typical Mahalo page
Adding a bit more context, the following 25 minute video from 2008 starts off with Matt Cutts talking about how he penalized a website for using deceptive marketing. Later into the video (~ 21 minutes in) the topic of search results within search results and then Mahalo come up.
Here is a transcription of relevant bits...
Matt Cutts: Would a user be annoyed if they land on this page, right. Because if users get annoyed, if users complain, then that is when we start to take action.
And so it is definitely the case where we have seen search results where a search engine didn't robots.txt something out, or somebody takes a cookie cutter affiliate feed, they just warm it up and slap it out, there is no value add, there is no original content there and they say search results or some comparison shopping sites don't put a lot of work into making it a useful site. They don't add value.
Though we mainly wanted to get on record and say that hey we are willing to take these out, because we try to document everything as much as we can, because if we came and said oh removed some stuff but it wasn't in our guidelines to do that then that would be sub-optimal.
So there are 2 parts to Google's guidelines. There are technical guidelines and quality guidelines. The quality guidelines are things where if you put hidden text we'll consider that spam and we can remove your page. The technical guidelines are more like just suggestions.
So we said don't have search results in search results. And if we find those then we may end up pruning those out.
We just want to make sure that searchers get good search results and that they don't just say oh well I clicked on this and I am supposed to find the answer, and now I have to click somewhere else and I am lost, and I didn't find what I wanted. Now I am angry and I am going to complain to Google.
Danny Sulivan: "Mahalo is nothing but search results. I mean that is explicitly what he says he is doing. I will let you qualify it, but if you ask him what it is still to this day he will say its a search engine. And then all the SEOs go 'well if it is a search engine, shouldn't you be blocking all your search results from Google' and his response is 'yeah well IF we ever see them do anything then we might do it'."
Matt Cutts: It's kinda interesting because I think Jason...he is a smart guy. He's a savvy guy, and he threaded the needle where whenever he talked to some people he called it a search service or search engine, and whenever he talked to other people he would say oh it is more of a content play.
And in my opinion, I talked to him, and so I said what software do you use to power your search engine? And he said we use Twika or MediaWiki. You know, wiki software, not C++ not Perl not Python. And at that point it really does move more into a content play. And so it is closer to an About.com than to a Powerset or a Microsoft or Yahoo! Search.
And if you think about it he has even moved more recently to say 'you know, you need to have this much content on the page.' So I think various people have stated how skilled he is at baiting people, but I don't think anybody is going to make a strong claim that it is pure search or that even he seems to be moving away from ok we are nothing but a search engine and moving more toward we have got a lot of people who are paid editors to add a lot of value.
One quick thing to note about the above video was how the site mentioned off the start got penalized for lying for links, and yet Jason Calacanis apologized for getting a reporter fired after lying about having early access to the iPad. Further notice how Matt considered that the first person was lying and deserved to be penalized for it, whereas when he spoke of Jason he used the words savvy, smart, and the line threaded the needle. To the layperson, what is the difference between being a savvy person threading the needle and a habitual liar?
Further lets look at some other surrounding facts in 2010, shall we?
How does Jason stating "Mahalo sold $250k+ in Amazon product in 2009 without trying" square with Matt Cutts saying "somebody takes a cookie cutter affiliate feed, they just warm it up and slap it out, there is no value add, there is no original content there" ... Does the phrase without trying sound like value add to you? Doesn't to me.
Matt stated "and if you think about it he has even moved more recently to say 'you know, you need to have this much content on the page,'" but in reality, that was a response to when I highlighted how Mahalo was scraping content. Jason dismissed the incident as an "experimental" page that they would nofollow. Years later, of course, it turned out he was (once again) lying and still doing the same thing, only with far greater scale. Jason once again made Matt Cutts look bad for trusting him.
Matt stated "I don't think anybody is going to make a strong claim that it is pure search" ... and no, its not pure search. If anything it is IMPURE search, where they use 3rd party content *without permission* and put most of it below the fold, while the Google AdSense ads are displayed front and center.
If you want to opt out of Mahalo scraping your content you can't because he scrapes it from 3rd party sites and provides NO WAY for you to opt out of him displaying scraped content from your site as content on his page).
Jason offers an "embed this" option for their content, so you can embed their "content" on your site. But if you use that code the content is in an iframe so it doesn't harm them on the duplicate content front AND the code gives Jason multiple direct clean backlinks. Whereas when Jason automatically embeds millions of scraped listings of your content he puts it right in the page as content on his page AND slaps nofollow on the link. If you use his content he gets credit...when he uses your content you get a lump of coal. NICE!
And, if you were giving Jason the benefit of the doubt, and thought the above was accidental, check out how when he scrapes the content in that all external links have a nofollow added, but any internal link *does not*
Matt stated "[Jason is] moving more toward we have got a lot of people who are paid editors to add a lot of value" ... and, in reality, Jason used the recession as an excuse to can the in house editorial team and outsource that to freelancers (which are paid FAR LESS than the amounts he hypes publicly). Given that many of the pages that have original content on them only have 2 sentences surrounded by large swaths of scraped content, I am not sure there is an attempt to "add a lot of value." Do you find this page on Shake and Bake meth to be a high quality editorial page?
What is EVEN MORE OUTRAGEOUS when they claim to have some editorial control over the content is that not only do they wrap outbound links which they are scraping content from in nofollow, but they publish articles on topics like 13 YEAR OLD RAPE. Either they have no editorial, or some of the editorial is done by pedophiles.
Here Jason is creating a new auto-generated page about me! And if I want to opt out of being scraped I CAN'T. What other source automatically scrapes content, republishes it wrapped in ads and calls it fair use, and then does not allow you to opt out? What is worse in the below example, is that on that page Jason stole the meta description from my site and used it as his page's meta description (without my permission, and without a way for me to opt out of it).
So basically Matt...until you do something, Jason is going to keep spamming the crap out of Google. Each day you ignore him another entreprenuer will follow suit trying to build another company that scrapes off the backs of original content creators. Should Google be paying people to *borrow* 3rd party content without permission (and with no option of opting out)?
I think Jason has pressed his luck and made Matt look naive and stupid. Matt Cutts has got to be pissed. But unfortunately for Matt, Mahalo is too powerful for him to do anything about it. In that spirit, David Naylor recently linked to this page on Twitter.
What is the moral of the story for Jason Calacanas & other SEOs?
If you have venture capital and have media access and lie to the media for years it is fine. If you are branded as an SEO and you are caught lying once then no soup for you.
If you are going to steal third party content and use it as content on your site and try to claim it is fair use make sure you provide a way of opting out (doing otherwise is at best classless, but likely illegal as well).
If you have venture capital and are good at public relations then Google's quality guidelines simply do not apply to you. Follow Jason's lead as long as Google permits mass autogenerated spam wrapped in AdSense to rank well in their search results.
The Google Webmaster Guidelines are an arbitrary device used to oppress the small and weak, but do not apply to large Google ad partners.
Don't waste any of your time reporting search spam or link buying. The above FLAGRANT massive violation of Google's guidelines was reported on SearchEngineLand, and yet the issue continues without remedy - showing what a waste of time it is to highlight such issues to Google.
The Google Buzz team has had quite a week. Their new product quickly lived up to its name, though mostly for the wrong reasons, generating buzz about its own privacy issues. Calling the original Google Buzz privacy settings lax would be a gross understatement. It created a storm of complaints, best put in perspective by Harriet Jacobs in her F*ck You, Google piece.
In short, when you logged into your Gmail account Google simply took all of your frequent contacts and mashed them up into an active social network without much input from people they were connecting. If you exchanged a lot of emails with your editor and your under-cover sources from the same Gmail account, now they were connected through your public profile if you didn't happen to catch the Buzz opt-out checkbox. Or what about using the same Gmail account for emailing your husband and your boyfriend? Well now they're introduced - you're welcome.
Yes, sounds like a pretty naïve and reckless way to implement a major feature but Google protested that they just wanted to help and meant no evil. After all, their CEO Eric Schmidt had an interesting take on expectations for privacy online: "If you have something that you don't want anyone to know, maybe you shouldn't be doing it in the first place". That was said nary two months before Google Buzz launched - I guess people like Harriet Jacobs and her abusive ex-husband just didn't listen.
Oops, Our Bad: Thanks For All the Users...
Since the launch, Google has done an amazingly quick about-face and pledged to do better. The latest set of changes make signing up for Buzz a tangibly more transparent experience, probably what it should have been at launch time. The press has mostly applauded their quick response and patted Google on the back for their responsiveness and keen focus on Gmail user experience.
Buzz already rivals Twitter for sheer network size
Those are some pretty impressive numbers for any online launch, but to achieve this in under three days is just unheard of. Actually, there are businesses that do generate this level of interests from their prospects in that short of a time-frame and Gmail deals with them on a daily basis: spammers.
The '9 Million-Post' Question
The question is did Google simply make a "mistake" and not consider these fairly serious privacy issues, or did the massive amount of spam Gmail churns through each day actually demonstrate effectiveness of a new business model?
The former is hard to believe when you consider the army of privacy lawyers Google has and their job to review privacy considerations in revenue-generating AdSense programs. This is especially critical in Gmail, where you are shown ads based on emails you exchange. Gmail achieves this by reading through all your email and matching you up with advertisers interested in addressing your daily struggles. After the initial outrage over this concept a few years ago most users have resigned to trust Google that they have their best interests in mind.
Your Trust, Google's Toilet Paper
Google Buzz violates this trust in a serious way. In light of Google's experience in this field, it is hard not to take Google's mea culpa with a huge dose of skepticism. After all, if Google had made Buzz an opt-in service - something that users had to enable rather than be tricked into joining - they would be just another social network trying to compete with Facebook and Twitter.
Leveraging millions of Gmail users was a shortcut simply too tempting to avoid. The fact that Google decided to revise Google Buzz activation process over the weekend is simply a red herring: they only needed a few days to convert some of the hundred million plus Gmail users into millions of Buzz users, and become the de-facto Twitter competitor over a single long weekend.
Google "fixing" this privacy snafu a few days later is equivalent to spammers adding an "Unsubscribe" link to an email that's already done its damage.
The strong impression from the last few days is that Gmail users were a pawn in a very cynical game: Google trying desperately to become a player in the social networking space, after the Orkut launch and their acquisition of a handful of other companies in this space failed to produce results.
We're Not Evil
This is a tough act to pull off when your motto is Don't Be Evil. It's been said that eventually Google's shareholders will push it to make product moves and decisions that end up hurting its brand in a quest for monetization. It will be interesting to see if Google comes out of this with their motto intact.
Ben Edelman: "Although I had asked that the Google Toolbar be "disable[d]," and although the Google Toolbar disappeared from view, my network monitor revealed that Google Toolbar continued to transmit my browsing to its toolbarqueries.google.com server"
[update: Matt Cutts contacted me and mentioned that this was due to the Vevo launch which occurred after that page was cached. Over time that means such pages like the one mentioned below should be purged from the Google search index.]
Google claims they try to be pretty fair with publishers and publishing business models. They are fine with indexing preview versions of a page and just showing a user that, you can make the full article free, you can make the first x clicks free.
OR you can put it all behind a paywall and not get any search exposure.
UNLESS you are Youtube.
In which case you can put whatever you want behind a subscribe wall, still have that registration-required/paywall content fully indexed in Google, and then force users to sign in to view the content.
On the cache copy of pages people still can view the pre-roll ads, but not the content :D
Search Google for "poker face", observe all the Youtube data in the search results, click the top Youtube listing, and watch them send you to a login page so they can better track you and target ads against you.
Many publishers that are having trouble figuring out search (from a business model perspective) would have no problem making a ton of money from search if they got the good ole home cooking treatment that Youtube currently enjoys (universal search promotion + cloaking forcing registration).
And this is where Google being rumored to acquire other content properties (like Yelp) becomes scary for users and publishers and advertisers alike.
To understand our position in more detail, it helps to start with the assertion that open systems win. This is counter-intuitive to the traditionally trained MBA who is taught to generate a sustainable competitive advantage by creating a closed system, making it popular, then milking it through the product life cycle. The conventional wisdom goes that companies should lock in customers to lock out competitors. There are different tactical approaches — razor companies make the razor cheap and the blades expensive, while the old IBM made the mainframes expensive and the software ... expensive too. Either way, a well-managed closed system can deliver plenty of profits. They can also deliver well-designed products in the short run — the iPod and iPhone being the obvious examples — but eventually innovation in a closed system tends towards being incremental at best (is a four blade razor really that much better than a three blade one?) because the whole point is to preserve the status quo. Complacency is the hallmark of any closed system. If you don't have to work that hard to keep your customers, you won't.
Open systems are just the opposite. They are competitive and far more dynamic. In an open system, a competitive advantage doesn't derive from locking in customers, but rather from understanding the fast-moving system better than anyone else and using that knowledge to generate better, more innovative products. The successful company in an open system is both a fast innovator and a thought leader; the brand value of thought leadership attracts customers and then fast innovation keeps them. This isn't easy — far from it — but fast companies have nothing to fear, and when they are successful they can generate great shareholder value.
Open systems have the potential to spawn industries. They harness the intellect of the general population and spur businesses to compete, innovate, and win based on the merits of their products and not just the brilliance of their business tactics. The race to map the human genome is one example.
But as soon as Google gets a market dominant position, you can bet on them locking it down to enhance ad revenues. The secret search relevancy algorithms, AdWords ad quality score, using AdWords rebates to push Google Checkout, always-on search personalization (even when logged out), mystery meat payout rates to AdSense publishing partners, universal search algorithms that allow them to arbitrarily promote their own websites, YouTube cloaking, etc etc etc
It looks like they jumped the gun on Yelp. Google was already integrating Yelp reviews in their AdWords ads before the acquisition was finalized.
What does it mean for the rest of us?
I am not sure.
It depends on if Google believes in what they say or what they do. They can't believe both.
But even amongst the traditional listings there are lots of variations in how they are displayed.
Here is a regular result
with a second indented result
sometimes the second indented result can show inline sitelinks
traditional single listing with 2 indented results under it (and then sometimes a non-indented 4th listing)
traditional single listing with 3 or 4 inline sitelinks
sometimes that has a second indented listing as well
traditional single listing with 8 sublinks below it (and this often has the second intented results below it too...though in such cases it is not indented)
traditional single listing with 4 links under it (often with dates near them) for forums & some blogs
And the above does not take into account handling of domains vs subdomains (or http vs https), using breadcrumbs in the search results, insertion of additional data (like a picture of a video or reviews from micro-format data), other helpful links (like a link to the businesses location on Google Maps), and all the types of vertical search data (videos, music, movies, local, news, real time, shopping/product results) being pulled into the regular search results. And then you can layer personalization and localization on top of the search results as well as yet another layer of change. And don't forget about any user based metrics or temporal metrics Google might be able to add with caffeine.
When you think of all the different ways data can be modified and displayed it makes information architecture a bit challenging, especially for new projects when you don't know where you will be at in a year, how much the market will change in that next year, and how many additional formats Google will create between now and then.