Some sites have seen pretty drastic drops in Google search traffic recently, related to indexing issues. Google maintains that it is a glitch:

Just to be clear, the issues from this thread, which I have reviewed in detail, are not due to changes in our policies or changes in our algorithms; they is due to a technical issue on our side that will be visibly resolved as soon as possible (it may take up to a few days to be visible for all sites though). You do not need to change anything on your side and we will continue to crawl and index your content (perhaps not as quickly at the moment, but we hope that will be resolved for all sites soon). I would not recommend changing anything significantly at this moment (unless you spot obvious problems on your side), as these may result in other issues once this problem is resolved on our side.

An example of one site's search traffic that was butchered by this glitch, see the below images. Note that in the before, Google traffic is ~ 10x what Yahoo! or Bing drive, and after the bug the traffic is ~ even.

Not that long ago I saw another site with over 500 unique linking domains which simply disappeared from the index for a few days, then came right back 3 days later. Google's push to become faster and more comprehensive has perhaps made them less stable, as digging into social media highlights a lot of false signals & often promotes a copy over the original. Add in any sort of indexing issues and things get really ugly really fast.

Now this may just be a glitch, but as Tedster points out, many such "glitches" often precede or coincide with major index updates. Ever since I have been in the SEO field I think Google has done a major algorithmic change just before the holidays every year except last year.

I think the reasons they do it are likely 3 or 4 fold

• they want to make SEO unpredictable & unreliable (which ultimately means less resources are spent on SEO & the results are overall less manipulated)
• they want to force businesses (who just stocked up on inventory) to enter the AdWords game in a big way
• by making changes to the core relevancy algorithms (and having the market discuss those) they can slide in more self promotion via their vertical search services without it drawing much anti-trust scrutiny
• the holidays are when conversion rates are the highest, so if they want to make changes to seek additional yield it is the best time to do it, and the holidays give them an excuse to offer specials or beta tests of various sorts

As an SEO with clients, the unpredictability is a bad thing, because it makes it harder to manage expectations. Sharp drops in rankings from Google "glitches" erode customer trust in the SEO provider. Sometimes Google will admit to major issues happening, and other times they won't until well *after* the fact. Being proven right after the fact still doesn't take back 100% of the uncertainty unleashed into the marketplace weeks later.

Even if half your clients double their business while 1/3 lose half their search traffic, as an SEO business you typically don't generally get to capture much of the additional upside...whereas you certainly capture the complaints from those who just fell behind. Ultimately this is one of the reasons why I think being a diversified web publisher is better than being an SEO consultant... if something takes off & something else drops then you can just pour additional resources into whatever is taking well and capture the lift from those changes.

If you haven't been tracking rankings now would be a great time to get on it. It is worth tracking a variety of keywords (at various levels of competition) daily while there is major flux going on, because that gives you another lens through which to view the relevancy algorithms, and where they might be headed.

One of the best ways to track Google's strategies is through visualizing & analyzing their acquisitions. Which is what the following image helps you do. Click on it for the full enlarged version :)

via Scores

Rel=nofollow to the Rescue

Years ago Google introduced rel=nofollow, claiming it as a cure-all for comment spam. Once in place, it was quickly promoted as a tool to use on any paid link. Google scared webmasters about selling links so much that many webmasters simply became afraid to link out to anyone for fear of falling out of favor with Google.

If You Don't Disclose You Are a Spammer

As the pool of links dried up due to the launch & spread of nofollow any ad network which used direct links was supposed to adopt nofollow or feel the wrath. Just ask Pay Per Post what Google can do to you if you sell links (to/through someone other than Google).

Google demanded that any form of paid link contain a machine readable and user readable disclaimer that it is paid for (even though in Google's marketing they highlight how some of their users are unaware the search results contain paid links).

What it came down to is if there was a monetary relationship associated with a link and you didn't disclose it then you were operating outside of Google's guidelines and may be considered a "spammer."

Selective Search Guideline Enforcement

I am one of many who have highlighted how by-and-large Google was responsible for killing off the link graph through their paranoia about "paid links," and their willingness to fund companies operating outside their guidelines that syndicate Google ads.

Our affiliate program on this site stopped passing link juice after a fellow SEO blogger outed it quite publicly. Other affiliate programs continue to pass PageRank. Highlighting Google's double standards invites more scrutiny and more selective arbitrary enforcement. Whereas promoting Google products earns free links. ;)

No Disclosure Required: WOOT!

Reading the news today I found out that VigLink bought out DrivingTraffic. Both are networks to help publishers monetize their outbound links. The claim about VigLink is the one of no-effort money:

"Quite simply, if you're a Web publisher who hasn't recognized the value of your outbound traffic, you are leaving money on the table," said Raymond Lyle, CEO and Co-Founder of Driving Revenue. "Dozens of our publishers make six figure incomes for a one-time investment of one minute of work. Who isn't interested in that?"

Note that "1 minute of work" doesn't really leave much time for disclosure. As stated in this video, the intent is to not offer any:

The page loads fast. And your site looks exactly the same. Even your links look and behave the same way. The only difference is that now when your visitors buy products or services you'll earn a commission. ... Once you have set up viglink you can sign in to view reports about your site. You can see how much money you are making every day and compare that with last week. You can see which merchants are the most profitable, and make decisions on who to link to in the future.

So basically Viglink is suggesting controlling who you link to based on whatever makes you the most money, and not providing any disclosure of the financial relationship.

AKA: paid links.

Presumably these VigLinks will still pass PageRank, but the affiliate stuff will be layered on top of the regular links using JavaScript. Pay affiliates using VigLink a bit of a higher percent for the exposure and you bought a ton of valuable inbound links for pennies on the Dollar.

Here is where it really gets screwed up: Google is an investor in VigLink.

Selectively allowing some links to pass link juice while arbitrarily blocking others indeed controls the shape of the web graph. It gives anyone who works with Google a strong competitive advantage in the organic search results over those who are not using Google endorsed technology.

Google also has a patent on automatically adding inline links inside content. Since they can't legally do it without permission of the webmaster, one presumes any implementation would be as part of a distributed ad network.

Makes you wonder about how evil undisclosed paid links are, no?

This image might need updated in the years to come, but it does a great job laying out how Google works when you type a query into their search engine. Search is so easy to do that it is hard to appreciate how complex it is unless you take a look under the hood. Which is exactly what this graphic does :D

Click the image to get the full sized beefy image :D

A side benefit of this graphic is that it should help prospective clients realize how complex SEO & PPC campaigns can be. So if anyone is trying to be an el cheapo with their budget you can use this to remind them how complex search is, and thus how time consuming and expensive a proper search marketing campaign is.

Generally I have not been a huge fan of registering all your websites with Google (profiling risks, etc.), but they keep using the carrot nicely to lead me astray. :D ... So much so that I want to find a Googler and give them a hug.

Google recently decided to share some more data in their webmaster tools. And for many webmasters the data is enough to make it worth registering (at least 1 website)!

AOL Click Data

When speaking of keyword search volume beakdown data people have typically shared information from the leaked AOL search data.

The big problem with that data is it is in aggregate. It is a nice free tool, and a good starting point, but it is fuzzy.

Types of Searches

There are 3 well known search classifications: navigational, transactional, and informational. Each type of query has a different traffic breakdown profile.

• In general, for navigational searches people click the top result more often than they would on an informational search.
• In general, for informational searches people tend to click throughout the full set of search results at a more even distribution than they would for navigational or transactional searches.
• The only solid recently-shared publicly data on those breakdowns is from Dogpile [PDF], a meta search engine. But given how polluted meta search services tend to be (with ads mixed in their search results) those numbers were quite a bit off from what one might expect. And once more, they are aggregate numbers.

Other Stuff in the Search Results

Further, anecdotal evidence suggests that the appearance of vertical / universal results within the search results set can impact search click distribution. Google shows maps on 1 in 13 search results, and they have many other verticals they are pushing - video, updates, news, product search, etc. And then there are AdWords ads - which many searchers confuse as being the organic search results.

Pretty solid looking estimates can get pretty rough pretty fast. ;)

The Value of Data

If there is one critical piece of marketing worth learning above all others it is that context is important.

My suggestions as to what works, another person's opinions or advice on what you should do, and empirical truth collected by a marketer who likes to use numbers to prove his point ... well all 3 data sets fall flat on their face when compared against the data and insights and interactions that come from running your own business. As teachers and marketers we try to share tips to guide people toward success, but your data is one of the most valuable things you own.

A Hack to Collect Search Volume Data & Estimated CTR Data

In their Excel plug-in Microsoft shares the same search data they use internally, but its not certain that when they integrate the Yahoo! Search deal that Microsoft will keep sharing as much data as they do now.

Google offers numerous keyword research tools, but getting them to agree with each other can be quite a challenge.

There have been some hacks to collect organic search clickthrough rate data on Google. One of the more popular strategies was to run an AdWords ad for the exact match version of a keyword and bid low onto the first page of results. Keep the ad running for a while and then run an AdWords impression share report. With that data in hand you can estimate how many actual searches there were, and then compare your organic search clicks against that to get an effective clickthrough rate.

The New Solution

Given search personalization and localization and the ever-changing result sets with all the test Google runs, even the above can be rough. So what is a webmaster to do?

Well Google upgraded the data they share inside their webmaster tools, which includes (on a per keyword level)

• keyword clickthrough rank
• clickthrough rate at various ranking positions
• URL that was clicked onto

Trophy Keywords vs Brand Keywords

Even if your site is rather well known going after some of the big keywords can be a bit self-defeating in terms of the value delivered. Imagine ranking #6 or #7 for SEO. Wouldn't that send a lot of search traffic? Nope.

When you back away the ego searches, the rank checkers, etc. it turns out that there isn't a ton of search volume to be had ranking on page 1 of Google for SEO.

With only a 2% CTR the core keyword SEO is driving less than 1/2 the traffic driven by our 2 most common brand search keywords. Our brand might not seem like it is getting lots of traffic with only a few thousand searches a month, but when you have a > 70% CTR that can still add up to a lot of traffic. More importantly, that is the kind of traffic which is more likely to buy from you than someone searching for a broad discovery or curiosity type of keyword.

The lessons for SEOs in that data?

• Core keywords & raw mechanical SEO are both quite frequently heavily over-rated in terms of value.
• Rather than sweating trying to rank well for the hardest keywords first focus on more niche keywords that are easy to rank for.
• If you have little rank and little work to do then there is lots of time to focus on giving people reasons to talk about you and reference you.
• Work on building up brand & relationships. This not only gives your link profile more karma, but it sends you a steady stream of leads for if/when you fall out of favor a bit with the search engines.

Those who perceive you well will seek you out and buy from you. But it is much harder to sell to someone who sees you as just another choice amongst many results.

Search is becoming the default navigational tool for the web. People go to Google and then type in "yahoo." If you don't have a branded keyword as one of your top keywords that might indicate long-term risk to your business. If a competitor can clone most of what you are doing and then bake in a viral component you are toast.

Going After the Wrong Brand Keywords

Arbitraging 3rd party brands is an easy way to build up distribution quickly. This is why there are 4,982 Britney Spears fan blogs (well 2 people are actually fans, but the other 4,980 are marketers).

But if you want to pull in traffic you have to go after a keyword that is an extension of the brand. Ranking for "eBay" probably won't send you much traffic (as their clickthrough rate on their first result is probably even higher than the 70% I had above). Though if you have tips on how to buy or sell on eBay those kinds of keywords might pull in a much higher clickthrough rate for you.

To confirm the above I grabbed data for a couple SEO tool brands we rank well for. A number 3 ranking (behind a double listing) and virtually no traffic!

Different keyword, same result

Informational Keywords

Link building is still a bit of a discovery keyword, but I think it is perhaps a bit later staged than just the acronym "SEO." Here the click volume distribution is much flatter / less consolidated than it was on the above brand-oriented examples.

If when Google lowers your rank you still pull in a fairly high CTR that might be a signal to them that your site should rank a bit higher.

Enough Already!

Enough about our keywords, what does your keyword data tell you? How can you better integrate it to grow your business?