Google SEO Correlation Analysis

I have never been a huge fan of correlation analysis. The reason being is that how things behave in aggregate may not have anything to do with how they would behave in your market for your keywords on your website.

Harmful High Quality Links?

A fairly new website was ranked amazingly quickly on Google.com for a highly competitive keyword. It wasn't on the first page, but ranked about #20 for a keyword that is probably one of the 100 most profitable keywords online (presuming you could get to a #1 ranking above a billion Dollar corporation). The site did a promotion that was particularly well received by bloggers and a few bigger websites in the UK press and at first rankings improved everywhere. Then one day while looking at its rankings using rank checker I saw the site simply fell off the map. It was nowhere. I then jumped into web analytics and saw search traffic was up. What happened was Google took the site as being from the UK, so its rankings went to page 1 in the UK while the site disappeared from the global results. In aggregate we know that more links are better & links from high trusted domains are always worth getting. And yet in the above situation the site was set back by great links. Of course we can set the geographic market inside Google Webmaster Tools to the United States, but how long will it take Google to respond? How many other local signals will be fixed to pull the site out of the UK?

Over time those links will be a net positive for the site, but it still needs to develop more US signals. And beyond those sort of weird things (like links actually hurting your site) the algorithms can look for other signals to push into geotargeting. Things like Twitter mentions, where things are searched for, how language is used on your website, and perhaps even your site's audience composition may influence localization. What is worse about some of these other signals is that they may mirror media coverage. If you get coverage in The Guardian a lot of people from the UK will see it, and so you might get a lot of Tweets mentioning your website that are from the UK as well. In such a way, many of the signals can be self-reinforcing even when incorrect.

Measuring The Wrong Thing

Another area where correlation analysis falls short is when one page ranks based on the criteria earned by another. Such signal bleeding means that if you are looking at things in aggregate you are often analyzing data which is irrelevant.

Sampling Bias

Correlation analysis also has an issue of sampling bias. People tend to stick with defaults until they learn enough to change. Unfortunately most CMS tools are set up in sub-optimal ways. If you look at the top ranked results some of the sub-optimal set ups will be over-represented in the "what works" category simply because most websites are somewhat broken. The web is a fuzz test.

Of course the opposite of the above is also true: some of the best strategies remain hidden in plain sight simply due to sheer numbers of people doing x poorly.

Analyzing Data Pairs Rather Than Individual Signals

Another way signals have blurred is how Google uses page titles in the search results. That generally used to be just the page title. But more recently they started mixing in

  • using an on-page heading rather than the page title (when they feel the on-page heading is more relevant)
  • adding link anchor text into the title (in some cases)
  • adding the homepage page's title at the end of sub-pages (when sub-page page titles are short)

As Google adds more signals & changes how they account signals it makes analyzing what they are doing much harder. You not only need to understand how the signals are used, but how they interact in pairs or groups. When Google uses the H1 heading on a page to display in the search results are they still putting a lot of weight on the page title? Does the weighting on the H1 change depending on if Google is displaying it or not?

Analysis is Still Valuable, but...

I am not saying that analysis is a waste of time, but rather that when you do it lots of do's and don'ts become far less concrete. The fact is that there are always edge cases that disprove any rule of thumb. Rather than looking for general rules one needs to balance things like:

  • risk vs reward
  • yield vs effort
  • focus vs diversity
  • investment vs opportunity cost

First Mover Advantage

Along the same lines, any given snapshot of search is nowhere near as interesting as understanding historical trends and big shifts. If you are one of the first people to notice something there is far more profit potential than being late to the party. Every easily discernible signal Google creates eventually gets priced close to (or sometimes above) true market value. Whereas if you are one of the first people to highlight a change you will often be called ignorant for doing so. :D

Consensus is the opposite of opportunity.

When you do correlation analysis you are finding out when the market has conformed to what Google trusts & desires. Exact match domains were not well ranked across a wide array of keywords until after Google started putting more weight on them & people realized it. But if there is significant weight on them today & their prices are sky high then knowing that they carry some weight might not be a real profit potential in your market. It might even be a distraction or a dead end. Imagine being the person who bets (literally) a million Dollars that Google will place weight on poker.org only to find out that Google changes their algorithmic approach & weighting, or makes a special exception just for your site (as they can & have done). That day would require some tequila.

As a marketing approach becomes more mainstream then not only do the cost rise, but so does the risk of change. As people complain about domain names (or any other signal or technique) it makes Google more likely to act to curb the trend and/or lower it's weighting & value. To see an extreme version of such, consider that the past year has seen lots of complaints about content farms. A beautiful quote:

Searching Google is now like asking a question in a crowded flea market of hungry, desperate, sleazy salesmen who all claim to have the answer to every question you ask.

And so Google promises action. Don't make Google look stupid!

History Holds the Key for Success

The only way to profitably predict the future is to accurately understand history.

  • "Our ignorance of history makes us libel to our own times. People have always been like this." - Gustave Flaubert
  • "History repeats itself, first as tragedy, second as farce." - Karl Marx
  • "We are the prisoners of history. Or are we?" - Robert Penn Warren, Segregation

Focus On Adding Value

Times are tough.

In times like these, clients tend to focus on the value proposition. "Throw it at the wall, see if it sticks" is not a phrase you hear a lot in recessions.

Instead, your customers will tend to have their eyes transfixed on your value proposition. "How does this spend make me better off?"

Whilst we may think search marketing services are essential, the spend on search services typically comes out of marketing budgets, and marketing budgets tend to be the first thing companies cut when things get tight.

So, they might need more convincing that usual.

If you weren't doing so already, it can be a good time to go over your proposals and pitch, and look to emphasize, and add to the value proposition you offer.

A few points to consider....

1. Address Genuine Needs

Address the need a client has, which may be different than the need they articulate.

This may seem obvious, but often people aren't quite sure exactly why they need search marketing, or they may have wrong ideas about it. Their genuine business need may be buried. You need to tease this out.

To do so, listen. Hard.

One common mistake people who are "fixers" - seos tend to be fixers - can make is that they'll go through the motions of listening, but really they're just waiting for an opportunity to launch into their solution.

A client will tell you a lot, and perhaps cover a lot of angles you hadn't thought of, if you let them talk long enough. They will like the fact you are interested in them and their problems, and it will make your eventual solution sound more considered and tailor-made.

Because it will be.

If you don't solve a genuine problem, your relationship is more likely to be a short one. Services that don't solve genuine business problems are more likely to get cut.

2.Go Beyond

Look for ways you can enhance your offering.

Look to solve genuine problems in closely related fields. For example, a client may lack a content strategy. They may want to publish content regularly, but haven't got around to doing so. You could enhance your offering by incorporating this work in your offer, reasoning that it dovetails nicely with your SEO strategy, thus killing two birds with one stone.

This can also get you on-going work, if pitched right, and may involve little more than hiring the services of a copywriter.

3. Establish Feedback Mechanisms

Feedback is important.

Not only does it give you added insight into what the client is thinking, it also offers you the opportunity to demonstrate your value proposition in action.

You said you would do X, you do X, then show them you've done X. This helps build trust.

Clients will often elaborate, if given the opportunity, which can give you more ideas on how to "Go Beyond", and how to "Address Their Real Needs".

4. Look At Jobs As Partnerships

If you've ever bought services, you know that selecting a service provider can be a pain. It is time consuming, and there is risk involved. A wrong choice can lead to opportunity cost, and having to repeat the process all over again.

No one wants that. People want partnerships with their suppliers. They want someone on their side.

Once you've landed a client, try to see them as a business partner. This is certainly how they will view you if they like you. They are unlikely to go back out to the market unless they are disastisfied, so try to make their business, your business.

Take the approach that you will boost your own business by building theirs.

5. Every Job Is An Opportunity To Build Hybrid Skills & Knowledge

Let's say you have a travel client.

Learn everything you can about the travel industry. Press the client for information. Research and understand the wider industry, not just the search marketing opportunity within that industry.

One of the golden things about being a consultant is that you get to look inside people's businesses. This information is valuable and difficult to obtain by other means, yet you're getting paid to learn it. You're learning about real business issues, who's-who, and the language of the industry.

You then become more valuable to any other travel-related client as you're now "a travel guy". You can pitch convincing to them, because you speak their language, understand their problems, and you've got industry history.

Starting A New SEO Business In 2011?

A new year brings new resolutions.

Some readers might be considering taking that giant leap from their boring day job into the wonderfest that is full-time SEO. Huge money! Party central! Hangin' at conferences with Matt Cutts! What could possibly go wrong?

Let's take a serious look at what your new life will look like.

It's Going To Hurt

SEO is a world of hurt.

When you start, you'll have little money. Your bills don't stop coming in. Google, rather uncooperatively, may not rank your sites for six months.

Maybe longer.

Perhaps you've already got a few sites ranking. You've got some steady adsense/affiliate money coming in, which is right about the time Update Oh-My-God happens.

A Google update, like a demented hurricane, trashes your site for no good reason. OK, maybe, maybe you had *some* links that were not, in the cold light of day, strictly-speaking, based 100% on merit. But hey, everyone else was doing it, right?

It will be no consolation that everyone else's sites will have been trashed, too. You will meet these people in SEO forums, gnashing their teeth as if the world has just come to end.

It has, of course.

There are few more heart-breaking moments than when Google sends an H-bomb crashing down on your dreams. Google say they do this to improve their "service", but mostly they do it "because it's fun".

Your SEO forum buddies will explain, sometimes using elaborate math, why everyone's rankings dropped. These explanations are bullshit and can be safely ignored. Well-intended they may be, but your buddies don't have a clue. Chances are they just read something in another forum, thought it sounded profound, so they repeated it.

The sad reality is few people are doing any real testing these days.

Even more annoying will be the person who claims his site hasn't been affected. He will lecture everyone else on how, in the latest update, Google is finally rewarding higher quality sites.

Don't worry. This sanctimonious fool will likely get his site trashed in the next update. It will then be his chance to gnash his teeth.

In SEO, everyone gets their turn eventually.

Right about this time, that autographed picture of Matt Cutts hanging on your wall will start to look sinister. You could swear the picture is pulsing red with the faint glow of hells-fire.

Feeling scared and alone, you take it down and hide it in the drawer.

Are You Serious?

Events, like those described above, are just life's way of testing to see if you're serious.

If you are serious, you climb back up on the horse, get back in that saddle, and go rope some steers. Or, if you're an SEO, not a cowboy, you start fixing your sites.

Alternatively, you could decide that the performance-based SEO lifestyle is way too difficult, and vow to become an SEO consultant instead. Being an SEO consultant really takes the pressure off. Mostly, you just talk about stuff. Repeat things you've heard in forums.

Firstly, gather together some cryptic sounding jargon - "latent semantic indexing" is always a crowd pleaser - and apply to talk at the SMXWebmasterWorldSearchEngineStrategies conference. Next, get your smiling, drunken self into a photo, with your arm around Matt Cutts. This implies you have an inside line at Google. Finally, knock together an SEO consultant web site to display it all to the world. Claim to be an "SEO Expert". Often.

One problem.

Being an SEO Expert is not a rare commodity. There are 22,345,947 SEO experts in India alone. And many work for less than your weekly beer bill. So unless you've got the sales skills of Tony Robbins, the solitary SEO consultant gig is a tough one.

You may decide to join an SEO agency. This is an easier gig, as you can focus 100% on SEO, surrounded by people who claim to know a lot more about SEO than they actually do. Many of your co-workers post regularly on forums.

You will soon enjoy the delights of heading off to a client site to tell a room full of hostile designers why their award winning flash site will have to be redesigned, from scratch, preferably using bare HTML.

Best of luck.

Following that lively exchange of views, you may wish to kiss the dark arts of SEO farewell, and move into the world of PPC.

PPC is a lot easier than SEO. Well, it is if you have a bank balance the size of Texas. If you don't have a lot of money, you'll spend all your time tweaking budgets, which, if you get them wrong, can end up costing you your credit limit. PPC is dangerous, but at least you can take that autographed photo of Matt Cutts back out of the drawer.

He cannot touch you now.

If you fail miserably at being an SEO and PPC consultant, don't despair. You can always take the easy way out.

Become a social media consultant.

Becoming A Social Media Consultant

The beauty of this gig is you don't need any technical chops at all.

Simply grab a book on public relations, rewrite it by dropping the word "Facebook", or "Twitter" in every second paragraph, and hit the speaking circuit. Rehash the same old stuff about "reach", "audience share", and "convergence" and mix it up with new terms like "re-tweet". If you're feeling confident, throw some Cluetrain Manifesto quotes in, like "Markets are conversations", and "Hyperlinks subvert hierarchy".

They love that stuff. No one knows what it means, but that simply validates your high fees.

The problem is the barrier to entry for becoming a social media consultant is set even lower than becoming an SEO consultant. That, and the fact everyone started calling "bs" on the whole thing last year.

My advice: don't quit your day job.

But I know you'll ignore me.

Have fun in 2011 :)

Google Gearing Up for Relevancy Changes

Over the past year or 2 there have been lots of changes with Google pushing vertical integration, but outside of localization and verticalization, core relevancy algorithms (especially in terms of spam fighting) haven't changed too much recently. There have been a few tricky bits, but when you consider how much more powerful Google has grown, their approach to core search hasn't been as adversarial as it was a few years back (outside of pushing more self promotion).

There has been some speculation as to why Google has toned down their manual intervention, including:

  • anti-trust concerns as Google steps up vertically driven self-promotion (and an endless well of funding for anyone with complaints, courtesy Microsoft)
  • a desire to create more automated solutions as the web scales up
  • spending significant resources fighting site hacking (the "bigger fish to fry" theory)

Matt Cutts recently made a blog post on the official Google blog, which highlighted that indeed #3 was a big issue:

As we’ve increased both our size and freshness in recent months, we’ve naturally indexed a lot of good content and some spam as well. To respond to that challenge, we recently launched a redesigned document-level classifier that makes it harder for spammy on-page content to rank highly. The new classifier is better at detecting spam on individual web pages, e.g., repeated spammy words—the sort of phrases you tend to see in junky, automated, self-promoting blog comments. We’ve also radically improved our ability to detect hacked sites, which were a major source of spam in 2010. And we’re evaluating multiple changes that should help drive spam levels even lower, including one change that primarily affects sites that copy others’ content and sites with low levels of original content.

It sounds like Google was mainly focused on fighting hacked sites and auto-generated & copied content. And now that hacked *GOVERNMENT* websites are available for purchase for a few hundred Dollars (and perhaps millions in personal risk when a government comes after you) it seems like Google's pushing toward fighting off site hacking was a smart move! Further, there are a wide array of start ups built around leveraging the "domain authority" bias in Google's algorithm, which certainly means that looking more at page by page metrics was a needed strategy to evolve relevancy. And with page-by-page metrics it will allow Google to filter out the cruddy parts of good sites without killing off the whole site.

As Google has tackled many of the hard core auto-generated spam issues it allows them to ramp up their focus on more vanilla spam. Due to a rash of complaints (typically from web publishers & SEO folks) content mills are now a front and center issue:

As “pure webspam” has decreased over time, attention has shifted instead to “content farms,” which are sites with shallow or low-quality content. In 2010, we launched two major algorithmic changes focused on low-quality sites. Nonetheless, we hear the feedback from the web loud and clear: people are asking for even stronger action on content farms and sites that consist primarily of spammy or low-quality content. We take pride in Google search and strive to make each and every search perfect. The fact is that we’re not perfect, and combined with users’ skyrocketing expectations of Google, these imperfections get magnified in perception.

Demand Media (DMD) is set to go public next week, and Richard Rosenblatt has a long history of timing market tops (see iMall or MySpace).

But what sort of sites are the content mills that Google is going to ramp up action on?

The tricky part with vanilla spam is the subjective nature of it. End users (particularly those who are not web publishers & online advertisers) might not complain much about sites like eHow because they are aesthetically pleasing & well formatted for easy consumption. The content might be at a low level, but maybe Google is willing to let a few of the bigger players slide. And there is a lot of poorly formatted expert content which end users would view worse than eHow, simply because it is not formatted for online consumption.

If you recall the Mayday update, Richard Rosenblatt said that increased their web traffic. And Google's October 22nd algorithm change last year saw many smaller websites careen into oblivion, only to re-appear on November 9th. That update did not particularly harm sites like eHow.

However, in a Hacker News thread about Matt's recent blog post he did state that they have taken action against Mahalo: "Google has taken action on Mahalo before and has removed plenty of pages from Mahalo that violated our guidelines in the past. Just because we tend not to discuss specific companies doesn't mean that we've given them any sort of free pass."

My guess is that sites that took a swan dive in the October 23rd timeframe might expect to fall off the cliff once more. Where subject search relevancy gets hard is that issues rise and fall like ocean waves crashing ashore. Issues that get fixed eventually create opportunities for other problems to fester. And after an issue has been fixed long enough it becomes a non-issue to the point of being a promoted best practice, at least for a while.

Anyone who sees opportunity as permanently disappearing from search is looking at a half-empty glass rather than one which sees opportunities that died reborn again and again.

That said, I view Matt's blog post as a bit of a warning shot. What types of sites do you think he is coming after? What types of sites do you see benefiting from such changes? Discuss. :)

How To Measure Bias In Google's Results

Here's an interesting study, conducted by Benjamin Edelman and Benjamin Lockwood, from the Harvard Business School. The study measures how much search engines, Google in particular, favor their own web services.

We find that each search engine favors its own services in that each search engine links to its own services more often than other search engines do so. But some search engines promote their own services significantly more than others.... we find that Google's algorithmic search results link to Google's own services more than three times as often as other search engines link to Google's services. For selected keywords, biased results advance search engines' interests at users' expense

People have debated this topic for a while, some saying the search engines can do what they like, others feel the search engines must be held to account.

However, the study brings up an important point. If Google claims to have algorithmic, "objective" search results, then it follows that Google should not favor their own companies properties, unless those properties achieve a top ranking based on their own merit.

Google can't have it both ways.

The problem, of course, is that Google could tweak the algorithm to favor whatever qualities its own properties display e.g. the PageRank of Google's own pages could be calculated - in truly cryptic and oblique fashion - as being of higher "worth". After all, there's no such thing as "objective" when it comes to editorial, which is the function of a search algorithm. There are merely points along a continuum of subjectivity.

But where it gets interesting is the study goes one step further. It tries to figure out what the user wanted when she searched. Did the user want to find a Google service at #1? And if not, then isn't Google doing the user a dis-service by placing a Google property at #1?

In principle, a search engine might feature its own services because its users prefer these links. For example, if users of Google's search service tend to click on algorithmic links to other Google services, whereas users of Yahoo search tend to click algorithmic links to Yahoo services, then each search engine's optimization systems might come to favor their respective affiliated services. We call this the "user preference" hypothesis, as distinguished from the "bias" theory set out above

They tested this theory using click-thru data. Regarldess of the search keyword, users almost always favor the #1 result - 72% of the time. So what if the user clicks further down, indicating that the first result is less relevant?

Gmail, the first result, receives 29% of users' clicks, while Yahoo mail, the second result, receives 54%. Across the keywords we looked at, the top-most result usually receives 5.5 times as many clicks as the second result, yet here Gmail obtains only 53% as many as Yahoo. Nor was "email" the only such term where we found Google favoring its own service; other terms, such as "mail", exhibit a similar inversion for individual days in our data set, though "email" is the only term for which the difference is large and stable across the entire period

There is a huge incentive for search engines, which increasingly crossing the line into publishing territory, to skewer the results towards their own properties. The traffic is valuable, and, whatismore, can be channeled away from competitors.

As Aaron pointed out a few months ago, if Google choose to enter a new vertical, such as travel or local, then you'd better watch out if you compete in those verticals. Regardless of how relevant you are to the search term, it's below-the-fold you'll likely be going.

So, yes, it may be Google's search engine, but they can't make claims about focusing on the user above all else, otherwise they'd return results the user wants, as opposed to possibly directing the user to Google properties due to other considerations. How can they claim "Democracy works", if they don't favour whatever site the link graph "votes" most relevant? And doesn't this come down slightly on the wrong side of "evil"?

So, What To Do?

If you feel Google can position their own sites where they like, then nothing.

Personally, I think any company can do what they like, until they reach a point where they become so influential, they can use their sheer size to reduce competition and choice. If we believe that free markets require healthy competition in order to thrive, then we should be wary of any entity that can reduce competition using anti-competitive behavior.

I'm not saying that is what Google is doing, but watch this space. Some European agencies are investing allegations of anti-trust violations.

The Commission will investigate whether Google has abused a dominant market position in online search by allegedly lowering the ranking of unpaid search results of competing services which are specialised in providing users with specific online content such as price comparisons (so-called vertical search services) and by according preferential placement to the results of its own vertical search services in order to shut out competing services

The fact Marissa Mayer said this:

[When] we roll[ed] out Google Finance, we did put the Google link first. It seems only fair right, we do all the work for the search page and all these other things, so we do put it first... That has actually been our policy, since then, because of Finance. So for Google Maps again, it’s the first link

....makes matters......interesting ;)

Secondly, if you're big enough, you could make a point of taking Google on. Check out Trip Advisors take on Google Places displaying Trip Advisors data in repackaged form, which could cause Google users to stay on Google, and not go to the Trip Advisor site:

Google is no longer able to stream in reviews from TripAdvisor to Places pages after the user review giant blocked it. TripAdvisor confirmed the move today in an email, stating that while it continues to evaluate recent changes to Google Places it believes the user does not benefit with the “experience of selecting the right hotel”. As a result, we have currently limited TripAdvisor content available on those pages,” an official says

But Google aren't really going to care much about you if you don't have some major clout.

Thirdly, stay out of any vertical Google is likely to want to own. It is likely that Google will be going after the big verticals, because a big company needs to score big on projects. Long tail stuff isn't going to make any difference to their bank balance, except in aggregate, so there will be millions of verticals in which you'll never face a direct threat.

This is also a timely reminder to build up your non-search traffic in case Google, or any other search engine, decides to change the game significantly in their favor. Encourage users to bookmark, develop your social media brand, build mailing lists, put some valuable content behind log-in/pay walls, and build membership sites. Relying on Google has always been a risky strategy, do diversify your traffic strategy where you can in 2011.

Google Approaches Its Breaking Point

Google's Take On SEOs

Google likes to make SEOs look like fools. Some are, but some are simply privy to less information. Or, in some cases, thrown under the bus by a new wave editorial policy in the gray area. Inconsistent enforcement is a major issue, but even if you go beyond that, the truth is most businesses have a range of revenue streams from pure as can be to entirely parasitic.

Google is Sleazier Than Microsoft

Recently we saw Matt Cutts play investigative reporter & literally create a story about how dirty some of Bing's affiliates are. Here is the litmus test though: when Microsoft became aware of it they immediately canned the shady distribution partner. Meanwhile, Google still funds toolbars that put AdSense ads in them AND to this day Google still funds google.bearshare.com, which *is* driven by the same kinds of home page changes that Matt found distasteful.

In Manufacturing Consent Noam Chomsky highlights that we should judge actions based on an equality of principals & that we are responsible primarily for our own actions. Yet Google complains about Microsoft. It took Microsoft less than a day to clean up their act, while Google still hasn't fixed issues that were highlighted publicly 6 years ago!

Many Subjective Warnings

Not only is Google trying to police their competitors, but recently they have offered warnings on all sorts of subjective issues, like...

Individually, each of those issues can be debated.

In our new site design our navigation is aggressively repetitive in some areas. The reason we did that was some people complained about not being able to effectively get around the site. To help make the navigation more intuitive and consistent we use drop downs and in some cases have 3 or 4 or even 5 links to the same location. Is that optimal from a search perspective? Probably not. But then again, search engines don't convert into paying customers. They are simply a conduit...a means to an end. When an engineer views a site they might not view it through the same lens as a customer would.

What is an unnatural link profile? Does it depend on who is building the links? We know that at an SEO conference when some of IAC's cross linking was highlighted Matt Cutts stated "those don't count" but didn't qualify it any further. Likewise when it was highlighted how Mahalo was link farming we were told that they deserved the benefit of the doubt. Since then the link farms have grown and mutated. I won't link at ask.inc.com/when-should-i-hire-a-company-for-lead-generation, but if I was told that the following is "natural" and "good to go" then I would have no problems building a few million links a week. Then again, I bet it would be "unnatural" if I did the same thing.

The part about treating Googlebot different from users is a bit perplexing. As technology has evolved this area has become quite blurry/murky.

  • Sometimes when clicking into big media sites that are 'first click free' I get kicked right to a registration page. In the past some iTunes pages would rank & force you into the iTunes software (though that may have recently changed).
  • Google ranks certain Youtube content in international markets, even where said content is unavailable.
  • Scroll cloaking has been around forever.
  • Tools like Google Website Optimizer can be used to alter user experience significantly.
  • There is an SEO start up which pushes search visitors to sites like CNN to a heavily ad wrapped & paginated version of the same content.
  • I accidentally screwed up using a rel=canonical on a page (cut the source code from a dynamic page and pasted it as the basis for a similar static page & forgot to remove the rel=canonical tag). Eventually I figured out what was wrong & fixed it, but both the correct and incorrect pages ranked for weeks at #1 and #2. And isn't the whole point of the rel=canonical tag to give the search engines a different type of header than an end user (telling the search engine that the content is elsewhere while telling the user nothing of the sort)?

But which of those is a problem? None of them? All of them? Does it depend on who you ask? Does it depend on perceived intent? And that is the tricky part because the same impact can be had with many different tools like 301 redirects, meta refreshes, javascript redirects, and rel=canonical. Should they penalize the technique, the intent, or the result? How do they imply intent?

Relevancy vs Market Manipulation

Spam is in the eye of the beholder, as is relevancy.

The thing is, Google is in a position to imply intent as they see fit. They are in a position to tilt the playing table as they see fit. They claim to be open and sometimes they are fighting the good fight, but businesses have a range of revenue streams from pure as can be to entirely parasitic.

The leaked internal Google documents about copyright and ads on trademarks certainly highlight that Google has no problem with a foot in each pond.

Syndication has long been a part of the media landscape, where portals chose what bits to mix in from where. But how much is fine & what should be done with duplicates? When does something go from 'legitimate syndication' to 'overt spam'? We see official Google blog posts which claim that AdSense ads are not allowed on unoriginal content, while Google has multiple large partners that wrap Google's search results in the AdSense feed and then serve it back to Google. Site categories which were described as 'shoot on sight' become viable enterprises when a person puts a web 2.0 design, venture capital & some public relations into the same basic business model. If Google is going to put out some 'thou shalt not' styled commandments under the label of 'fact vs fiction' they should have consistent enforcement of obvious issues that have been brought up publicly numerous times, including on the very post highlighting the policy. But we know they won't! They only care about dirty business practices if they are not getting a taste of the revenue stream (as shown by their BearShare partnership while policing Bing affiliates).

Based on that sort of activity, when Google announces a preference while promoting "openness" it is easy to see it as a step backward, hypocritical, or even as a farce. Embrace, extend, extinguish.

As Google pushes more to advertise itself on its ad network it is no surprise that they were eventually forced to disclose payout percentages. But it took a lawsuit to do it.

After purchasing Youtube Google rolled out their universal search & was fine with aggressively promoting Youtube over other video services. Only recent government reviews have pushed Google to give 3rd party services a fair shake, but the network effects and lead are likely already too great to overcome.

Due to past search bias, Google might get blocked out of completing the ITA deal. The good news going forward for publishers is due to increasing regulatory heat Google will only go after a small number of verticals where they payouts are huge. The regulatory blowback will be too great for them to try to be all things to all people.

When Google's head spam fighter is doing public relations AND the Washington Post covers his lobbying you know Google is nearing a breaking point.

Delays, Oh No

We were hoping to launch today, but we still do not have all the bugs worked out for all our modules/plugins to make them compatible with Drupal 7. Further, our programmer mentioned that some of the Drupal 7 documentation is missing, which makes the above task even harder. He is making great progress with the upgrade, but between the design coming in a bit late + me getting sick for a long while + all the integration issues we are going through, we are estimating our re-launch date to be either January 31 or February 1st.

I realize that is about 2 weeks away, but I would rather be conservative on the estimate rather than promise it will be 3 days and keep moving the goal post over and over again every few days. If we can launch sooner we will, but barring our internet connection dying permanently or yet another major illness, we should be launched by the 1st of February at the very latest.

Sorry for the delays, but on a positive note, this also gives us more time to make more custom graphics for our training area and do more updates within the training section. It also allows me to blog somewhat regularly over the next week, before sorta disappearing publicly to work on the membership area of the site when re-launched.

SEO Traffic is the CLEANEST and MOST VALUABLE Traffic Online

Microsoft Revenue Per Click Equals Google's

Microsoft adCenter has recently increased their revenue per click to match Google, in spite of having a small chunk of the search market share (maybe 25% between Bing and Yahoo! Search to Google's ~ 75%).

All we hear about Google's love for the scientific method, the superiority of their relevancy algorithms, them creating the best thing for advertisers, etc. has prettymuch been reduced to fluff.

Google is much more aggressive at forcing searchers down a set path, has a broader ad feature set, has more (~ 3x) search marketshare, and yet Microsoft is able to compete. And they have done this against a competitor which keeps making incremental changes to build additional yield.

The Arbitrage Game

How was Microsoft able to increase their yield so much? If you go back 5 years, at the time Yahoo! powered a greater share of search traffic then than Microsoft does now, and their ads were powering both MSN and Yahoo! Search. How did Microsoft catch up with Google when Yahoo! failed to compete?

One word: arbitrage.

I have long railed on Yahoo! for screwing advertisers with fraudulent traffic sources. Arbitragers destroyed the value of Yahoo! clicks & it wasn't until 2010 that Yahoo! allowed you to opt out of the fraudulent traffic.

Even today, Yahoo! still arbitrages search traffic through their home page's trending now section.

Notice the word highlighted in yellow. In most cases Yahoo! will typically spike one or two commercially oriented keywords into their trending box. Having ranked for numerous of these keywords, I can tell you that they can drive thousands of search clicks...which can be an expensive shot of traffic if you are paying $5 a click for them. The 'high blood pressure' might be a Dollar or two, but I have seen some expensive finance keywords in there as well.

A Look Under the Hood of Smaller Search Engines

The thing to understand about paid sources of traffic is that as soon as you add the element of payment there becomes a set of mixed incentives along the value chain.

I won't tell you which search engine it was, other than to say it was a publicly traded one, but about 4 years back a second tier search engine sent me a spreadsheet of [keywords * their bid prices] and wanted me to "generate traffic" for them.

1). Direct Partnership - Pull our ads to display on the site(s) for high paying keyword terms. The traffic must be unique and convert well for the advertiser (search engine traffic is the best). We can display ads in a variety of format and target the top terms on our network. Makes for a good compliment to other revenue streams.

2). Aggressive Referral Partnerships - I will compensate you and/or any other contacts in the black hat SEO realm up to 10% of all revenues generated by referred partnerships. (There are some SEO guys out there doing 1K+ per day in revenue - 10% = 100.00 additional per day for the life of those accounts). I am definitely willing to compensate nicely for referral of these contacts for Direct Partnership deals.

That second tier set up is of course why so many affiliate blogs recommend signing up for every affiliate network in the world. But the big issue with Yahoo! was that (in spite of being a major leading search engine) they were still operating like the 3rd tier folks, with certain publishers being able to access high payouts and CPC stats. Some of the folks running the Overture feed where whoring out out to others & one well known webmaster even has the word "clickbot" in his nickname. Yahoo! made it hard for advertisers to opt out, and that is what killed their click value.

What coincided with Microsoft's increasing revenues per click? They reined in the arbitrage folks.

"Although the Yahoo-Bing integration has been ongoing for several months, during which time we were able to adapt well to the volatile environment, in mid-December we began to experience average revenue per click decreases and the strategies we customarily deploy for responding to such decreases were not as effective," said Geoffrey Rotstein, CEO. "As a result, we are maintaining substantially lower traffic levels until we have better insight into the factors contributing to this issue. The Company is currently working diligently with the teams at Yahoo! in an effort to implement any necessary adjustments to this new marketplace."

"We have always been able to adapt quickly and positively to changes in the industry as a result of our intense focus on data and analytics. We intend to apply this same discipline to respond to these issues, as we continue to receive information from Yahoo! that will assist us to adapt our system for the new advertising marketplace," added Ted Hastings, President. "We intend to make whatever changes are required within our Company to ensure a fast and sustainable response to this new market".

SEO = Still Amazing

But the purpose of this post is to point back to the value of SEO clicks. Advertisers spend over $30 billion a year buying ads from search engines & the organic search results still get the bulk of the clicks. Of course search engines are pushing to eat the organic results as well, but for anyone who has a strong organic traffic stream it is easy to under-appreciate the value until you realize how scarce and expensive pure & clean search traffic is.

Make hay while the sun shines! :)

Google Product Search Ecommerce Play

In a "oh what is the brown stuff oozing from my pants" moment for some e-commerce site owners, Google has quietly entered the space of pulling in manufacturer data directly into Google product search:

To make these pages even better, we plan on working with suppliers and manufacturers to get product data straight from the source.

We are starting this effort through a business partnership with Edgenet, a provider of product data management solutions. Manufacturers and suppliers can work directly with Edgenet's Ezeedata service to submit high-quality product data and images to Google. For more information, you can visit their website, at www.edgenet.com .

Example here.

In the past Google has also beta tested sneaking paid inclusion into their product search, plus they have already started hard-coding their ebook results in the organic search results in the US (without disclosure). at some point you can count on this huge block of product information Google is pulling in to appear directly in the organic search results, pushing many ecommerce organic search results below the fold. Boutiques.com was just the start of a trend.

What makes this trend scarrier is that everyone is doing it: Google, Yahoo!, Bing, Ask, etc.

The mental model I have come to view search through is this: if a search engine can cut you out of the supply chain while having similar quality then they consider you to be at best irrelevant and at worst a spammer. Alternatively, the more your offering looks like a search engine, the more likely it is to be viewed as spam.

The big issue with this is network effects. Outside of brand corrosion & legal issues, there is basically no limit to how far search engines can push. Sure the above focus is on ecommerce, but don't forget that Google is buying Metawebs + ITA Software. And they have the ability to create vertical databases on the fly. If you want their search traffic you have to opt into being scrapped and disitermediated, likeso:

You can differentiate by having product information. But Google scrapes it. You can differentiate through consumer & editorial reviews. But Google scrapes it. You can differentiate by brand, but Google sells branded keywords to competitors. No matter what you do, Google competes against you. You can opt out of being scraped, but then you get no search traffic (& the ecosystem is set up to pay someone else to scrape your content + wrap it in ads).

If you are a big player (like TripAdvisor) you can tell Google to get stuffed & re-negotiate more favorable terms. Smaller players don't have that luxury. Without that leverage, Google doesn't feel they have a spot on the commercial web.

These sorts of trends make the concepts of branding and positioning more important. If Google (and similar companies) aim to consolidate down markets into fewer players then it makes sense to be a #1 in a smaller niche market than a #5 in a bigger one.

Link Exchange Request Emails

A lot of folks have been hammering away at sending out automated link exchange emails for Wordpress driven sites.

The hallmarks of many such efforts

  • URL with something cheesy like "partners" or "friends" or "roundtable"
  • automated emails without a name that mention a search engine ranking and (falsely) apologize for being sent multiple times
  • auto-generated content that is overly boastful & looks like it comes from one of those internet marketing review sites that has fake comment bots which say *everything* is the best thing since sliced bread / a genius in motion / a deity of your choice
  • Thumbshot previews
  • a bit of technical trickery

Nice bit of false empathy there. ;)

The technical trickery mentioned above is that if you visit the link they put in the email the linking post will appear *all over* the site that is "linking" to you. But if you open up a new browser from a different IP address and try to visit the parent category page before visiting the individual post page you will see that the post is only visible to a person who knows exactly where it is. So the people are not only mass automated email spammers, but they lie at hello as well (by deceiving folks into thinking there is an on-the-level exchange of some sort, while screwing them over with a page that is invisible to everyone but them).

The stuff is so out of hand that even new age doomer movies about 2012 are using it & are sending the emails to sites about SEO, offering sources of 'enlightenment.' :D

Clearly they are enlightened. ;)

Some tips & strategies:

  • The easiest way around such issues is to delete unsolicited commercial messages, especially if they are not personalized. But if you want to give someone the benefit of the doubt, then the best way to do so is check the source code of the page inside Google's cache. If the page isn't cached by Google then generally Google probably doesn't care much about it. (Yes there are exception to that, but the people who are sending unsolicited emails probably do not deserve too much benefit of the doubt.)
  • If you are out sending emails asking for links then it goes without saying that you don't want to look like the above folks (though I have received *far* worse emails from some SEO companies & PR folks). Automated tools can be dangerous things when in the hands of tools!

Pages