Just before last week's SES conference Google announced the launch of their new caffeine infrastructure:

For the last several months, a large team of Googlers has been working on a secret project: a next-generation architecture for Google's web search. It's the first step in a process that will let us push the envelope on size, indexing speed, accuracy, comprehensiveness and other dimensions. The new infrastructure sits "under the hood" of Google's search engine, which means that most users won't notice a difference in search results. But web developers and power searchers might notice a few differences, so we're opening up a web developer preview to collect feedback.

In the new infrastructure so far I think there is...

• an increased weighting on domain authority & some authoritative tag type pages ranking (like Technorati tag pages + Facebook tag pages), as well as pages on sites like Scribd ranking for some long tail queries based mostly on domain authority and sorta spammy on page text
• perhaps slightly more weight on exact match domain names
• perhaps a bit better understanding of related words / synonyms
• tuning down some of the exposure for video & some universal search results

You can check out the new results here and CompareCaffeine.com offers side by side comparisons of new Google + old Google - similar to the recent blind search service which compared Google, Yahoo!, & Bing results.

As highlighted by Matt Cutts, the aim of the infrastructure change is to allow further evolution of search...so if you analyze the results a bunch now then what you are aiming for might look quite different than the search results which appear in November.

An article in The Register offers background info on Google Caffeine, and additional analysis has been done by David Naylor and Bill Hartzer.

This WMW thread mentions some relevant background on Google's approach to storage. In his post on the update John Andrews mentioned how smaller chunking of data could allow the algorithms to make SEO more challenging (or at least more holistic):

Smaller chunks means faster SERP generation…. and possibly more specific quality management (smaller more specific binning of URLs if desired) How this plays out for SEO is interesting now… and especially whether or not we will be able to influence various aspects independently from the whole.

The ROI on effective SEO campaigns is simply unbelievable, and Google is going to do everything in their power to diminish the ROI of algorithmically focused optimization efforts. As the cost of memory drops and the algorithms improve, the next couple years might separate the men from the boys in the SEO space. Those improvements will drive many SEO practitioners into parallel fields like niche publishing and public relations. 5 years ago was the perfect time to start building your empire. But starting today is far better than starting tomorrow.

In one of the more absurd public privacy invasions online, Google has announced they are going to use FICO scores to help advertisers target ads at consumers in different credit buckets.

I wonder if at some point in time if AdWords advertisers selling the scammy government grand & biz-op offers will get to use this data to target poor people with low credit scores. It only makes sense that Google would spin this positively stating that it is good for brand advertisers to find credit-worthy customers, which is the story that was marketed in the above linked piece:

Consumers with high FICO scores demonstrate some unique attributes that show they shop carefully for the best cards. For example, shoppers begin using search earlier in their application process, they use the term "best credit cards" at three times the rate of lower FICO shoppers, and they are more likely to use branded terms.

Consumers with high FICO scores use non-branded search terms more than branded -- approximately 60% of high FICO searchers. They tend to search on terms, such as "travel rewards," "low rate," and "balance transfer."

From a marketer's perspective this makes a lot of sense. Smart people who manage their credit well look for tangible benefits in their financial choices...they don't just blindly buy the brand.

The problem is that (so long as the current bankruptcy "reform" remains in tact and taxpayers bail out any banking losses) bankers have little to no incentive to reach people with good credit scores. People who pay their credit cards on time are seen as deadbeats while the least credit worth are the profitable market segment because they use credit ignorantly and accrue billions of dollars in unneeded fees every year:

Overall, we find that debt literacy is low: only about one-third of the population seems to comprehend interest compounding or the workings of credit cards. Even after controlling for demographics, we find a strong relationship between debt literacy and both financial experiences and debt loads. Specifically, individuals with lower levels of debt literacy tend to transact in high-cost manners, incurring higher fees and using high-cost borrowing. In applying our results to credit cards, we estimate that as much as one-third of the charges and fees paid by less knowledgeable individuals can be attributed to ignorance. The less knowledgeable also report that their debt loads are excessive or that they are unable to judge their debt position.

Bankers have historically hidden these fees in illegible 30+ page contracts, as mentioned by Elizabeth Warren

I teach contract law at Harvard Law School and I can't understand my credit card contract. I just can't. It's not designed to be read. Read the Government Accountability Office (GAO) study on this. The GAO looked at credit cards and they said: "Nobody can understand this stuff." Are you kidding me? And understand when you've got terms that say: "In effect, we'll charge anything we want any time we want for any reason or no reason at all," what's the point of reading it?

She later commented on the ideal credit card customer

Every credit card for a credit card company is like a lottery ticket. They're just waiting to see who's going to maybe stumble a little. Maybe get into trouble on a car loan. Maybe nothing at all except they just look vulnerable. They're just in the right zip code. They're just the right profile for people who won't be able to run any place else. And those are the ones you slam. Those are the ones you hit with the 29 percent interest rate, the 35 percent interest rate, the new fees. And then, because of course if you can't pay it, then you get hit with a fee for not paying or for paying late, for going over limit. And the game is afoot. With any luck at all from the credit card company's perspective, these people will become little annuities that will just keep generating profits for the credit card companies for months, for years, maybe forever.

The idea of only servicing legitimate debt needs of customers that can afford their credit card bills has made banking industry executives so angry that they are threatening hitting consumers with lots of bogus new "conveninece" fees:

Now Congress is moving to limit the penalties on riskier borrowers, who have become a prime source of billions of dollars in fee revenue for the industry. And to make up for lost income, the card companies are going after those people with sterling credit.

Banks are expected to look at reviving annual fees, curtailing cash-back and other rewards programs and charging interest immediately on a purchase instead of allowing a grace period of weeks, according to bank officials and trade groups.

This new consumer-credit profiling Google is offering will be far more profitable to use on the poor, the weak, the desperate, the ignorant, and the uneducated. In early research Lawrence Page and Sergey Brin stated

Since it is very difficult even for experts to evaluate search engines, search engine bias is particularly insidious. ... We believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm.

So were they right back then? Or are the right now? They can't be both.

Update: Sandra from Google's PR team emailed me the following

I've included an outline of the research methodology below the body of this note. Please let me know if you have questions or need clarification on any of the below -- or anything else, for that matter.

Best,

Sandra

-----

* Compete conducted a clickstream analysis on their opt-in panel of 2 million US online consumers, to associate FICO score categories with sites in the Google Content Network.
o The analysis took a look at the online behavior of Compete's opt-in panelists who shopped for or applied for a credit card online between January and March 2009, for the 30 days prior to the application and/or research.
+ Compete, via a sister company that provides secure matching of certain characteristics (one of which is FICO scores) to anonymous/anonymized individuals in the Compete panel, segmented the opt-in panelists into one of three categories, based on their FICO score: Super Prime (720 and above), Prime (600 to 719), and Sub-Prime (below 600).
+ Individual scores and personally identifiable information were not used by Compete, nor were they received by Google.
o Google provided Compete with a list of all sites in the Google Content Network.
o Compete compared how panelists in each FICO band searched and where the panelists spent time on the GCN, and ranked each GCN site based on its ability to reach consumers in particular FICO score bands.
o The ranking/scores of the GCN sites were passed on to Google -- not any information about the credit scores of individuals.

There are a lot of parallels between Google AdWords and SEO, and a lot of the beginner mistakes are the same for both traffic acquisition strategies. I figured it would be worth outlining some of the most common ones to help save you money on your search engine marketing campaigns.

1. Weak Domain Name
2. All Search Traffic Driven to Homepage
3. No Link Building
4. All Links to the Homepage
5. No Link Anchor Text Variation
6. No Focus on Quality
7. Lacking On Page Optimization
8. No Site Structure
9. Site With No Value Add
10. Competitive Saturated Market With Inadequate Budget
11. Picking a Market for 1
12. Pick a Market Which Does Not Monetize
13. Over-Aggressive Monetization From Day 1
14. What Other Common SEM Errors do You See?

1. Weak Domain Name

Google AdWords

When I interviewed Perry Marshall about AdWords he recommended split testing URLs because the URL can and does have a major impact on your ad clickthrough rate.

Since Google factors click-through rate (CTR) into their quality scores, anything that influences CTR influences your click prices. And while competitors can and will steal your AdWords ad copy, they CAN'T steal your domain name.

SEO

There are many potential errors that can be made with domain names. Two of the more common errors are creating a domain name that is impossible to remember and creating a name that restricts expansion.

Recommendations

Some people feel the need to limit their domain name budget to $10, but it is a foolish strategy. Almost every piece of marketing you do will be influenced by your domain name. Your domain name has limited recurring costs associated with it, but can represent a huge recurring market advantage or disadvantage. Yeah for CreditCards.com, and boo for cheapest-online-apply-credit-cards-and-loans.info.

1. If you are using Google AdWords for a new product or a non-branded product then test clickthrough rates across multiple domain names.
2. Make sure your domain name allows you to expand as needed. This is sorta an error I made early on with this site...I had no idea how successful the site would become when I started it and did not anticipate us creating the #1 SEO training program back when I thought of selling an ebook.
3. Avoid names that are impossible to remember. If you intend to create something that is easy to market online and offline then your domain name must pass the phone test, which typically means avoiding hyphens & numbers. This is especially true if you are trying to build a big brand.
4. If you feel your company may expand internationally it is best to buy any matching domain extensions where you might intend to eventually do business.
5. Exact match domain names can create a big SEO advantage if you can afford them - since some engines may give them a relevancy boost and your domain name influences the anchor text people use when they link at your website.

2. All Search Traffic Driven to Homepage

Google AdWords

1 page can only be relevant for a certain sector of search queries. In an efficient market anyone who directs all traffic to the homepage will lose a lot of money.

Every additional click you force users to make has some amount of slippage. When using Google AdWords / pay per click marketing a small change in conversion rates can be the difference between sustained profits and sustained losses.

SEO

It is sorta impossible to make a page "optimized" for hundreds or thousands of popular keywords because eventually after you add enough different keywords in the page copy it ends up reading bad and it harms conversion rates.

With SEO efforts mis-directing traffic is not as obvious as it is with AdWords because you don't have to pay for every click. But giving users an irrelevant experience still means you are throwing money away and only operating at a fraction of your potential.

Recommendations

With the prevalence of Google (and web search in general) every page of your site is the front door. We navigate via search. So map out keywords against URLs and try to offer the most relevant user experience whenever possible.

Observe how we map out core keywords, variations, and modifiers.

Some Google AdWords advertisers take perceived relevancy one step further and use the search query to help define the page content through the use of keyword insertion into their page copy and/or altering the page based on geographic information based on your computer's IP address.

3. No Link Building

Google AdWords

The equivalent of links to AdWords is keywords in your AdWords account. If you only advertise on 1 or 2 keywords you miss out on a large stream of relevant traffic.

SEO

If you build it they will come is simply not true in the search game. If it was easy to rank for competitive keywords without links then few companies would buy AdWords ads. You can't typically rank a new site until you have some level of awareness. Search engines follow people. Links are seen as votes of trust.

Recommendations

With AdWords, don't just bid on 1 keyword. Look for additional relevant variations that make sense. If you don't mind splashing out $50 you can also look at what competing sites are advertising on using SEM Rush, Keyword Spy, SpyFu, and/or KeyCompete. There are so many new tools popping up in this market segment that I have not had the time to review them all.

For SEO, download SEO for Firefox and the SEO Toolbar and look at how many links competing sites have and how many domain names those links come from. You will likely need to build some number of links in the range of what competing sites have (from a similar set of sites) to rank. Today is the perfect day to start building links. And yesterday was even better. ;)

4. All Links to the Homepage

Google AdWords

Since you are buying the links from the search engines based on keyword, this problem would be corrected by solving issue #2.

SEO

A variation of the above thinking. Most quality sites have useful content somewhere that people link to editorially. If all your links point at the homepage then that means you are not using anchor text from external links to boost your internal page ranks. In most markets that creates a big loss considering that some of those pages would get a lot of traffic with just a few more deep links which would yield higher rankings.

Recommendations

Search is a winner take most market. Analyze your traffic patterns, rankings, and target keywords to ensure you are promoting key pages. Look at top competing sites and keyword ranking values to gain additional insights.

Create linkworthy content that people would want to link at and push market it. The objective (vs self-interested) viewpoint here is "if you did not own your site what is unique about it that would make you want to visit it every week and/or recommend it to a friend?"

5. No Link Anchor Text Variation (or AdWords Ad Copy Variation)

Google AdWords

You shouldn't use the exact same ad copy on all of your keywords. You should segment it out by trying to understand user demand and create compelling advertising text that is relevant to the search query, relevant to the user demand, and relevant to your landing page. If you use a single generic boilerplate ad copy you are loosing a lot of money because your ad will not look as relevant as some of the top competing ads.

SEO

When people link to things naturally there tends to be some variation involved. If all your inbound links say "my keywords" then that can look suspicious...particularly if you are buying lots of links.

Recommendations

With AdWords, at a minimum you would want to use dynamic keyword insertion. But if you sell a lot of different products then you should try to find a way to match up small groups of relevant keywords against a set of ad copy. Make your core keywords stand out on their own, and be willing to be somewhat less descriptive with low search volume backfill keywords.

With SEO you should try to mix up your link anchor text when you are manually building links. If you create original compelling content that people want to link at (and push market it to the right audience) then that will also pull in natural anchor text.

6. No Focus on Quality

Google AdWords

Some advertisers are compelled to go after "cheap" clicks. But some of the more expensive keywords are expensive because they are associated with significant and valuable consumer demand.

SEO

Google algorithms estimate the probability of a new site being quality or low quality. If you start off with 2,000+ "free" directory links you align your site with sites that are often of lower quality. Similarly, if you try to promote watered down or average content then few people will be receptive to those efforts.

Recommendations

There is nothing wrong with buying cheap traffic, but make sure you track the business value you get from that traffic. If you buy "cheap" traffic from 3rd tier ad networks and/or keywords without any commercial intent those will not build your business anywhere near as well as developing a solid traffic stream from valuable industry keywords on leading search engines.

Start your link building efforts with quality links first. As your site gets more trusted you can fill in some lower quality links as well, but you don't want to do it first, and you don't want to do it in bulk.

When you decide to do push marketing for link building make sure the content you are promoting is unique, original, useful, compelling, & citation-worthy.

7. Lacking On Page Optimization

Google AdWords

Quality user experience and usability are crucial to converting well. When users come from search to your site they are switching channels. The more cues you can give them that they are in the right place (like relevant page headings + navigation) the higher your conversion rates and visitor value should be.

SEO

For really competitive queries links are crucial, but you can rank for many less competitive keywords and keyword variations without lots of links (because there is much less competition for those keywords). And even if you have lots of links, it is still typically hard to rank for keyword phrases that are not in your page copy.

Recommendations

With Google AdWords you can reach many of the stray searchers by using a combination of phrase match and broad match, and then using negative match to filter out irrelevant searches.

For every person searching for "seo" or "sem" there are probably 10 people searching for more obscure queries like "how do I promote my business on Google?" You can see how our page about link building ranks for hundreds of related keywords.

This is probably the single most powerful graphic explaination of why having lots of useful on-page content:

With SEO you can reach a lot of the searchers by using alternate word forms, alternate word orders, related phrases, and keyword modifiers in your content.

8. No Site Structure

Google AdWords

If your AdWords ad campaigns are not well organized then you are likely losing money. A strong site structure also helps ensure that your AdWords account has a strong structure, which can aid profitability.

SEO

If your site is not structured well then...

• navigation will likely be hard or confusing
• some of your key pages may not get much of your link authority
• some of your unimportant pages may accumulate a lot of your link authority

Recommendations

Most successful websites have a structure in which key pages which are mapped out against user demand and search volume.

• Create separate AdWords campaigns based on goals. Perhaps you can have campaigns for brand related searches, seasonal offers, public relations, campaigns that are based on ROI metrics, and even backfill campaigns like misspellings.
• Some content management systems (CMS) have major errors with duplicate content and site structure issues. A review of that topic is beyond the scope of this article, but search for the name of the CMS and SEO prior to implementing it to verify there are no serious issues and/or that there are easy fixes on the market.
• Set up site categories and sub-categories that are aligned against the keywords people use to search for your products and services.
• If you blog (or publish content regularly) reference older related materials when relevant.
• If your content is in a database you can use automated contextual links to help fix some site structural issues and redistribute PageRank down toward lower pages in your site structure.

9. Site With No Value Add

Google AdWords

If your site does not add much value it can be quite hard to sustain profit margins in the AdWords market. Affiliates routinely copy the work of each other and drive up click prices, which kills profit margins.

SEO

My very first profitable website was a no value add website that I got some spammy links for. The site did make thousands of dollars in affiliate commissions (a gift from God at the time), but that income was only made ***because*** I was a bad speller and misspelled some casino brand names back before search engines integrated spell correcting aggressivley. Such a site would simply go nowhere today.

Google often considers sites without value add to be unneeded duplication and/or spam. If you ever get a chance to read some of the Google Remote Quality Rater Documents you can see what Google believes is associated with "value add."

Recommendations

• In competitive AdWords markets competing businesses are forced to keep improving their business processes and efficiencies to be able to afford increasing bids from competing businesses.
• If you have a lower lifetime customer value than competing businesses you may eventually be driven out of the market.
• With some seedy affiliate offers in many cases the only people with sustained profit margins are basically those who are surprisingly sleazier than the rest of the market or those who are barely breaking even themselves, but are using their blog to build a downstream of followers that they get commissions from.
• Some (perhaps most?) affiliate networks ***will*** shave your commissions AND steal your keyword list if you send them the data.
• If you don't have a value add and want to play catch up in a competitive SEO market you need to have some sort of competitive advantage (be it nepotism, domain name, market experience, etc.).
• Making paid things freely available, creating useful software or tools, and having deeper & better editorial are 3 great ways to add value and win marketshare.

10. Competitive Saturated Market With Inadequate Budget

Google AdWords

In some markets it is hard to compete buying traffic without having a strong brand. If Geico pays Google $30 a click, but only pays affiliates $10 per lead then there is no way an affiliate can compete against Geico on the core industry keywords like auto insurance.

SEO

Want to rank for hotels and insurance? Me too. But I am uncertain if I have the resources to do it from scratch in a lasting manner given the algorithmic trends promoting well branded business and how corporations are increasing their SEO budgets.

Recommendations

• Have big ideas, but set reasonable goals, and measure progress.
• Do the math in advanced to estimate how much you can afford to pay for a click.
• Pick and chose your spots in the Google AdWords market. If after you do significant testing and optimization a word is still losing money consider dropping it.
• Try to pick a market position you feel you can dominate. The #20 result for "insurance" produces traffic worth ~ $0. The #2 or #3 result for "pet insurance" yields much more.
• Make at least 1 incremental improvement to your web business everyday.
• Aggressively re-invest early profits into growing your website and building a moat.

11. Picking a Market for 1

Google AdWords & SEO

If there is no demand for an idea then it is quite hard to create demand through search engine marketing. Search engine marketing works best when it captures existing demand.

Recommendations

• Keyword research tools can give you estimates of search volume.
• Since AdWords is so much quicker and easier to test than building a full site and implementing an SEO campaign, you can use AdWords to test market demand and interest for an offer before spending money building and marketing a full website.
• It can be good to be out front of trends (as one of the easiest ways to win a market is to be the first person in it), but just as easily you can go after an established high money market with your own original spin or angle.

12. Pick a Market Which Does Not Monetize

Google AdWords

If similar competing business models have much higher visitor value you may have to change your business model to compete. Some low earning business models might simply be precluded from participating in the AdWords market in a meaningful way.

SEO

There is nothing wrong with building a site about a topic you are passionate about and interested in without knowing how well it will monetize, but if you are trying to build a business you should pick something with a high enough visitor value to create enough profit potential to make it worth the time and money investment.

Recommendations

If you are planning on participating in the AdWords market, but have a low margin business then you should look for ways to increase profit margins, customer order size, and lifetime customer value.

If you run an editorial site it can be a good idea to under-monetize off the start to build market momentum without people viewing you as a competitor, but it can be hard to bolt on a business model if you have spent a lot of time servicing the wrong market segments.

13. Over-Aggressive Monetization From Day 1

Google AdWords

If you are buying traffic there is no problem with trying to monetize it. But most website visitors will not convert.

SEO

Sell in line text links & have pop ups? Is ever other post an affiliate link? If so, why would anyone want to subscribe to an ad stream when there are many useful alternatives to look at?

Recommendations

• Since most website visitors will not convert to paying customers on the first visit, you should look to establish a relationship with them by giving them a free offer and/or some reason to come back to your website. You can see the offer we make at the bottom of our pages and on our join now page.
• Existing leading trusted sites that have built up a following benefit from cumulative advantage. If your site is brand new and driven by editorial content it is a good idea to give away more value than you capture. Under-monetize until you build enough market momentum to make your rankings stick even when you do monetize.
• Consider monetizing some areas of your site more aggressively while not monetizing other sectors of your site, but instead using them for public relations and link building.

14. What Other Common SEM Errors do You See?

Rel=Nofollow Wastes PageRank

About 2 weeks ago Danny Sullivan highlighted that Google follows Javascript links, and that sculpting PageRank using rel=nofollow no longer works. Matt Cutts shared that second bit to the shock and awe of the SEO industry at the recent SMX conference.

And Nofollow Has Done so For Over a Year Now

While Matt Cutts only recently announced the change, this change is something that was done over a year ago:

More than a year ago, Google changed how the PageRank flows so that the five links without nofollow would flow one point of PageRank each.

Matt explained why they never disclosed the change back then:

At first, we figured that site owners or people running tests would notice, but they didn’t. In retrospect, we’ve changed other, larger aspects of how we look at links and people didn’t notice that either, so perhaps that shouldn’t have been such a surprise. So we started to provide other guidance that PageRank sculpting isn’t the best use of time.

Why Google Engineers Once Pushed Nofollow PageRank Sculpting

Originally Google created rel=nofollow in what was claimed as an attempt to minimize the effects of blog comment spam on their search results. But the tag never decreased blog spam, it only decreased the ability of bloggers to influence search rankings by leaving back-scratching comments on each other's blogs.

Matt Cutts quickly extended nofollow's purpose to include use on paid text link ads as well. But given that Google AdWords sells links (and often to scammers) some people may have seen trade issues with forcing the new proprietary nofollow tag onto the web. Promoting PageRank sculpting gave Google a way to legitimize a tag which otherwise added no value to anyone except search companies.

After enough time passed and Google saw too much collateral damage popping up from rel=nofollow usage, they pulled the rug out from underneath it. Nofollow already had enough momentum, and was a functional part of the web. After a Google employee slipped nofollow into a working draft of the HTML 5 specifications it was time time to clean up the mess and inform SEOs about the nofollow change that happened over a year ago.

Some SEO Professionals Claimed Huge Benefits From PageRank Sculpting

Over the last year many SEOs have claimed that nofollow tests worked amazingly well which show up directly in the bottom line. And ironically, sharing/hyping this incorrect information worked well from a marketing perspective because...

• it makes them look cutting edge and allows them to sell additional services
• writing about things which are new, uncertain, and untested yields links (because for every person who is an SEO expert there are 1,000 ditto-heads linking to whatever sounds new or important)

What the SEOs were testing on their high profile public SEO websites was more a reflection of branding and marketing efforts. As they made noise in the marketplace their brand spread and that made more sales. We recently (maybe a month ago?) added nofollow to some links on our site, and we failed to see the lift that other SEOs claimed. And the SEOs that claimed to see the obvious huge amazing lift failed to report the drop off when Google changed how they handled nofollow, which sorta shows the error in the testing method.

Why Fake SEO Experts Recommended Using Nofollow Everywhere

It is no surprising that many self-proclaimed experts aim to misinform novices, as beginners are typically the biggest piece of a market and their topical ignorance makes them the easiest to monetize.

This is precisely why get-rich-quick email list internet marketers make so much money. There is always a new, desperate, and gullible crop to feed off of - an Eternal September. And until they get burned a few times and hardened by the market (and/or go bankrupt) they convert at rates well above what other market segments convert at. Greed makes it easy to make poor financial decisions, especially when matched against seasoned marketers and promises of automated wealth generation.

A More Holistic SEO Strategy

Part of my SEO philosophy has been to try to get the easy wins that you can figure out, but not to know the relevancy algorithms in intimate detail because it gets hard to isolate testing variables as sites get more established, and when you are competing for core keywords in big, competitive markets the SEO game comes down to industrial strength link building, public relations, social networking, branding, advertising, and other aspects of classical marketing.

Most of the SEO Market Misses Big Changes

Think of how many SEO blogs there are (literally thousands), and...

• a month after a major update happen a lot of people say nothing changed (even though Google confirmed the change and we highlighted it on our blog, backed by hard ranking data across many industries)
• nobody said anything when Google changed how they treat nofollow (we didn't notice the change because we have not used it much on many of our sites because we were afraid it would be taken as an SEO flag, given how Google profiles SEO professionals)

Lots of alleged testing in the SEO industry, but most of the stuff shared publicly is nonsense or misguided junk worth less than nothing.

What About "Experts" Who "Test" Everything?

About 6 months ago I talked to a person who claimed to be an expert at fine-tuned testing, and I was surprised as to how clueless they were about the influence of domain names on SEO. Even after I told them and showed examples they still didn't get it. They were clueless even after seeing the evidence. Domains are one of the few variables that are exceptionally easy to test, and it really validated my opinion that excessive testing can be a waste of time, as that the well known self-labeled "expert tester" was so ignorant about something that is so easy to test. Another self-promotional expert recently claimed that hyphenated domains were the way to go because he has data on 40,000 customers who are all using his misinformation. (Of course he didn't word it that way, but a sampling error he made, and 40,000+ people are losing money because of that advice).

Some People Know The Algorithms, but do Not Share

The one disclaimer I would on this front is that there are some SEOs who likely know the relevancy algorithms better than many Google engineers do. Guys like David Naylor, Greg Boser, Fantomaster, and Eli can do a lot of deep-algo testing based on how many sites they operate and how good they are at doing it. But those guys spend a lot of time and money doing their testing, and don't share their advanced research publicly until they feel it makes sense to from a strategic standpoint, as noted in our recent interview of Eli:

Isn't the value of many aggressive SEO ideas inversely proportional to the number of people using them? What makes you decide what ideas to share and when to share them?

In many cases that's absolutely correct. I've shared several techniques that have died within days of posting them. Just to list a few examples, my Abandoned Wordpress series, Wikipedia Series, and Amazon.com exploits. In all these cases I know before I ever post it that it'll die moments after I do. So most of the time I'll post it out of greed. They are usually techniques I've been using for several years and have since retired them out and quit using them. Naturally with any technique others are bound to figure it out. When I start seeing them popup underground and are being used against me in increasing numbers when I'm no longer using them myself I might as well wreck it.

If you only have a few sites testing many variables is much harder than many people try to make it seem, and it takes a serious investment and skill level to be at the level of the above mentioned names.

SEO "Experts" Jumping from 1 Bad Recommendation to an Over-Reactive Increasingly Worse Strategy

Based on the current Google information on nofollow, some SEOs are already recommending that you strip the ability of commenters to add any outbound links to comments so you can hoard more PageRank. And some are suggesting putting comments in an iframe. But in most cases, such advice is at best misguided. Why?

1. Comments offer free relevant textual content that helps your pages rank for a wider array of related keywords.
2. Allowing some relevant outbound linking makes the page more useful, and makes some people slightly more likely to want to comment.
3. When you are competing for core keywords in big, competitive markets the SEO game comes down to industrial strength link building, public relations, social networking, branding, advertising, and other aspects of classical marketing.

Anything that makes your site more of an island (especially for new sites that need to buy market-share and momentum any way they can) makes it harder to compete against more open sites and well established competitors. If you close off a marketing channel then you are simply ceding a marketing advantage to a newer (or a more savvy) competitor.

Google Webmaster Central have just updated their toolset, so now might be a good time to review it.

For those new to Webmaster Central, Google offer a set of very useful tools that provide an insight into how Google handles your site. You can find out how your site is crawled, what keywords you rank for, and if there are any problems.

You can sign up here.

Lets step through the features, and share some ideas on how to best use the data Webmaster Central provides. For those already familiar with Webmaster Tools, it would be great if you could offer newer users some of your tricks and tips in the comments.

Check Out Our New Look

Once you've signed up, you'll be greeted with a dashboard screen. This is where you list all your sites.

Not a major change, but a nice, clean layout that makes it easier to navigate. The dashboard gives you a quick overview of top search queries, crawl errors, and links to your site.

A nice feature is that you can have data forwarded to your email address.

Google outlines the changes here.

The Toolset

Google have provided a useful toolset to help diagnose site problems, and give you insights into how to perform better in Google's index.

Site Maps

Webmasters should submit a Google Sitemap to ensure their site is crawled.

Whilst not necessary, as Google should be able to crawl your site via links, it's a good idea to have the sitemap in case there are problems during the regular crawl. To create a Google sitemap, you can use Google's generator script. There are also various third party tools to help you do this.

Also see "Subscriber Stats" below.

Crawler Access

You can use this tool to test your robots.txt file. Check out this article about why you might use a Robots.txt. You can also use this tool to generate a robots.txt file, based on drop down options.

You can also use this tool to remove a URL from Google. Careful with that one ;)

Site Links

You can specify which site links you want to show. Sitelinks are the indented links below a listing in the search engine result pages.

Sitelinks look like this:

It's a good idea to specify pages that users will find most useful, and that display a consistent theme for your site. You can also block irrelevant pages from appearing in the site links using this tool.

Settings

If your site targets a specific country, you can specify this information. This is very useful if you have a .com, but target visitors in a country outside the US. The tool makes no difference if you already have a country specific TLD, however.

The preferred domain setting enables you to specify which URL you want crawled, the non-www version, or the www version.

Why would you do this?

www.acme.com and acme.com can be seen by crawlers as being two different domains. This can sometimes lead to duplicate content issues. So it's a good idea to pick one or the other, and specify it in Google tools.

The crawl rate setting allows you to slow down the search spiders crawl rate if you're having problems with the spider gobbling to many of your server resources. By default, Google sets the crawl rate for all sites.

Top Search Queries

Find out which keywords you rank for, which keywords send you the most traffic, and which results users clicked through on. If you rank for a lot of keyword terms, but aren't getting much click through, you might want to review the wording of your title tags.

You can also sort this data by device, such as mobile. You can find out which Google country domains send you the most traffic, as well as sorting by image search, and you can specify a date range.

Links To Your Site

You can find out which sites are linking to you, and what anchor text they're using. You can also download this data in the form of spreadsheets.

Keywords

Find out the most common keywords found on your site.

Are these words consistent with the theme of your site? If not, you need to balance out the content by adding more on-topic pages.

Internal Links

Shows which pages are most heavily linked internally, and from where. You can also see if pages are linked from non-www or www "domains". Aim to get internal linking consistent i.e. from either non-www or www but not both, and make sure your money pages are well linked.

Subscriber Stats

Find out how many people subscribe to your RSS feed via Google Reader. You can also submit a feed as a sitemap.

Crawl Errors

This useful diagnostic tool will show you which pages Google is having problems crawling. There is a breakdown by web, mobile, and mobile WML.

Crawl Stats

Use this tool to find out how many pages are crawled per day, and the time spent downloading those pages. This can reveal problems such as slow servers and page load errors.

Crawl stats also give you a PageRank distribution chart. However, like the toolbar, take this data with a grain of salt. I've found it bears little relationship to site ranking, and the real PR of pages is only known by Google.

HTML Suggestions

Use this tool to find duplicate and missing title and descriptions tags.

Tips On How To Use This Data

A recent Google presentation outlined this process for using Webmaster Tools to fix and find broken links. The bigger your site is, the more likely broken links are to crop up. This tool is also very useful if you've recently moved your site, and now have different URLs.

• Download your crawl error sources report from Webmaster Tools
• Fix any "on site" not found links
• Group external not found links by domain and sort
• Evaluate external not found links to prioritize
• Contact external sites to get links fixed
• Set up redirects for links that can't be fixed
• Implement a 404 page
• Make reviewing and resolving not found links a periodic task
• Evaluate how top ranking data changes over time, and look for ways to fix SEO issues if any rankings disappear

Drive More Traffic

Webmaster tools will tell you which keywords you rank for, some of which, you may not have been aware.

Go to the keyword ranking page, and see where you rank for certain terms. Are any of these terms valuable to you? You can estimate the monetary value of keywords by using Google Adwords estimator.

Find the page that is ranking well, and optimize further, either by tweaking the on-page content, boosting that page in your internal linking structure, or increasing the number of external links.

Remove Duplicate Tags & Descriptions

You're unlikely to ranked for pages that Google thinks might be duplicates. Make sure all your pages have unique title and description information. You can use the HTML Suggestions tool to do this. Again, this is a task you should perform periodically.

Enhance 404 Error Pages

Got any page not found errors?

You can make your 404 page a bit more useful by embedding a Google widget that helps your visitors find what they're looking for by providing suggestions based on the incorrect URL. This will hopefully prompt people to search further, rather than click back.

Enhance Clickthrough

The top queries tool features a column that shows you which pages users clicked through on most often.

If you've got a lot of pages ranking, but few click-thrus, you need to tweak your title tag data. Also, load up those search results and see what title tags your competition is using. Someone else is getting those click-thrus, so it's a good lesson to help you zone in on user intent for certain keywords. Keep tweaking until more pages receive click-thrus.

Got any tricks and tips? Add them to the comments.

Further Reading

• Google Webmaster Central Channel On YouTube
• Official Google Webmaster Central blog