SEM Rush has long been one of my favorite SEO tools. We wrote a review of SEM Rush years ago. They were best of breed back then & they have only added more features since, including competitive research data for many local versions of Google outside of the core US results: UK, Russia, Germany, France, Spain, Italy, Brazil.

Recently they let me know that they started offering a free 2-week trial to new users.

Set up a free account on their website & enter the promotional code located in the image to the right.

For full disclosure, SEM Rush has been an SEO Book partner for years, as we have licensed their API to use in our competitive research tool. They also have an affiliate program & we are paid if you become a paying customer, however we do not get paid for recommending their free trial & their free trial doesn't even require giving them a credit card, so it literally is a no-risk free trial.

What is SEM Rush?

SEM Rush is a competitive research tool which helps you spy on how competing sites are performing in search. The big value add that SEM Rush has over a tool like Compete.com is that SEM Rush offers CPC estimates (from Google's Traffic Estimator tool) & estimated traffic volumes (from the Google AdWords keyword tool) near each keyword. Thus, rather than showing the traffic distribution to each site, this tool can list keyword value distribution for the sites (keyword value * estimated traffic).

As Google has started blocking showing some referral data the value of using these 3rd party tools has increased.

Normalizing Data

Using these estimates generally does not provide overall traffic totals that are as accurate as Compete.com's data licensing strategy, but if you own a site and know what it earns, you can set up a ratio to normalize the differences (at least to some extent, within the same vertical, for sites of similar size, using a similar business model).

One of our sites that earns about $5,000 a month shows a Google traffic value of close to $20,000 a month.
5,000/20,000 = 1/4 = 0.25

A similar site in the same vertical shows $10,000
$10,000 * 0.25 = $2,500

A couple big advantages over Compete.com and services like QuantCast for SEM Rush are that:

• they focus exclusively on estimating search traffic
• you get click volume estimates and click value estimates right next to each other
• they help you spot valuable up-and-coming keywords where you might not yet get much traffic because you rank on page 2 or 3

Disclaimers With Normalizing Data

It is hard to monetize traffic as well as Google does, so in virtually every competitive market your profit per visitor (after expenses) will generally be less than Google. Some reason why..

1. In some markets people are losing money to buy marketshare, while in other markets people may overbid just to block out competition.
2. Some merchants simply have fatter profit margins and can afford to outbid affiliates.
3. It is hard to integrate advertising in your site anywhere near as aggressively as Google does while still creating a site that will be able to gather enough links (and other signals of quality) to take a #1 organic ranking in competitive markets...so by default there will typically be some amount of slippage.
4. A site that offers editorial content wrapped in light ads will not convert eyeballs into cash anywhere near as well as a lead generation oriented affiliate site would.

SEM Rush Features

Keyword Values & Volumes

As mentioned above, this data is scraped from the Google Traffic Estimator and the Google Keyword Tool. More recently Google combined their search-based keyword tool features into their regular keyword tool & this data has become much harder to scrape (unless you are already sitting on a lot of it like SEM Rush is).

Top Search Traffic Domains

A list of the top 100 domain names that are estimated to be the highest value downstream traffic sources from Google.

You could get a similar list from Compete.com's Referral Analytics by running a downstream report on Google.com, although I think that might also include traffic from some of Google's non-search properties like Reader. Since SEM Rush looks at both traffic volume and traffic value it gives you a better idea of the potential profits in any market than looking at raw traffic stats alone would.

Top Competitors

Here is a list of sites that rank for many of the same keywords that SEO Book ranks for

Most competitors are quite obvious, however sometimes they will highlight competitors that you didn't realize, and in some cases those competitors are also working in other fertile keyword themes that you may have missed.

Overlapping Keywords

Here is a list of a few words where Seo Book and SEOmoz compete in the rankings

These sorts of charts are great for trying to show clients how site x performs against site y in order to help allocate more resources.

Compare AdWords to Organic Search

These are sites that rank for keywords that SEO Book is buying through AdWords

And these are sites that buy AdWords ads for keywords that this site ranks for

Before SEM Rush came out there were not many (or perhaps any?) tools that made it easy to compare AdWords against organic search.

Start Your Free Trial Today

SEM Rush Pro costs $79 per month (or $69 if you sign up recurring), so this free trial is worth about $35 to $40.

Take advantage of SEMRush's free 2-week trial today.

Set up a free account on their website & enter the promotional code in the image located to the right.

If you have any questions about getting the most out of SEM Rush feel free to ask in the comments below. We have used their service for years & can answer just about any question you may have & offer a wide variety of tips to help you get the most out of this powerful tool.

Google has recently began refining search queries far more aggressively. In the past they would refine search queries if they thought there was a misspelling, but new refinements have taken to changing keywords that are spelled correctly to align them with more common (and thus profitable) keywords.

As one example, the search result [weight loss estimator] is now highly influenced by [weight loss calculator]. The below chart compares the old weight loss estimator SERP, the current weight loss estimator SERP & the current weight loss calculator SERP. Click on the image for a larger view.

There are 2 serious issues with this change

• disclosure: in the past refinement disclosures appeared at the top of the search results, but now it often ends up at the bottom
• awful errors: a couple months after I was born my wife was born in Manila. When I was doing some searches about visiting & things like that, sometimes Google would take the word "Manila" out of the search query. (My guess is because the word "Manila" is also a type of envelope?)

Here is an example of an "awful error" in action. Let's say while traveling you find a great gift & want to send it to extended family. Search for [shipping from las vegas to manila] and you get the following

The search results contain irrelevant garbage like an Urban Spoon page for Las Vegas delivery restaurants.

How USELESS is that?

And now, with disclosure of changes at the bottom of the search results, there isn't even a strong & clean signal to let end users tell Google "hey you are screwing this up badly."

In some ways I am inspired by Google's willingness to test and tweak, but in others I wonder if their new model for search is to care less about testing and hope that SEOs will highlight where Google is punting it bad. In that case, they just roped me into offering free advice. ;)

"To thine own self be true"

In a word?

Prescient!

10 links in a single AdWords ad unit!

Then more ads below it. Then a single organic listing with huge sublinks too. And unless you have a huge monitor at that point you are "below the fold."

Negative advertising in AdWords is not allowed. So long as you build enough brand signals & pay the Google toll booth, public relations issues & reputation issues won't be accessible to searchers unless they learn to skip over the first screen of search results.

While it is generally against Google's TOS for advertisers to double dip in AdWords (outside of the official prescribed oversize ad units highlighted above), Google is doing exactly that with their multitude of brands.

BeatThatQuote is back yet again.

The line between ads & content is getting blurry. Mighty blurry.

Is it time yet for a new slogan?

Google: the sales engine!

I recently got put in a test bucket for Google's new layout with a "search results" bar near the top of the page. Generally this impacts the search results in a couple ways:

• First off, it is a much better looking design. In the past when the search results would move up and down with Google Instant it really felt like a hack rather than something you would see on the leading internet company's main website. Now with the results fixed it feels much cleaner & much more well put together.
• The more stable basic layout of the SERP will allow Google to integrate yet more vertical data into it while making it still look & feel decent. Google may have localized search suggestions & the organic results for a significant period of time, but the combination of them with this new layout where the search results don't move feels much more cohesive.
• To get the white space right on the new layout Google shifted from offering 5 Instant suggestion to 4. The Google Instant results don't disappear unless you hit enter, but because the interface doesn't change & move there isn't as much need to click enter. The search experience feels more fluid.
• The horizontal line above the search results and the word "Search" in red in the upper left of the page is likely to pull some additional attention toward Google's vertical search features, helping Google to collect more feedback on them (and further use that user behavior to create a signal to drive further integration of the verticals into the regular organic search results).
• On the flip side of this, in the past the center column would move up & down while the right column would remain stationary, so I would expect this to slightly diminish right column ad clicks (that appeared at the top even when the organic results moved downward) while boosting center column clicks to offset that.
• In the past, when Google Instant would disappear from view, that would pull the center column organic results up a bit.
  • This always-on bar shifts the pixels above the first search result from about 123 to 184...so roughly 60 pixels downward.
  • As a baseline, a standard organic listing with no extensions is about 90 pixels tall, so this moves the search results down roughly 2/3 of a listing, which should drive more traffic to the top paid search ads & less to the organic results below them (offset by any diminished clicks on the right column ads).
  • This is a much cleaner way of taking advantage of white space than some of the cheesy & ugly-looking stuff they recently tested.

I tried to line up the results pretty closely on the new test results to show what they look like with Google Instant results showing & after you hit enter. Scroll over the below image to see how the result layout doesn't really change with Google Instant hidden or extended.

And here is an example image showing how the location is sometimes inserted directly into both the organic search results and the search suggestions.

Here is an image using Google's browser size tool to show how end users see the new search results. Note that in this example I used a keyword where Google has comparison/advisor ads, so in markets where they do not yet have those you would move all the organic results one spot up from what is shown below.

The "Best" of Big Media

Large publishers who lobbied Google hard for a ranking boost got it when Panda launched:

“A private understanding was reached between the OPA and Google,” an office assistant with e-mail evidence told Politically Illustrated. “The organization is responsible for coordinating legal and legislative matters that impact our members, and one of the issues was applying pressure to Google to get them to adjust their search algorithm to favor our members.”

At the same time, said "premium publishers" were backfilling their websites padding them out with auto-generated junk created by companies like Daylife, where some of the pages offer Mahalo-inspired 100% recycled content.

My suspicion is that Google did not care about the auto-generated "news" garbage for a number of reasons

• it helps subsidize the big media interests
• they don't want to hit big media & cause a backlash
• it is quite easy for Google to detect & demote whenever they want to
• it gives Google more flexibility going forward when deciding how to deal with issues (if everyone is a spammer then Google has more flexibility in deciding how to handle "spam" to maximize their returns.)

It is the exact same reason that Google says link buying is bad, while tolerating "sponsored features" sections on large newspapers:

Machine Generated Journalism

Where Google winds up in trouble on this front is when start ups that create machine generated content go mainstream. (Unless Google buys them, then it is more free content for Google!)

The leaders of Narrative Science emphasized that their technology would be primarily a low-cost tool for publications to expand and enrich coverage when editorial budgets are under pressure. The company, founded last year, has 20 customers so far. Several are still experimenting with the technology, and Stuart Frankel, the chief executive of Narrative Science, wouldn’t name them. They include newspaper chains seeking to offer automated summary articles for more extensive coverage of local youth sports and to generate articles about the quarterly financial results of local public companies.

Official sources using "automated journalism" is a perfect response to Google's brand-focused algorithms:

Last fall, the Big Ten Network began using Narrative Science for updates of football and basketball games. Those reports helped drive a surge in referrals to the Web site from Google’s search algorithm, which highly ranks new content on popular subjects, Mr. Calderon says. The network’s Web traffic for football games last season was 40 percent higher than in 2009.

How expensive cheap is that technology?

The above linked article states that "the cost is far less, by industry estimates, than the average cost per article of local online news ventures like AOL’s Patch or answer sites, like those run by Demand Media."

Once again, even the lowest paid humans are too expensive when compared against the cost of robots.

And the exposure earned by the machine-generated content will be much greater than Demand Media gets, since Demand Media was torched by the Panda update AND many of the sites using this "algorithmic journalism" were given a ranking boost by Google due to their brand strength.

The improved cost structure for firms employing "algorithmic journalism" will evoke Gresham's law. This starts off on niche market edges to legitimize the application, fund improvement of the technology & "extend journalism" but a couple years into the game a company that is about to go under bets the farm. When the strategy proves a winner for them, competing publishers either adopt the same or go under.

That is the future.

Across thousands of cities, millions of topics & billions of people.

Even More Corporate Boosts

Just because something is large does not mean it is great across the board. Businesses have strengths and weaknesses. Sure I do like love shopping on eBay for vintage video games, but does that mean I want to buy books from eBay? Nope.

Likewise, Google's friend of a friend approach to social misses the mark. Do I care that someone I exchanged emails with is a fan of an athlete who promotes his own highlight reels? No I do not.

In a world where machine generated journalism exists, I might LOVE one article from a publication while loathing auto-generated garbage published elsewhere on the same site.

Line Extension & "Merging Without Merging"

At Macworld in 2007 Eric Schmidt said "What I liked about the new device and the architecture of the Internet is you can merge without merging. Each company should do the absolutely best thing they can do every time, and I think he's shown that today."

If you don't have the ability to algorithmically generate content to test new markets then one of the best ways to "merge without merging" is to sell traffic to partners via an affiliate program.

Google has no problem promoting their own affiliate network, investing in other affiliate networks, or inserting themselves as the affiliate.

Google is also fine with Google scraping 3rd party data & creating a content farm that inserts themselves in the traffic stream. After they have damaged the ecosystem badly enough they can then buy out a 2nd or 3rd tier market player for pennies on the Dollar & integrate them into a Google product featured front & center. (It is not hard to be better than the rest of the market after you have sucked the profits out of the vertical & destroyed the business models of competitors).

Others don't have the ability to arbitrarily insert themselves into the traffic stream. They have to earn the exposure. But if other people want to play the affiliate game, they need to have "brand."

Affiliates Not Welcome in the Google AdWords Marketplace

At Affiliate Summit last year Google's Frederick Vallaeys basically stated that they appreciated the work of affiliates, but as the brands have moved in the independent affiliates have largely become unneeded duplication in the AdWords ad system. To quote him verbatim, "just an unnecessary step in the sales funnel."

In our free SEO tips we send new members I recommend setting up AdWords and adCenter accounts to test traffic streams, so that you have the data needed to know what keywords to target. But affiliates need not apply:

Hello Aaron Wall,

I just signed up for the Get $75 of Free AdWords with Google Adwords. After receiving an e-mail stating that I was to call an 877 number of Google Adwords, I was told in my phone call that affiliate marketing accounts were not accepted. I guess I confused by this statement. Is this in error? Or am I not understanding the Tip #3 for setting up an account for Google Adwords for promoting a website?

Thank you in advance for your time.
Sincerely,
Carole

The same Google which allows itself to shamefully carry a "get rich quick" AdSense category considers affiliate marketing unacceptable.

Non-AdSense Affiliates Classified as Doorway Pages, Not Welcome in the Organic Search Results?

The exact same thing is happening in the organic search results right now. Maybe not on your keywords & maybe not today, but if you are an affiliate, the trend is not your friend. ;)

I have heard recently from multiple friends that some of their affiliate sites were penalized for being doorway & bridge pages. At the same time, another friend showed me some BeatThatQuote affiliates ranking thin websites.

What is worse, is that in many instances, Google considers networks of similar sites to be spam. Yet at the same time the quickly growing Google Ventures is investing in companies like Whaleshark Media - a roll up currently consisting of 7 *exceptionally* similar websites in the same vertical.

Larger companies like BankRate can run a half-dozen credit card affiliate websites & an affiliate network. And they can create risk-adjusted yield by buying out smaller competitors, largely because Google won't penalize them based on the site being owned by a fortune 500. However the independent affiliate is forced to sell out early due to the risk that Google can arbitrarily decide they are a doorway site at anytime.

The absurd thing is that if independent webmasters don't include revenue generation in their website then they don't have the capital *required* to invest in brand & further improving their website. How do you compete against automated journalism when Google gives the automated content a ranking boost? And if you want to do higher quality than the machine generated content, how do you hire employees if you are not even allowed to monetize?

I suppose there is AdSense.

Even though AdSense publishers are Google's affiliates they are still welcome to participate in Google's ecosystem.

Risks to Small Businesses

Small businesses not only have to compete against algorithmic journalism, Google's algorithmic bias toward brands, arbitrary "doorway page" editorial judgements cast against them by engineers & significant algorithm changes, but they also have to deal with loopholes Google leaves in the system that allow them to be arbitrarily removed from the ecosystem.

Google showing you "closed in error" wouldn't be such a big deal if they didn't copy code, violate patents, deal in patents to spread their ecosystem, aggressively bundle & advertise, engage in price dumping, and behave in other anti-competitive ways to put their "incorrect facts" in front of billions of people.

The big issue Google is facing on the content quality front is the incentive structure. They have got that wrong for a long time now. They may think that these big changes are motivating people to improve quality, but realistically the lack of certainty is prohibiting investment in real quality while ramping investment in exploitation.

How can anyone invest deeply over the long term in a search ecosystem where Google...

• competes against advertisers in the ad auction
• hosts content that they automatically preferentially insert into their search results
• actively invests in companies that arbitrage their search results, and
• considered running an SEO firm?

Google would spin Performics out of DoubleClick, and sell it to holding firm Publicis.

Only one major force inside of Google hated the plan. Guess who? Larry Page.

According to our source, Larry tried to sell the rest of Google's executive team on keeping Performics.

"He wanted to see how those things work. He wanted to experiment."

The problem with that is that most honest economic innovation (eg: not just exploitation) comes from small businesses. Going into peak cheap oil where food riots are becoming more common & pensions are about to blow up, we need the kings of information to encourage innovation, rather than relying on doing whatever is easy & trusting established old leaders while retarding risk taking from (& investment in) start ups.

In some markets being successful means staying small, building deeper into a niche, and keep adding value until you have a strong position. However some ecommerce sites that were not associated with big brands were torched by the Panda update.

Betting on Brand

As Google has tilted their algorithm toward brand, some ecommerce companies that focused on winning relevant niches are now watering down their competitive advantages by betting the company on brand:

CSN Stores is today consolidating its 200+ shopping sites into a single ecommerce website under one brand: Wayfair.com.
...
So why the change to Wayfair.com? Primarily for obvious branding reasons: the company has long been spending a huge amount of money on marketing a lot of separate websites, and now they can focus on advertising just one.
...
Other reasons for the consolidation of the separate shopping site are search engine optimization – which was apparently much needed after Google’s recent Panda update – and the fresh ability to make recommendations to shoppers based on their collective purchase history.

But, as some brands abuse Google the same way the content farms did, is that a good bet? I don't think it is.

What is so Bad About Content Farms?

• low quality
• headline over-promises, content under-delivers
• anonymously written
• written by people who are often ignorant of what they are writing about
• add nothing new to the ecosystem, just a dumbed-down reshash of what already exists
• done cheaply & in bulk, in a factory-line styled format
• contains frequent spelling and grammatical errors
• primarily focused on pulling in traffic from search engines
• exists primarily to promote something else (ads or the above-the-fold ecommerce product listings)
• etc. etc. etc.

Such behavior is *not* unique to the sites that were branded as content farms & is quickly spreading across fortune 500 websites.

Big Brands Become Content Farms

A friend sent me an email which highlighted how a well-known brand was ordering thousands of pieces of "content" in bulk for their branded site.

Here is the email, with blurring to protect the guilty.

The only difference between the "content farms" and the branded sites engaging in content farming is the logo up in the top-left corner of the page. The business process from how the content is created, to who it is created by, to what they are paid to create it, to the interface it is ordered through, on to how it is published is exactly the same.

Many of the same authors who had some of their eHow "articles" deleted are now writing dozens of "articles" for fortune 500 websites.

When Panda happened & I saw corporate doorway pages (& recycled republished tweets) ranking I hinted that we could expect this problem. I thought it would start with parasitic hosting on branded sites & maybe a few opportunistic brand extensions.

Then I expected it would likely take a couple years to go mainstream.

But with the economy being so weak (and back in yet another undeclared recession, actually honestly never having left the last one) this shift only took 6 months to happen. At this point I expect it to spread quickly, especially as the economy gets worse. The above fortune 500 company is one that got a strong boost from Panda & as their downstream traffic from Google picks up over the next month or 2 you can expect many of their competitors to copy the strategy.

This isn't a US-only phenomena. A community member sent me the following, from another fortune 500 company.

Now that fortune 500s are doing almost everything that smaller players could do (but with more capital, more scale, more algorithmic immunity, requiring smaller players to link to them to be listed & in some cases while replacing humans with algorithms) AND get the Google brand boost the future is growing more uncertain for independent webmasters that lack brand, relationships, and community.

Big brands are basically pushed across the finish line while smaller webmasters must run uphill with a 80 pound backpack full of gear - in ice & snow, naked, while being shot at. What's worse, is that brands are now being bought, sold & licensed - just one more tool in the marketer's toolbox (presuming you have the cash).

disclaimer: I am not saying that all content farming is bad (I am fairly agnostic...if it works & people like it, then it works), but the above trend highlights the absurdity of Google's notion of whether something is spam based not on the offense, but rather who is doing it, especially as big brands just quietly turned into content farms.