Tracking the Growth of Competing Sites

Question: How do I track the progress of competing sites over time? How do I know what keywords my competitors are ranking for, and which ones they are improving on?

Answer: There are a way variety of means to track competing sites.

Online Ranking Checkers

Online tools such as Digital Point's keyword ranking tool, the ShoeMoney SERP tool, the our rank checker, or the new tool at ZoomRank show where keywords rank. The Digital Point tool also shows you where a keyword ranks over time, but the problems are who wants to check a lot of these keywords one at a time, and where do you get the opening list of keywords?

Competitive Research Tools

There are numerous organic search competitive research tools on the market. SpyFu is a paid tool which offers limited free data, and SEODigger and URLTrends are free tools which shows you keywords that a site ranks for. AdGooroo also takes a look at similar data, with more focus on paid search. If you can afford to spend $10,000+ you may also want to consider trying HitWise.

Keyword Research Tools

After the competitive keyword research tools you may also want to look at the keyword tools promoted by the search engines (Google, Yahoo, MSN) and companies like Wordtracker and Keyword Discovery. The Google Traffic Estimator will tell you what terms Google thinks are most valuable, and you can also check keywords in bulk and track them over time by buying software like Agent Web Ranking.

Don't forget to also consider macroeconomic factors and seasonal traffic trends.

Look at the Competitor's Website

Looking for a few more keywords that the competitor may have found but the market missed? Beyond those competitive research tools the easiest spots to look for keywords on competing sites are

  • the internal navigation of competing sites (especially look for pages that are bizarrely over-represented in their navigation)

  • a competitor's homepage page title and headings (you can see the page titles with Xenu and the last two data points with OptiSpider)
  • abnormal patterns in their inbound anchor text (use a tool like Backlink Analyzer)
  • if they participate in AdSense and allow site targeting, buy AdSense ads targeting their site to dig into their traffic stats

Look at Their Reach

As far as general traffic trends for a site go, Compete.com, Quantcast, and Alexa all give snapshots of site traffic trends. That data tends to be rough though...especially for small sites. The two big ways to track site growth from a market penatration and search representation standpoint are to look for changes in the rate of citation by using the following data points and tools

and look for the rate of the growth of the site's content, reach, and trust using the following metrics

  • number of RSS subscribers (use Bloglines subscribers as a sample estimate if they don't have any other numbers public) and/or number of comments on blog posts for sites with feeds

  • how often you see well read channels mentioning the competitor
  • how often you see (search, contextual, or affiliate) ads for the competitor
  • number of pages on the site indexed in Google
  • the ratio of the pages that are in the regular index vs Google's supplemental index (having most of your pages in the primary index is a good thing)

Lowering Your Search Rankings for a Keyword That is Getting You in Legal Trouble

Question: A client of mine wanted to target high traffic terms in his industry. His industry was not very competitive, and some of the target terms were competing trademarks. We rank top 5 for them in Google, and now legal troubles are occurring. We removed all references to the competing trademark on our site, but still have some rogue inbound links that we can't get removed that have the anchor text which targets the competing mark. What should we do?

Answer: There are four options which can help get you out of this situation.

Work With the Competition

If you have a nepotistic relationship with the competitor and recommend them then perhaps you can both be strengthened as category leaders.

Is the Lawsuit Cheap Marketing?

Anything involving Google and search is still a ripe field for media exposure. If you think your chances of winning are good enough, and the potential return is much larger than the risk of losing consider letting them sue you. It is probably not your fault that Google ranked you, but also seek legal advice outside of reading this post... I am not a lawyer.

Tank Your Rankings for that Keyword

If the links point at a page other than the homepage, consider removing the URL from your site, and then use the Google URL removal tool when the 404 error shows.

If you bought those links cash usually works to help remove them. Use their on site contact information and the email in their whois. Call the number in the whois data. Pay them to take down the links.

Improve the Rankings of Competing Pages

The tips offered in my search engine reputation management post work here as well. Follow the tips in that post to help make other competing pages rank better.

You can also work on improving the rankings of other competing pages while lowering your rankings. Feel free to push a couple strong pages if you are just trying to end the confrontation, but if you feel they are dirty you may also want to help surface some ugly news that was ranking on page 3. Do that and they may care less about your rankings, and shift their focus to those other sites.

Why I Don't Like .biz Domain Names

Question: I have a .biz website that ranks well in some of the major search engines for a few keywords, but does not rank as well as I would like for many other keywords. Should I consider switching to a .com domain name?

Answer: In the long run I think it is worth moving away from .biz if you are creating a real long-term business. The web was created to share information, and businesses are generally viewed less admirably than the individuals that work in them. As long as relevancy algorithms are based largely on links, then a .biz extension could hurt your exposure in most fields.

Most web companies that control large traffic flow have taught their userbase (and those who they trust to vote) that commercial = spam. To put this in perspective, some journalists write entire articles about businesses and then do not link to the businesses because they feel that doing so would be too promotional. In competitive fields sometimes only a few links separate a business that is getting 5% of the traffic or 30% of the traffic a search engine offers.

If you are starting your business on a .biz it doesn't matter if you have the best information in the world...the bias of .biz (and toward business in general) is going to hurt your exposure, making your business less efficient and increasing your business cost. The small businesses that are best sustainable on the web are those that function as businesses yet have the feel of non-commercial sites (and acquire links as though they are non-commercial sites).

Small Niche Keyword Research Modifiers

SEO Question: My site is focused on a small niche and I can't find any related keywords to write content about. How do I optimize my content? What should I do?

Answer: Track what you are ranking for and look for meaningful patterns and descriptive related keywords in that data.

If few people are searching for something, but you are in a growing field, then that might be great market timing which helps makes you a market leader. Make sure you create things like an industry glossary, and actively participate in communities related to your topic so you have top of mind awareness to people in related fields. Research why people are linking at competitors, and what is missing in the marketplace.

If you are already in an established field, then use keyword tools to look for broader keywords or parallel keywords. Observe what descriptive modifiers people are using for higher volume keywords. The odds are pretty good people will use those same type of modifiers when searching for your topic. You can also use this cast a wider net concept by bidding for related broader keywords on AdWords and track the referrals.

Getting Out of a Bad Community

SEO Question: When I did a related search on Google it said that my site was related to some gambling sites. How do I help my site no longer be related to these sites?

Answer: It sounds like you may have a spammy link profile that needs cleaned.

Site Content Cleanliness Check:

Do a site:mysite.com search in Google for your domain, while also searching for a few offensive words to ensure nobody has added spammy pages and / or spammy outbound links to your site.

Outbound Link Profile:

The first thing you want to do is look at who you are linking to. If you are linking to any off topic or spammy stuff get rid of those links. If you are not linking out to any industry authorities link out to a few authority sites in your industry to help search engines understand what sites your site is related to.

Clean Up Your Inbound Link Profile:

Stop renting any links that associate your site with bad sites (ie: have co-citation with spam).

The next thing you may want to do is consider looking through your backlinks in Yahoo! while searching for offensive words (ie: casino, poker, gambling) to see if you have any dated offensive links that you may need to request be removed.

Build Up Your Legitimate Inbound Link Profile:

While you are taking down the bad links also look to make sure you have plenty of good links that help put your site in the community it belongs in. Submit your site to a few of the major trusted directories, get other locally or topically relevant authoritative links (industry magazines / bloggers / trade associations), then use tools like hubfinder, touchgraph, MyBlogLog, Alexa related sites, the Google Directory, and other similar sites that help organize link profiles and/or visitor profiles to define communities.

When possible get links from those quality sources. If it seems like your site could not get many links from those quality sources then do what you have to in order to be considered comment worthy and link worthy. If you don't know how to do that, do the following:

Selling SEO Services on a Performance Basis

SEO Question: Some prospective customers have offered to pay me on a performance basis. Should I consider providing search engine marketing services for these types of clients?

Answer: There are many ways to structure these sorts of deals, but generally I would probably avoid most of the offers because a lack of willingness to pay until results are achieved is often an indication of a lack of trust.

Trust and Toxic Customers:

If a customer does not trust you enough to pay you until after you show results they may not trust you to access their site or implement your suggested changes. This lack of willingness to consider optimization elements away from search is the single common problem associated with most toxic customers.

It is easier to push your own good idea than to push a bad idea owned by a person who is rigid and hard to work with.

Selling All Traffic as PPC:

If they will not give you access to the site you can still give suggestions that you hope they will implement, and you can still build links. Selling traffic as PPC minimizes the upfront commitment on both sides, and allows the SEO to still get paid even if their site can't convert. These deals can be structured in many flexible ways:

  • set minimum and / or maximum spend caps

  • set contract term length and opt out clauses
  • set price as being flat rate per click, or allow both partners to adjust rates when it makes sense
  • do not charge for brand related searches
  • specify what traffic sources are valid (ie: pay for Google searches and traffic from other engines, but do not pay heavily for a Digg homepage story)

As an SEO, selling traffic on a PPC basis protects you from conversion errors on their site, and may make them more likely to listen to your conversion advice. A good SEO should be able to sell traffic for less than comparable PPC traffic.

If you set your price point high enough you can start off by selling them relevant PPC traffic and then easing off the PPC spend as your organic optimization gains traction. Some SEO companies may set up third party sites to drive traffic, which protect them if a client decides to cut their budget.

I believe Barry Lloyd was one of the first to sell SEO services on a per click basis, but I believe he has since moved on to selling PPC management software.

Pay Per Ranking:

Some clients think they need a few certain trophy phrases even if that is not the case. Some deals with a single trophy phrase or wide related keyword nets can also be sold on a pay per ranking deal, with so much being paid for ranking in different engines for different keyword phrases. Make sure both parties have the same idea as to what the goals are and how long a page must rank to receive payment, and when the payments are due.

Affiliate Websites:

If a merchant reveals an appealing vertical, but does not want to pay enough to make it worth your time, consider setting up an affiliate website marketing related offers. This allows you to chose whatever affiliate programs pay the most, while ensuring you get paid recurring revenues even while you are not actively promoting your site.

Sites Worth Partnering on a Pay for Performance Basis:

  • Related Sites: If you already have a related site that can drive significant traffic a partnership makes sense, but probably as an affiliate rather than an SEO. But if you help them on their site it should be easy to provide value if you already know their site well.

  • Large Brands: If you see structural errors that are holding back large brands AND they are willing to act on your advice they may see significant upside so will the SEO, but most large brands will be adverse to these type of deals.
  • Small niche players: (perhaps even local niche sites) that take limited time to work on are also nice to work with, but be careful not to do too many projects like these or they can weigh you down during shake ups. Algorithmic shake ups are periods of opportunity if you have free time to roam, but may be periods of hurt if you have too many clients.

Sites Not Worth Partnering On:

  • New Sites in Competitive Fields: If you have to go through all the work to build up a new site you are probably better off building up your own site than building up someone else's site from scratch. The one time these types of deals make sense is if you really believe in the upside potential of an idea and can get an early equity stake.

  • Thin Content Sites: If their site is already doing exceptionally well, but has serious issues and is just waiting to get nuked then they may blame you for the fault that was just waiting to happen. Stick clear of thin content sites and sites that are designed more for bots than for humans.

Keyword Research for Niche Terms

SEO Question: I am trying to do keyword research for a client, and he is focused on a niche phrase that does not show up on any of the major keyword research tools. I was wondering if there was an accurate way to estimate search volume for these long tail keywords.

Answer: In this case, almost any sampling method you can think of is going to be wildly inaccurate.

If something does not get much search volume the easiest way to estimate search volume is to pull out your credit card and run a Google AdWords campaign targeting the keyword. Make sure your targeting is for search only and that your daily spend limit and bid prices are high enough that your keyword is showing for most (if not all) searches. If your ads are targeted to broad match (instead of phrase or exact) Google will also show your ads for many related keywords based on your keyword list. It also helps if you have an AdWords account that is aged and trusted so that they give you significant exposure right away.

I think a more practical solution than to look for exact keyword volume for niche terms is to use the keyword research tools to show their low search frequency, then use the same tools to show what words are important, and for parallel keywords to come up with rough estimates for search volumes for the relevant basket of related keywords. Then use a tool like my keyword list generator to come up with a relevant list of keywords to bid on.

After you get some market feedback from that PPC account, use your server logs and ad campaign stats to track short and long keywords your ads were relevant for, adjust your page copy, internal anchor text, and inbound link anchor text to help focus on the most important phrases, while also covering related phrases in a way that avoids keyword cannibalization.

A few other ways you can come up with relevant keyword phrases are:

  • track what you are ranking for in organic search results

  • look at the navigational structures, page titles, and page copy from competing sites
  • see what keywords Google recommends based on your URL
  • see what AdSense ads Google would target to your pages. Look at their ad copy, copy from their landing pages, and copy from other relevant parts of their website.
  • On rare occassions competitors may make their stats public for one reason or another.
  • You can also pose as an interested potential site buyer to acquire statistics.

There are also services such as HitTail that aim to help you extract other useful keywords based on the keywords you are already ranking for, and private tools such as HitWise and KeyCompete may show you a few keywords that public tools do not.

Do Pre-made Stores or Pre-made Online Merchant Websites Have Any Value?

Question: I recently purchased a website at mydomain.somecompany.com as a store to sell somecompany.com's products and was wondering if the site had any value, and how I could do SEO on the site if I couldn't change the content?

Answer: Many companies sell the right to resell their products online, but most of them that require you to use their domain or subdomain for your stores are probably selling junk. Truth be told, I have got similar questions from other people who told me that customer support at some of these firms charged for site customization, and even started off many question sequences with how much room is left on your credit card.

The idea of paying for the ability to sell someone else's stuff online is a bit absurd. Amazon, eBay, Google, Yahoo! and thousands of other sites all have affiliate programs. Some of the better opportunities with product catalogs also have fairly open API programs that make it easy to integrate their data into your site.

There are many problems with trying to market a site that you can not touch

  • if you can't change a site then it is hard to create something remarkable that people would want to link at or share with friends

  • if someone's marketing is so closed off that they don't even allow you to change anything then their marketing is probably missing out on many other great feedback points as well
  • if you can't change a site then it is hard to get past duplicate content filters (good search engines will only want to rank 1 of these subdomains for any query which means that the rest of the affiliates are out of luck)
  • It is important to build up your own doman name if you are serious about building something of value. If you are stuck on someone else's subdomain, then if they ever dislike you or if their business fails so does yours.

Ideas and pieces of software are recycled all the time. This site uses Apache, MySQL, and MovableType on the back end, but the front end design is unique and these words are things I typed. There is nothing wrong with using pieces, but if someone wants you to use their entire system then they are probably going to hold you back from your full potential.

People buy packaged solutions for the same reason they buy diet pills and exercise programs that work while they sleep:

Why do you think there are so many diet pill, fitness equipment, workout 10 minutes a day, crap commercials selling billions of dollars worth of solutions that don't work? The reason is people want to believe they work, they are unwilling to do the things actually required to successfully loose weight and be in shape (diet AND excersize regularly at least 30mins a day)

So instead, they buy the marketing because they'd rather lie to themselves and believe a pill / fancy ab equipment and 10 minutes will work.

Instead of starting from some boxed in, closed off opportunity someone else offers you, I think you would be better off to start with something you are interested in and go from there. It costs under $200 a year to register a domain name, host it, install Wordpress and start writing. And if $200 is more than you have then Blogger.com allows you to custom map a Blogger blog to a URL of your choice for only $10.

How Do I Submit My Site to Google and other Search Engines?

SEO Question: Can you recommend a product/service that will enable me to get my site listed with all the search engines? I have the site hosted with GoDadddy, they have a service for $30 that supposedly gets you registered with multiple search engines. Does this sound like a good idea? Is there a better way to do this?

SEO Answer: Many web hosts operate at almost no profit margin, and provide some cheesy no value submission service to thicken up their margins. It might be a bit much to call these search engine submission services a complete fraud, but I would classify most hosting company automatic website submission offerings as having no real value to webmasters.

If you want to get indexed by the major search engines get links from quality sources, like trusted web directories, well known blogs (search Technorati or Google Blogsearch), and other sites that are relevant to your offering. Search engines follow people and trust what other people already trust. Links act as votes of trust, so building quality links not only helps get your site indexed fast, but will also allow you to achieve a top ranking quicker.

Get Your Blog Out of Google's Supplemental Result Hell

Blog Indexing Question: My ranking for my core keyword went up, but most of my site was recently put in Google's Supplemental Index, and I saw my income and traffic drop sharply. I have not built any links recently or made any changes to my site. How can I fix this and get my site top rankings again?

Answer: Google has been tightening down on their duplicate content filters. They have also been using PageRank scores to determine whether to index a page, if they should stick it in their supplemental results, and perhaps how strongly they should apply various filters (such as duplicate content filters).

Between slightly lower internal PageRank scores (minor issue) and increasingly aggressive duplicate content filters (major issue) and significant duplication from page to page on your site (major issue) much of your site is in Google's supplemental index.

Get Real Links:

Google's Matt Cutts stated:

In general, the best way I know of to move sites from more supplemental to normal is to get high-quality links (don’t bother to get low-quality links just for links’ sake).

Since you said you have not built links in a great deal of time and few people are talking about your site in the active parts of the web the key is to write more about things that people are talking about or would comment on. Great content helps build links. You have to keep blogging if you want to keep your mindshare up.

Others pointing more link equity at your site from external sources should help improve your PageRank scores. PageRank is a large part of what is used to determine if a page is of high enough quality to stay indexed (or put in the supplemental index), and how aggressively duplicate content filters (and other filters) should be applied against it.

If you get strong editorial deep links from other bloggers that should also help search engines crawl that portion of your site better to make up for any information architecture related issues that may be causing certain portions of your blog to be inadequately crawled.

Make Longer Posts:

Since your posts tend to only be a sentence or two long, most of the pages are rather similar to each other. You may want to post more text in each post, and turn comments on to have more unique text on each page.

Reduce Sitewide Repetitive Features:

You need to make your page titles and meta descriptions unique on each page.

You may also want to resort your code order to put unique content higher in the page content and have duplicated and sitewide template related issues occur later on.

Don't Link at Garbage:

Since your site has a rather low PageRank you may want to only list your blogroll on your home page instead of every page of your blog. Take out other parts of your site that heavily duplicate each other from page to page. Also consider removing your sitewide links to some of the unimportant pages on your site to flow more of your link equity throughout your site.

I would also recommend removing the tagging pages on your site as your site is already navigable via your categories, and the tags create low value noise pages that reduce your link equity distributed on the quality pages. I also think it is foolish to link at all those auto-generated Technorati pages...that wastes a lot of your link authority. I would also recommend not linking to some of the pages you don't want Google to index, such as those printer friendly pages. You may also want to block those printer friendly URLs using the Robots.txt protocol.

You have canonical URL issues which can be fixed. If www.mysite.com and mysite.com are showing different PageRank scores 301 redirect the less popular version of your URL to the more popular version.

You also have a few broken links on your site that could be fixed.

All these changes should facilitate better indexing of your blog posts.

Issues for Commercial Sites:

Many commercial sites (especially thin product database sites) also fall into the Google Supplemental index. The above examples all apply to those types of sites, but in addition you could consider the following

  • add an editorial element to your site to improve your sitewide authority score

  • enable customer feedback and reviews to get more unique content on your pages

Keep in mind that if you do not do any offline marketing or much marketing outside of search, your site is more prone to large swings from search related fluctuations than some other sites which have more brand equity and/or do other forms of marketing.

Pages