Get Your Blog Out of Google's Supplemental Result Hell

Blog Indexing Question: My ranking for my core keyword went up, but most of my site was recently put in Google's Supplemental Index, and I saw my income and traffic drop sharply. I have not built any links recently or made any changes to my site. How can I fix this and get my site top rankings again?

Answer: Google has been tightening down on their duplicate content filters. They have also been using PageRank scores to determine whether to index a page, if they should stick it in their supplemental results, and perhaps how strongly they should apply various filters (such as duplicate content filters).

Between slightly lower internal PageRank scores (minor issue) and increasingly aggressive duplicate content filters (major issue) and significant duplication from page to page on your site (major issue) much of your site is in Google's supplemental index.

Get Real Links:

Google's Matt Cutts stated:

In general, the best way I know of to move sites from more supplemental to normal is to get high-quality links (don’t bother to get low-quality links just for links’ sake).

Since you said you have not built links in a great deal of time and few people are talking about your site in the active parts of the web the key is to write more about things that people are talking about or would comment on. Great content helps build links. You have to keep blogging if you want to keep your mindshare up.

Others pointing more link equity at your site from external sources should help improve your PageRank scores. PageRank is a large part of what is used to determine if a page is of high enough quality to stay indexed (or put in the supplemental index), and how aggressively duplicate content filters (and other filters) should be applied against it.

If you get strong editorial deep links from other bloggers that should also help search engines crawl that portion of your site better to make up for any information architecture related issues that may be causing certain portions of your blog to be inadequately crawled.

Make Longer Posts:

Since your posts tend to only be a sentence or two long, most of the pages are rather similar to each other. You may want to post more text in each post, and turn comments on to have more unique text on each page.

Reduce Sitewide Repetitive Features:

You need to make your page titles and meta descriptions unique on each page.

You may also want to resort your code order to put unique content higher in the page content and have duplicated and sitewide template related issues occur later on.

Don't Link at Garbage:

Since your site has a rather low PageRank you may want to only list your blogroll on your home page instead of every page of your blog. Take out other parts of your site that heavily duplicate each other from page to page. Also consider removing your sitewide links to some of the unimportant pages on your site to flow more of your link equity throughout your site.

I would also recommend removing the tagging pages on your site as your site is already navigable via your categories, and the tags create low value noise pages that reduce your link equity distributed on the quality pages. I also think it is foolish to link at all those auto-generated Technorati pages...that wastes a lot of your link authority. I would also recommend not linking to some of the pages you don't want Google to index, such as those printer friendly pages. You may also want to block those printer friendly URLs using the Robots.txt protocol.

You have canonical URL issues which can be fixed. If www.mysite.com and mysite.com are showing different PageRank scores 301 redirect the less popular version of your URL to the more popular version.

You also have a few broken links on your site that could be fixed.

All these changes should facilitate better indexing of your blog posts.

Issues for Commercial Sites:

Many commercial sites (especially thin product database sites) also fall into the Google Supplemental index. The above examples all apply to those types of sites, but in addition you could consider the following

  • add an editorial element to your site to improve your sitewide authority score

  • enable customer feedback and reviews to get more unique content on your pages

Keep in mind that if you do not do any offline marketing or much marketing outside of search, your site is more prone to large swings from search related fluctuations than some other sites which have more brand equity and/or do other forms of marketing.

Published: February 12, 2007 by Aaron Wall in blogs Q & A

Comments

February 13, 2007 - 6:20pm

Thanks Aaron, this blog post (as well as the comments) have been very informative. The one question that still lasts with me though is that you don't seem to have your category pages disallowed in your robots.txt. With the issues that Shoemoney encountered and the potential to run into an internal duplicate content issue, wouldn't you want to have those pages taken out of the picture in Google's eyes?

digitalangel
February 13, 2007 - 7:13pm

If you want to view ONLY your supplemental results you can use this command site:www.yoursite.com *** -view

February 13, 2007 - 9:49pm

Thanks so much... it's surprising how helpful even the comments can be. I really grew a lot from this article.

February 13, 2007 - 10:07pm

Hi Tacimala
I don't think having category pages get indexed is a bad thing as long as there is a limited number of them. Plus that helps redistribute link authority a bit.

I think where there are issues is when there is like 25 different ways to navigate to the same content. Right now I have 2 internal navigation routes (date archive and category archive) and the post itself.

And even amongst those ideas there are ways to optimize, like using monthly archives is probably better than using weekly archives, for example.

Hi Nick
Yeah...the comments have been great, IMHO.

SnowSphere Magazine
May 15, 2007 - 4:39pm

I've recently noticed about one third of my pages have gone supplemental. I've spent hours reading blog posts about this, but I can't find a valid reason.

The site, which is an online travel/snow sports magazine is made up of completely unique content on each page. The site is well linked to, and many of the pages that have gone supplemental are deep linked to.

The content is not duplicated anywhere else on the web. For months the pages ranked in the first page of Google for their terms and now they've dropped out, for seemingly no reason. The home page is still in the main index, as are some of the newer pages, but it seems that some of the older ones are dropping out.

If anybody could offer any explanation, I'd love to hear it. The site is www.SnowSphere.com

thanks

February 14, 2007 - 2:49am

Just wanted to say thanks, for this and the last week or so's post - they've really given me food for thought as it seems a lot has changed without me noticing :)

Will look forwards to applying some of the ideas when i return from this break - particularly the prevention of dud pages getting indexed spoken about last week, that's a real shift in thinking for me but makes a lot of sense.

Many thanks Aaron, and to your insightful readers too.

rajkumar
August 6, 2007 - 2:49pm

Any one tell me method for showing Supplemental pages on site

March 26, 2007 - 5:51pm

Hi there,

Can anyone tell me please what's the exact cause of the supplemental problem for my site and how to fix this problem.

I will be really thankful for any help
Regards
bushib

June 27, 2007 - 10:08am

This is such a bad if our sites comes into google supplemental results, my websites was top in all major keywords on my website but i dont know what happened i was out from all keywords.
please guide me if any one know how to comes back in search results.
http://www.gordoniihoodia.net

DarrenC
February 17, 2007 - 7:56am

My categories are some of the highest traffic generating pages on my blog, and are ranking really well in Google.

I used the sitemap plugin for WP and that helped loads, but I have seen a lot of /feeds which are supplimental, and since these are no use really, I have blocked them via robots.txt.

March 11, 2007 - 10:22am

I put some code on robots.txt too, I hope this will make my blog will increased on visit and pagerank too ;)

February 17, 2007 - 7:28pm

I always thought that tagging blog entries for Technorati just allowed visitors another conduit through which to find my site...but you're saying it actually dilutes link popularity in the process?

My site is just a personal travel log that I'm sort of using to learn about SEO techniques and search engine behavior (as if the latter is actually possible). I recently went up another notch in PageRank, though I'm not quite sure why. I started using the Technorati tags a few months back. Should I discontinue this practice in future posts? Thanks!

prabhat
June 29, 2007 - 10:58am

This site is very informative but i have a problem, If i have created blog on blogger.com then how can i found robot.txt file for my blog to use this code. Can you help me?

User-agent: Googlebot
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$

February 19, 2007 - 12:21pm

After the PR update the hot Topic is Supplemental page and how to remove it. Well this is a great problem for blogspot blogs too. As the link of next 5 posts are given on label(that is the category) pages and the pages are empty so if google bot doesn't find any new content. So pages comes under SUPPLEMENTAL RESULTS. And also to remove content pages from supplemental. It is require that you have pr 3-5 atleast

raheel
February 19, 2007 - 4:37pm

Can someone please run by some link: queries to see if you're seeing anything like this with your page titles having extra words appended with Supplemental pages?

February 19, 2007 - 10:52pm

What do you mean when you say

"I also think it is foolish to link at all those auto-generated Technorati pages...that wastes a lot of your link authority."

Do you mean that it is bad to link to pages that are dynamically generated by let say ColdFusion programming that pulls from info with in a SQL dbase? I have been wondering if I need to change my site structure on the first two levels from dynamic to a fixed page. Before I do the work I want to know if it is worth it.

Thank you in advance.

February 20, 2007 - 1:10am

It is not bad to link to any specific type of page as long as you find it useful. The technology to deliver the page is not the issue I was concerned with. I was making that point more because bloggers give Technorati far more authority than Technorati gives back to bloggers, IMHO.

June 30, 2007 - 7:45am

Hi Prabhat
I think you probably set your blog to be private (not get indexed). I think you probably need to check the administrative settings and set it to public.

February 12, 2007 - 10:10am

Aaron a few months ago I noticed my blog had fallen into the supplemental index. A little research and I found the reason was Google was indexing the feeds and causing the real pages to go supplemental.

I'm not sure if it's the same issue you helped Shoemoney with, but I think it's related.

For me the solution was to use robots.txt and to take advantage of Googlebot's ability to handle wild card characters. Adding

User-agent: Googlebot
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$

did the trick and within a matter of days the posts started coming out of the supplemental index. I think it took a week for all of them to be back in the index.

I wrote a post about what I did and some other possible duplicate issues with WordPress blogs. I hope you don't mind if I link to it, but if you do feel free to remove the link. I've noticed other people having the issue and I thought some might find the information useful. Hope you don't mind.

WordPress Posts Going Supplemental

February 12, 2007 - 10:28am

Good advice Aaron. A few things I'll add:

First order of business, like you said, is getting high quality links. Having said that, if you got a site that's 70% supplemental, tweak internal link structure on pages that are either 1) most linked to and/or 2) still in the main index. Once a page goes supplemental, regular Googlebot will not come around to refresh its cache. So any changes you make to supplemental pages will not have any effect until Supplemental Googlebot comes around. I also believe links from supplemental pages are devalued, so even if Google refreshed its cache, its not going to have much effect, especially considering supp pages have low PageRanks to start with.

That means if your site is 99% supp except your home page, tweak your home page links first. The key is to keep most of the page's value from leaving the site - not by removing outbound links or nofollowing them but adding more links to internal pages.

A few easy plugins I use to increase internal links:
- Recently posted plugin
- Related posts plugin

I'd also go back and weave links into published blog posts. More internal links you have per page, the better, and links in articles IMO carry more weight than sidebar links. Google's link devaluation methods are getting more sophisticated as time goes by. If you have a post that's ranking #3 for a term, this is another way you can push it up a couple of notches.

If you have thousands of supplementals, I would try to focus on just a few pages at a time, instead of getting them all out at once. You want to avoid spreading your link juice too thin.

February 12, 2007 - 12:12pm

Good post Aaron, I just had a similar though in mind, and I happen to look up at my rss feed, and seen you made a new post, cause I was having similar issues my blog was getting put into the supplemental results, thanks man.

February 12, 2007 - 12:46pm

What I have done is to show more post on the front page. Normally I show, 20-25 posts. The result of this was that during the last Page Rank update, my blog went up to 6 from 0. And most of the inner pages are between 4-5.

February 12, 2007 - 3:31pm

I was hoping I could get some advice on how to change the description that shows up for my web site in Google.

Right now if you Google my company, New Demographic, you get the description:

"New Demographic provides workshops, seminars, and keynote speeches about mixed race identity, interracial relationships, and parenting mixed children."

This description is about 2 years old, and no longer describes what my company does. I think the reason it keeps showing up is because that's the description in dmoz.org for New Demographic.

I have my description metatag set with the following:

"An anti-racism training company that goes beyond diversity buzzwords to tackle the real issues behind race and racism."

And yet Google is still using the outdated dmoz.org description. I put in a request to dmoz.org weeks ago asking them to update the description, but nothing has changed.

Any ideas what I can do? Thanks in advance.

February 12, 2007 - 4:46pm

I am not a PHP guy, how do I remove the BlogRoll from all my pages except the home page?

Rafa
February 12, 2007 - 5:39pm

I have a blog facing the same problem. At first, the problem was all the feed URLs getting indexed but I fixed that , as well as fixing a duplicate content issue elsewhere.
Still most of my blog is supplemental, and link building opportunities in my industry (adult) are not much of an option.
I`m thinking that since most of my posts link to external site (galleries,etc) that might be considered "garbage" this might be another problem. Do you think nofollowing all external links is a good idea?
THanks!

February 12, 2007 - 5:58pm

Hi Carmen
look into NoODP

Hi Dave
it really depends on your template, but I don't want to hand out php advice because you are probably better at it than I am

Hi Rafa
I would probably redirect links rather than use nofollow

February 12, 2007 - 7:58pm

So basically we have to look for anything causing duplicate content issues on our blogs: A Blog usually makes a duplicate of your post, how do we handle that?

February 12, 2007 - 9:12pm

Does blocking your feeds from Googlebot impact google's blog search?

February 12, 2007 - 11:27pm

Dave, Aaron is right in that it can depend on the WP theme you have installed, but I think the code you are looking for is:

<?php get_links_list(); ?>

The get_links)list(); is a WP template tag and will display all the links (including your blogroll) you have set through your admin interface under the links tab.

If you change the line above to:

<?php if(is_home()){ get_links_list();} ?>

it should only display those links on the hone page. The if(is_home()) checks to see if the page in question is the main page of your blog.

The default location for the above code will be in your sidebar.php file, but it can be different depending on the theme you have installed for your blog.

If you're not experienced in editing php files make a backup of the file before you edit it just in case.

Hope that helps.

February 12, 2007 - 11:31pm

Oops. It looks like the two lines of code I posted didn't work and got interpreted as php. Aaron I hope I didn't cause any problems, though I don't think so.

Let me try again to write the code

You want to change

<?php get_links_list(); ?>

to

<?php if(is_home()){ get_links_list();} ?>

I hope my posting works this time. Sorry about that.

February 12, 2007 - 11:42pm

How can you tell that pages are in the supplemental index? I can't tell the difference in a Google search.

ziemek
February 13, 2007 - 5:48am

Andrew,

to tell if a page is in the supplemental index type in site:www.yourwebsite.com

and you will get a list of indexed pages by google
those that have the supplemental pages will be specified by "- Supplemental Result " at the end of the page url

February 13, 2007 - 5:56pm

Thanks for advices. The only thing I would be happy to know is when is this going to happen. Is there anyway to get a domain unflagged as crapped ? I know there is site reinclusion request but I know this can take some time and that you need to state that you where spamming. What if it wasnt the case. Have any tricks for that beside linking linking and hopping ?

Dan Horton
July 4, 2007 - 6:10pm

This supplemental thing is mighty confusing especially for an SEO trainee. I have repeat titles etc on my blog which iv'e just started, doesn't seem to have a great impact on it so far.

Im learning the ways of the force with DaveN and he guided me to your book so thank you.. it's great. Bit difficult to know which unique perspective to come from but im looking at SEO in the yorkshire region.
danhorton.co.uk

Dan

Add new comment

(If you're a human, don't change the following field)
Your first name.
(If you're a human, don't change the following field)
Your first name.
(If you're a human, don't change the following field)
Your first name.