« Unintended Consequences
Poor .info Domain Name Strategy, Afilias »

Robots.txt Overview, Generator, & Analyzer

Jul
21

We recently compiled a 6 page robots.txt primer and added a robots.txt generator and robots.txt analyzer to our suite of free SEO tools.

Subscribe to our blog via email or RSS to get more great posts like this one.

Posted in:

by Aaron Wall

Your Thoughts? [6]

July 21, 2008

comments

Login or Register to post comments.

Nice work Aaron, this will be a great help to people not usually used to using robots.txt.
The generator in particular is user friendly and helpful.

Thanks.

Diablos on July 21, 2008 03:51 PM

thanks, aaron. i didn't know that spaces in user-agent strings are a bad idea.

corey on July 21, 2008 04:09 PM

This post and the http://tools.seobook.com/robots-txt/ post are excellent - succinct and cover some very helpful elements.

Thanks Aaron.

Ben

yet another ben on July 23, 2008 03:26 PM

Glad you found them helpful Ben. :)

Aaron Wall on July 24, 2008 01:21 AM

great tool.It really helps. I tried it myself.Also went to see the robot.txt tutorial. There are lots of goodies here.Thanks a lot. It really helps especially newbies like me.

axedbydax on July 24, 2008 01:48 PM

Nice tool Aaron.

I thought our robots would pass your analyser and I was embarrased to find some faults! So thanks!

Mike

mikesed on August 19, 2008 10:44 PM

Login or Register to post comments.

Professional SEO Training

Improve your rankings, traffic, and profits today. The SEO Book training program offers you:

  • Over 100 training modules, covering topics like: keyword research, link building, site architecture, website monetization, pay per click ads, tracking results, and more.
  • An exclusive interactive community forum
  • Members only videos and tools
  • Additional bonuses - like data spreadsheets, and money saving tips
  • Every order comes risk free, and with the best selling SEO Book as a free bonus

Watch this video to learn more.

Improve your rankings today!