Cache Crawler... New SEO Tool Idea

Jul 4th

Feature requests... feature requests.

So recently Waxy held a contest for creating a tool to visually see the history of a Wikipedia page. The winning programmer got like $200, which in terms of SEO spend is not much money for a tool that many people could use.

While search engine APIs may have limited longterm value I am hoping that they last a while and are not too evil with their TOS. If they are of course more people will just scrape the data. They may try to block scraping, but tools and spam techniques evolve with the engines.

I am thinking our rate will usually be above $200, and I don't want to make the price something where people place the lowest bid. We will just come up with a price and then throw the idea out there and see what comes back. I can pay for the tools, or if it really takes off and others want to support some of the ideas they can help donate too.

If people would be willing to program decent SEO tools for a decent price I could probably think up at least 50 tools to be programmed.

With that in mind, I think the SEO community should have a mass cache tool, to know when stuff was cached. Here are the desired features (so far):

  • works with the Google API

  • checks Google's cache feature (cache:www.example.com) for cache date.
  • the tool should have three main modes it functions in:
    1. allows bulk upload of a list of pages and returns the cache date of each page, also informing users of what pages are not cached.

    2. allows you to enter a URL and return the cache date from the first 1,000 URLs.
    3. allows you to enter a URL and returns which pages are freshly cached. also allows you to set a fresh date to return URLs spidered since then when you do a bulk upload of URLs.
  • If it is possible return if the page is in the supplemental results.
  • The data should be easily exportable to CSV for further manipulation.

Is this a good tool idea? Bad idea? Have any feedback on how to make the idea better? Ideas on how to market it? Suggested award amount? Did I use too many vowels in the post? please give feedback :)

Published: July 4, 2005

New to the site? Join for Free and get over $300 of free SEO software.

Once you set up your free account you can comment on our blog, and you are eligible to receive our search engine success SEO newsletter.

Already have an account? Login to share your opinions.

Comments

July 4, 2005 - 7:53pm

You used just the perfect amount of vowels in my opinion. ;) A cache tool would be great... it would be great to track that over time, as well, for not only your own sites, but sites that you may buy links from.

October 28, 2009 - 12:27pm

Weird I am posting comment to a 4-year old post but is there any tool as you mentioned in this post, or any tool where we can upload list of pages to check if they show within Google Cache or not. At least MajesticSEO should have this tool IMO; lots of links they show don't show within Google's Cache.

October 29, 2009 - 6:50am

well the google api is junk now. and such tools could easily be made for personal use...but if they were public they would probably get blocked by Google.

I believe you can get cache date in seo for firefox though. so set the results to 100 per page (add &num=100 to the search page address string) and run it with SEO for Firefox turned on and then search for a specific site (using the site: function).

and then you could export that data using the built in export to CSV option

note that you would want to use a time delay on pulling in that data point and Google may block you temporarily from accessing their cache if you pull in too much data too fast.

New to the site? Join for Free and get over $300 of free SEO software.

Once you set up your free account you can comment on our blog, and you are eligible to receive our search engine success SEO newsletter.

Already have an account? Login to share your opinions.

  • Over 100 training modules, covering topics like: keyword research, link building, site architecture, website monetization, pay per click ads, tracking results, and more.
  • An exclusive interactive community forum
  • Members only videos and tools
  • Additional bonuses - like data spreadsheets, and money saving tips
We love our customers, but more importantly

Our customers love us!






    Email Address
    Pick a Username
    Yes, please send me "7 Days to SEO Success" mini-course (a $57 value) for free.

    Learn More

    We value your privacy. We will not rent or sell your email address.