DMOZ Shut Down

Last August I wrote a blog post about how attention merchants were sucking the value out of online publishing. In it I noted how the Yahoo! Directory disappeared & how even DMOZ saw a sharp drop in traffic & rankings over the past few years.

The concept of a neutral web is dead. In its place is agenda-driven media.

  • Politically charged misinformed snippets.
  • Ads cloaked as content.
  • Public relations propaganda.
  • Mostly correct (but politically insensitive) articles being "fact checked" where a minor detail is disputed to label the entire piece as not credible.

As the tech oligarchs broadly defund publishing, the publishers still need to eat. Aggregate information quality declines to make the numbers work. Companies which see their ad revenues slide 20%, 30% or 40% year after year can't justify maintaining the labor-intensive yet unmonetized side projects.

There is Wikipedia, but it is not without bias & beyond the value expressed in the hidden bias most of the remaining value from it flows on through to the attention merchant / audience aggregation / content scraper platforms.

Last month DMOZ announced they were closing on March 14th without much fanfare. And on March 17th the directory went offline.

A number of people have pushed to preserve & archive the DMOZ data. Some existing DMOZ editors are planning on launching a new directory under a different name but as of the 17th DMOZ editors put up a copy at dmoztools.net. Jim Boykin scraped DMOZ & uploaded a copy here. A couple other versions of DMOZ have been published at OpenDirectoryProject.org & Freemoz.org.

DMOZ was not without criticism or controversy,

Although site policies suggest that an individual site should be submitted to only one category, as of October 2007, Topix.com, a news aggregation site operated by DMOZ founder Rich Skrenta, has more than 17,000 listings.

Early in the history of DMOZ, its staff gave representatives of selected companies, such as Rolling Stone or CNN, editing access in order to list individual pages from their websites. Links to individual CNN articles were added until 2004, but were entirely removed from the directory in January 2008 due to the content being outdated and not considered worth the effort to maintain.

but by-and-large it added value to the structure of the web.

As search has advanced (algorithmic evolution, economic power, influence over publishers, enhanced bundling of distribution & user tracking) general web directories haven't been able to keep pace. Ultimately the web is a web of links & pages rather than a web of sites. Many great sites span multiple categories. Every large quality site has some misinformation on it. Every well-known interactive site has some great user contributions & user generated spam on it. Search engines have better signals about what pages are important & which pages have maintained importance over time. As search engines have improved link filtering algorithms & better incorporated user tracking in rankings, broad-based manual web directories had no chance.

The web of pages vs web of sites concept can be easily observed in how some of the early successful content platforms have broken down their broad-based content portals into a variety of niche sites.

When links were (roughly) all that mattered, leveraging a website's link authority meant it was far more profitable for a large entity to keep publishing more content on the one main site. That is how eHow became the core of a multi-billion Dollar company.

Demand Media showed other publishers the way. And if the other existing sites were to stay competitive, they also had to water down content quality to make the numbers back out. The problem with this was the glut of content was lower ad rates. And the decline in ad rates was coupled with a shift away from a links-only view of search relevancy to a model based on weighting link profiles against user engagement metrics.

Websites with lots of links, lots of thin content & terrible engagement metrics were hit.

Kristen Moore, vp of marketing for Demand Media, explained what drove the most egregious aspects of eHow's editorial strategy: “There’s some not very bright people out there.”

eHow improved their site design, drastically reduced their ad density, removed millions of articles from their site, and waited. However nothing they did on that domain name was ever going to work. They dug too deep of a hole selling the growth story to pump a multi-billion Dollar valuation. And they generated so much animosity from journalists who felt overwork & underpaid that even when they did rank journalists would typically prefer to link to anything but them.

The flip side of that story is the newspaper chains, which rushed to partner with Demand Media to build eHow-inspired sections on their sites.

Brands which enjoy the Google brand subsidy are also quite hip to work with Demand Media, which breathes new life into once retired content: "Sometimes Demand will even dust off old content that’s been published but is no longer live and repurpose it for a brand."

As Facebook & Google grew more dominant in the online ad ecosystem they aggressively moved to suck in publisher content & shift advertiser spend onto their core properties. The rise of time spent on social sites only made it harder for websites to be sought out destination. Google also effectively cut off direct distribution by consolidating & de-monetizing the RSS reader space then shutting down a project they easily could have left run.

As the web got more competitive, bloggers & niche publications which were deeply specialized were able to steal marketshare in key verticals by leveraging a differentiated editorial opinion.

Even if they couldn't necessarily afford to build strong brands via advertising, they were worthy of a follow on some social media channels & perhaps an email subscription. And the best niche editorial remains worthy of a direct visit:

Everything about Techmeme and its lingering success seems to defy the contemporary wisdom of building a popular website. It publishes zero original reporting and is not a social network. It doesn’t have a mobile app or a newsletter or even much of a social presence beyond its Twitter account, which posts dry commodity news with zero flair for clickability.

As a work around to the Panda hits, sites like eHow are now becoming collections of niche-focused sites (Cuteness.com, Techwalla.com, Sapling.com, Leaf.tv, etc will join Livestrong.com & eHow.com). It appears to be working so far...

...but they may only be 1 Panda update away from finding out the new model isn't sustainable either.

About.com has done the same thing (TheSpruce.com, Verywell.com, Lifewire.com, TheBalance.com). Hundreds of millions of Dollars are riding on the hope that as the algorithms keep getting more granular they won't discover moving the content to niche brands wasn't enough.

As content moves around search engines with billions of Dollars in revenue can recalibrate rankings for each page & adjust rankings based on user experience. Did an influential "how to" guide become irrelevant after a software or hardware update? If so, they can see it didn't solve the user's problem and rank a more recent document which reflects the current software or hardware. Is a problem easy to solve with a short snippet of content? If so, that can get scraped into the search results.

Web directories which are built around sites rather than pages have no chance of competing against the billions of Dollars of monthly search ads & the full cycle user tracking search companies like Google & Bing can do with their integrated search engines, ad networks, web browsers & operating systems.

Arguably in most cases the idea of neutral-based publishing no longer works on the modern web. The shill gets exclusive stories. The political polemic gets automatic retweets from those who identify. The content which lacks agenda probably lacks the economics to pay for ads & buy distribution unless people can tell the creator loves what they do so much it influences them enough to repeatedly visit & perhaps pay for access.

BOTW Promo Code

BOTW emailed a coupon out good for July of 2011 for 15% off.

Promo Code: SAVE15
Valid through July 31

BOTW emailed a coupon out good for all of April 2011 which saves you 25% off your listing fees. If you do the one-time submission the price is $399, but this coupon will save you $100, making it a $299 one-time submission fee. If you prefer to spend less upfront, they also offer an annually recurring listing for only $112.

BOTW doesn't hand out promotional codes as often as some other directories do, so if you were considering listing your site there I recommend taking advantage of this promotion. BOTW is one of the top directories and I try to submit almost all of our websites there because it is a great source of a highly-trusted link. Their editorial review process tends to be a bit stricter than directories like the Yahoo! Directory, but once you are in you only have to pay the one-time fee.

Respect For DMOZ

Ah, DMOZ.

Whilst I was scanning through Barry's blog at SEL, I came across this post he mentioned entitled R-E-S-P-E-C-T for DMOZ.

A DMOZ editor complains "Everybody loves Google, everybody loves Wikipedia - so why doesn't everybody love DMOZ?"

My blog post would be rather long if I listed all the reasons why I think people don't love DMOZ, so I'll stick to fisking the contents of the editors post.

For those who don't know what DMOZ is, and that would be the vast majority of web users, DMOZ is a largely redundant internet directory that came about back when Yahoo! Directory was too slow at processing listing requests.

Webmasters familiar with DMOZ will appreciate the obvious irony, given that you can now get a Yahoo Directory listing in a couple of days, whereas DMOZ is a hit and miss affair, specializing mostly in "miss".

Let's take a look at the points raised:

Ask people how they search the web, and most will tell you what Google does well, what Wikipedia does well - and what DMOZ doesn't do well.

Ask people how they search the web, and you'll be hard pressed to find anyone who knows what DMOZ is, let alone outline it's faults.

The reasons for that will become obvious.

Perhaps you heard something on the news about the Somali pirates and want to learn more. .... Now what about Somalia in general? How did it get to this point? What's the history of the country, and what's going on with their government? How do you find answers to these questions without wasting a lot of time? This is where DMOZ shines....you can go to DMOZ's Somalia category. Start with Guides and Directories to find background information.

Students of philosophy will recognize this as an argument by selective observation. Cherry picking, in other words.

Well, it would have been had the author cherry picked an example that actually supported her argument. If you go into the recommended category, Africa/Somalia/, what will you find?

One listing.

For Wikipedia.

You just couldn't make this stuff up.

One could go into the sub categories, and whilst there are some useful listings there, there is nothing I couldn't find in greater detail in Google or Wikipedia. Helpfully, DMOZ frequently suggests I actually go to Wikipedia instead.

Who am I to argue?

Anyway, let's compare another search to see how well DMOZ does.

If I want to find out about SEO, I get presented with this category Web Design and Development: Promotion

Whilst there are some fine resources listed there, is this a useful reflection of SEO in 2009? Who are Majon International, for example? Why is Eric Ward seemingly the center of the SEO universe? Nothing against Eric, BTW.

Likewise, if I want to find out about New Zealand, it seems that "Hallidays Timber Limited" is very important, as they are the only site listed at the top level, as is - of all things - Usenet.

I could go on.

I'm sure there are great DMOZ categories, but like all things DMOZ, it's very much a hit and miss affair. Wikipedia and Google are a lot more "hit", and a lot less "miss", which is why people use them, and not DMOZ.

Sometimes they use 'relevant' keywords and page titles to game the system and achieve a higher ranking than they really deserve

Couldn't let that one go.

Apparently using keywords and page titles "games" the system. If they thought that were true, then why is DMOZ supposedly ""gaming the system" using titles and relevant keywords, too?

DMOZ certainly does irony well.

There's all sorts of relevant information to be found on the web, and the broader the topic the more useful DMOZ is.

Well, quite frankly, no it isn't.

If I want broad information, I use Wikipedia.

Wikipedia is infinitely more useful than DMOZ because it solved the problem DMOZ failed to do. It ran an editing system that anyone could contribute to easily, thus creating enormous value in terms of relevant, timely content. Updating and editorial was both transparent and immediate, which needs to happen, lest the information become outdated.

DMOZ chose to place editorial control in the hands of a small cabal of editors, and in so doing, made the directory opaque, unresponsive, and outdated.

That's the final irony.

The editorial policy of DMOZ killed DMOZ.

Official: General Web Directories Are Dead - JoeAnt is PageRank 3, More Aggressive Hand Editing by Google

[Update: After I made this post, Google engineers fixed JoeAnt's PageRank. Thanks Google!]

I just noticed that JoeAnt is now a PageRank 3. I have submitted hundreds of sites to hundreds of directories. For sites at the lower end of the quality spectrum (lets just call some of my experience academic) I simply would not submit them to JoeAnt, because I knew they would not list them. Many of those same lower quality sites were accepted in other directories like Business.com and the Yahoo! Directory.

While JoeAnt's editorial guidelines generally enforced higher quality than most directories, here are some of the things that may have hurt them when they were compared to the few general directories which have not had their PageRank scores edited:

  • smaller size
  • limited partnerships with other businesses
  • a lower price-point, which may have lowered the perceived value by the right customers and attracted some of the wrong customers
  • a name that sounded playful, rather than being business oriented
  • a smaller advertising budget
  • limited brand strength

I used to track 100s of general directories, and now I believe about 5 of them are showing their natural PageRank scores. The rest have been hand edited. Many of them were quite abusive and deserved to have their PageRank scores edited, but not JoeAnt, IMHO.

With so many of the clean link sources getting edited by Google, it is getting much harder for small businesses to compete with larger businesses for keywords on the commercial web unless they are ran by publicity whores. I am not sure if Google thinks that enhances information quality or helps mom and pop webmaster provide better services, but I guess we will see in a few years. The ironic part of it all is that if they force everyone to become marketing experts and PR agents to compete, then they undermine the long-term value of (and the need for) their paid search ads.

Why did a Google engineer chose to hand edit JoeAnt's PageRank score when similar directories with lower editorial standards like the Yahoo! Directory and Business.com did not get edited?

Business.com Directory Coupons - 20% Off Promotional Code

Update: The below coupon code no longer works, but we have found a couple new 2011 Business.com coupons. :)

In a recent interview Jessica Bowman offered the following Business.com coupon code: AVIVA, which is good for 20% off Business.com directory listings. Inclusion typically costs $299 a year, so with the coupon you would pay $239 for your submission.

Another nugget from the interview:

We currently have five editors and one editorial manager out of a total staff of nearly 100 employees, and all Business.com listings, whether pay-per-click Featured Listings or internet directory Standard Listings, are reviewed by a member of our Editorial Team.

Not bad margins on that business model if all the listings are wrapped in Google PPC ads and it only takes 6 editors to review it.

Say it Isn't So Joe(Ant)

I have always been a big fan of JoeAnt.com. So sad to see so many thin affiliate sites in the recently added box, and then get a second round of shock to see AdSense top and to the left on the individual category level pages. What leads webmasters to demote their hard build sites into a status of irrelevancy?

This is why I realized actively updating a directory list was fighting a lost cause. Over time they all go downhill. Automation and AdSense killed the model.

AdSense ads are Gay.

How would an editor feel after seeing all their listings dropped below a huge AdSense block with irrelevant gay quiz ads?

[Video] Submitting to Web Directories to Build Your Link Profile

Video Summary:

This video is 15 minutes 17 seconds long. Directories are easy sources of links, but links from lower quality web directories may not get indexed by some major search engines, may not carry much weight in Google, and may put your site in a bad community. This video covers evaluating the quality of a directory as a link source.

Resources Mentioned in This Video:

Examples of Quality Directories:

  • Curlie - formerly DMOZ, which was also referred to as the Open Directory Project or ODP. Free submission, but it may take a long time to get listed as it is ran by volunteers. The original DMOZ shut down on March 17, 2017.

  • Yahoo! Directory - $299 per year for commercial listings. Free for non-commercial listings. EDIT: dir.yahoo.com was shut down on December 27, 2014
  • Business.com - allows listings in multiple categories and you can point a few deep links at important pages on your site. EDIT: no longer an actual directory
  • BOTW
  • JoeAnt
  • Gimpsy

Find More Directories:

  • Drill down on in the ODP for categories of directories. ex: ODP Software Directories

  • Search for your keywords, related keywords, or your keywords + directory. Sites that rank might be decent link sources (depending on other quality signals).
  • Look at inbound links pointing to competing websites.
  • Use lists of directories. Please note that many directory lists are nepotistic (recommending their own directory as being the next best thing) or heavily influenced by advertising, and small niche high quality directories that are not on lists of 1000 cheesy directories are probably better than lists of directories commonly used to spam search engines. Each list will have some good directories and many junk ones. PageRank is nowhere near as important as other quality signals. Here are a few lists: Strongest Links, SEO Company, ISEDB, and Search Engine Guide.

Things I Should Have Mentioned That I Forgot:

  • The Google Directory is a clone of DMOZ, organized by PageRank. EDIT: directory.google.com was shut down in July of 2011.

  • If a directory does not charge a submission fee take extra effort to make sure your description and title are clean and proper (ie: factual and not keyword stuffed). Emulate other listings.
  • It is important to mix your anchor text and descriptions to make your link profile look natural. Emulate other listings in your category, and try to use your keywords in some of your link anchors if the directory will allow it.
  • Directories count more in verticals where the competition is weak and not well integrated into the web. If your competition is frequently mentioned in the active portions of the web on news sites, blogs, and social sites then you will need to be mentioned on there as well if you want to compete.
  • An established site well integrated into the web which already has a clean link profile can be more risky with what sites they get links from, whereas a new site or a site with limited authority would likely do better building links from the higher quality sources first, then maybe getting lower quality links later, only after their site has proven trustworthy.

Suggest Sites for DMOZ Inclusion

You can now submit your site to the Open Directory Project again. Submissions were down for a while.

DMOZ is Back Online

The day after the founder posts about the death of DMOZ they are back online with editing open, but new URL submissions are still down.

Thanks to Martjin.

Yahoo! Directory Registration & Search Engine Marketing

SEO Question: Is a Yahoo! Directory registration worth it? How do I know what directories are worthwhile? What directories should I submit to? Do you have any good site submission tips?

SEO Answer: If you have a business, and are serious about SEO, I generally would recommend submitting your site to the Yahoo! Directory. There are a lot of criteria to consider.

Your Site Name:

If your site name is MyKeywords.com make sure that your site lists your company as My Keywords. Do not run the words together in your logo, in your page title or text, or in the title of your directory submission when submitting your site to important directories. By separating the words in your site name you get better anchor text because the search engine sees the separate words in your links. Descriptive links from trusted editorial sources can be seen as a sign of quality.

If your keywords are not included in your site name, and it would be easy to alter your page title and logo, you may want to consider making some of your keywords as part of your logo design and official looking site name, so that you can get those words in your submission title.

If your keywords do not look like they are part of your official branded site name do not get too aggressive with keyword stuffing unless you are willing to risk a Yahoo! Directory editor editing your business name and potentially giving you less than ideal anchor text.

SERP Analysis:

When I buy quality links I am primarily buying them for either direct traffic or the effect they may have on my Google rankings. So the place to start analyzing category analysis is the search results.

Some sites will rank well based on being deceptive, creative, and spammy, but those rankings will quickly change over time, and those are not the ideal sites to pattern your link profile after. It is better to look at the top ranking related sites which you believe are credible sites that deserve the position.

For example, if you are a retailer of a product, but most of the higher quality top ranked sites in your category are manufacturers, it might make sense to dress up your site and write your directory listing description to make it look more like you are a manufacturer which also sells goods directly rather than just a retailer, that way you can submit your site to a category that lists you alongside.

The co-citation you are buying when you chose a category is a large part of the value of a directory listing.

Site Description:

Write your site description to help reinforce your category selection. Bias it toward making your site sound relevant for the category you want to be listed in. For example, if you want to be listed as a manufacturer and are submitting to a manufacturing category make sure your description says something like manufacturer of ...

Don't put too much hype in your site description. Look at other sites listed in your category to see how they are listed. The main goal of the description is to sell the category placement, and do differentiate your site from other sites listed in your category.

Directory Category Analysis:

There are a few main criteria when considering what directory category to submit your site to.

  1. the odds of you being rejected

  2. the co-citation value
  3. the global link authority of that category (ie: PageRank)
  4. the number of listings in your category

The odds of being rejected:
The odds of your site getting rejected from a paid directory for submitting to the wrong category are going to be quite low. For a free submissions or submissions to directories ran by editors, like DMOZ, getting the category selection correct is far more important than with a paid directory.

For a paid directory you probably want to submit to the best category which is reasonably relevant to your site. If they are too liberal with category placement the directory is probably of low quality and not trusted much, but even with high quality directories usually you can fudge it a bit. And, worst come to worst, they will typically list you in the category you belong listed in even if they do not give you the placement you desire most.

Co-citation value:
The co-citation you are buying is a large part of the value you are buying when you buy a directory listing. Consider the types of sites you want to be grouped with from the above SERP analysis section.

Yahoo! paginates the directory category listings pages by popularity, so if there are over 20 listings in your category and your site is new, you may want to spend the $50 to $300 a month it costs to sponsor your category, at least until your site's popularity increases and you are one of the top 20 results in your category.

Category link authority:
Some areas of a directory are over-represented within the overall directory structure, or may be well referenced by external resources. For example, Yahoo! lists the blogs category rather high in their overall category structure. Want another example of a directory category getting a bit of overexposure?

When Yahoo! created their own search engine, their official search engine guidelines linked to their SEO resources category. Hundreds of companies listed in the SEO services category, but there were only about a dozen listings in the SEO resources category.

Number of Links in Your Category:
If your category has less than 20 links then it is clear you will be listed next to the other listings. If your site is new and your category has more than 20 links then you may need to buy a category sponsorship to be featured at the top of the category to get the desirable co-citation.

Two other things to look at with the number of links in your category:

  • If you have a top sponsorship position in your category, or if you are bootstrapping it, and your brand is not that strong yet it may be cheaper to rank your category page than to rank your site off the start.

  • If your category has few links, or the other listings are not too relevant to your business, do not expect the Yahoo! Directory editors to want to list your site there.

Submitting to Other Directories?

I still think this post from April about web directories and SEO is a good primer for considering the quality of various directories, and how search engines may evaluate them.

A couple things I would add to that post:

  • Aged sites and/or sites with clean link profiles which are well trusted in Google are given a bit more leniency on what links may count and how many bad links they can get away with. If you have an aged trusted site you may want to dig a bit deeper for links, but for newer or untrusted sites you are best off just getting links from some of the higher quality directories.

  • If you are applying to become an editor at DMOZ, or other volunteer ran directories, make sure you start with a small category and sell topical passion more than you sell your commercial interests in the topic.
  • If you submit to a directory which allows multiple deep links with your listing, like Business.com, make sure you consider what pages will earn the most. For example, I have a 600+ page site where about 20% of the earnings come from one page. Getting your top earning pages a few more links can significantly increase their earning potential, but also note that if your deep idea is an uncompetitive niche there might be other links that you can get that will not leave such an obvious roadmap for competitors.
  • If your brand or core keywords could commonly be misspelled, like Client Side SEM vs Clientside SEM, you may want to submit your site to a couple average to lower quality directories with misspelled anchor text.

Pages