"Borrowing" Content: How The Little Guy Can Fight Back

The problem: big publishers "borrowing" stories from smaller publishers, redrafting them, and republishing them. Because the bigger publisher has greater domain authority, "their" story achieves higher rank.

Can you pick "who made who" in the following examples?

Of course, the publishers of these specific examples may not of been aware of each others existence. Great minds can think alike. But there are so many examples of coincidence out there, one suspects it isn't all purely a matter of chance.

Whilst borrowing of ideas is nothing new, if you're a publisher, content borrowing can wreak havoc with your seo strategies. The big and powerful sites dominate, and the little guy often gets relegated. Google's linking algorithms reward the already rich, and make them richer.

The sad reality is that whilst the web started out with the intention of being a democracy of information, it now closely resembles the power structures of the offline world. By the time you read this article, it has very likely been reproduced without attribution.

Here are a few ideas on how the little guy can fight back.

1. The Power Of Relationships/Distribution

It is said that business can be boiled down to two essential elements: to sell something for more than it costs to produce, and the ability to nurture relationships.

If you're a publisher, you can leverage the relationship you have with your readers in order to protect you from "the borrowers". Once you readers, and indirect competitors, are aware of your work, it becomes harder for your competitors to talk to that same market, using your ideas.

Think about what you can do for your readers to instill a sense of loyalty. Give them something of value. Make them feel indebted to you. Give people a stake in your success.

Consider allowing people to republish your content under certain conditions. i.e. when you have sufficient page rank, allow others to copy parts of your work, so long as they link back to the original. Such a policy might turn those who would ordinarily steal from you into allies who supply free link juice.

2. Branding

Try to make your content an identifiable part of your brand.

For example, part of your brand might be your a stylistic approach to writing. It would be very difficult to directly rip off Dave Barry, because he writes himself into his articles.

Typically, the more generic a piece is, the easier it is to borrow, so try to weave something unique to you, or your site, into the article. Perhaps use arguments and points that rely on a link to one of your previous articles, in order that they make sense.

3. Launch Hard

When you create a new site, or a new piece of content, shout loud about it.

Use all the channels. Twitter about it. Email your subscribers. Submit your article to aggregation networks. Pitch your article to other publishers with whom you've built a strong relationship, and who you know will link back and credit you.

In tight communities, like the SEO world, it will be harder to rip you off if you've made yourself visible in all the channels the community uses.

4. Create A Publicity Storm

If you're very sure of your ground i.e. someone has blatantly copied and republished your content without permission, you could create a lot of media mileage by outing them. If they won't acknowledge you, then their direct competitors might be very open to highlighting the borrowers contemptible practices. Use the same approach you do when you launch hard.

Say it often, and spread the (negative) message wide. Done correctly, a publicity storm might generate more back-links links than the original article. In any case, they'll certainly think twice about taking your stuff in future.

5. Fire Fight

Another approach is the cavalry charge.

Contact the publisher, contact the people linking to them, have your friends write about the culprit. Do it each and every time someone steals your work. Do it on Twitter. Do it on their site. You could even hire an army of cheap Mechanical Turks to do the job for you.

You can find links using Yahoo's Site Explorer, duplicate posts using Google Blog Search, and Technorati.

6. Become Big

The problem with the above approaches is that they can take a lot of your time.

Where you really want to be is so big that your direct competitors wouldn't dare take your stuff. The smaller upstarts who take your stuff won't be able to rank against you anyway.

This last point is where I'd put most of my efforts. As frustrating as it is, the web is a very difficult environment in which to enforce copyright. Spending a lot of time fighting that fact won't make the interweb leopard change it's spots.

It's like the spam reporting approach. Do you spend all your time dobbing in spammers above you in the vain hope they'll all disappear, or do you beat them by building an authoritative, trusted domain?

Nurture those relationships to help you get there :)

Uncertainty, Risk & Profit Potential

Given the recent economic uncertainty in the world, I find myself reading more stuff about economics...a field which I currently find far more complex and fascinating than SEO.

I came across an article pitching the idea of another potential great depression. I don't know if that will happen, but these 3 bullet points from that article are particularly appealing to the entrepreneur in me:

  • Circa 2000 – It doesn't matter that Internet stocks are trading at multiples of revenue because 'these companies are going to change the way we do business'.
  • Circa 2005 – It doesn't matter that people are borrowing 125% of the home purchase price because 'the price of homes always goes up'.
  • Circa 2009 – US government 'T-bills and T-bonds are risk free', so the federal government can borrow unlimited amounts of money. This example of bubble-mentality thinking not only ignores the defaults by countless governments, it also ignores the history of US sovereign defaults (gold in 1933 and silver in 1967) as well as the continuing debasement of the sorry US dollar from inflation.

Whenever and wherever people are looking to pay for certainty and safety, they are paying a premium for that privilege, often yielding a net negative real return. The future does not mirror the recent past, but we are inclined to operate in a herd/bubble mentality. This, and our emotions, are why it is so easy to lose money in the stock market. By the time US actors and rappers are asking to be paid in Euros, all the dumb money is on that side of that trade, and the market is about to shift the other direction.

Business opportunities are like buses, there's always another one coming. - Richard Branson

Business opportunities are like buses, but you can't just sit around waiting for them to pull up. If an opportunity has to be "proven" before you are willing to try it, then maybe there won't be much opportunity left by the time you go after it. If there is already a "make easy money using xyz" ebook on the market, then the opportunity is probably already closed for most new market participants.

Group-think is the enemy of success. You usually have to create and believe in the value system you are selling to others for it to spread. You can't create the ideas and movements that spread if you are only following someone else's lead.

How does this concept of uncertainty vs profit potential apply online?

  • Overture (and Google) built their search marketplaces on uncertainty
  • early domainers built their empires on uncertainty
  • the first bloggers built on uncertainty
  • those trying new online business models and publishing formats right now are building into uncertainty

I got on the web in 2003, way late to the party (and broke). But in my first year of observing the web I saw that search was going to become the center of the web, noticed that domain names were important (buying domains SeoBook.com WhiteHatSeo.com BlackHatSeo.com & SearchEngineHistory.com), and quickly built a blog (because I saw other bloggers getting lots of links - primarily because they published blogs).

Simply by interacting online and observing trends you can see where the web is headed in a way that most people can not. Where others see risk, you see opportunity. Your knowledge of fields like search, blogs, commerce, affiliate marketing, and adverting lower the risk of failure for any new project you start. Each additional discipline you are aware of adds value to your other skill sets.

The cost of testing things online is minimal. Even less if you already have built up a widely read distribution channel. In the coming years new trends will augment or take the place of blogging, search, and domain names. But you have to be willing to "take risks" if you want to reap big rewards.

With ads falling off a cliff, people have been ramping up other attempts at monetization, looking for ways to be better than free and find new ways to monetize their data.

Facebook is trying to study sentiment (could that be used as an investment tool)? Fred Wilson highlighted the bloat that exists at many Web 2 companies, which holds them back from profitability. Chris Anderson, who promotes the concept of free with his new book, notes that free is pushing against its limits, and entrepreneurs are going to have to start charging during a period of limited VC backing:

What about the oldest trick in the book: actually charging people for your goods and services? This is where the real innovation will flourish in a down economy. It's now time for entrepreneurs to innovate, not just with new products, but new business models.

Time to catch the bus. Are you feeling "risky"? Today is the day. No point waiting around until things seem "safe." :)

Has Yahoo! Search Marketing Lost Their Minds?

Imagine selling web traffic as a commodity in a blind auction, while touting its value based on the traffic being targeted, relevant, precise, and trackable. Then imagine taking away the default keyword tool on the internet that has been written about in thousands of marketing books, ebooks, and web pages - and replacing it with nothing. Then imagine signing up some seedy publishing partners that run clickbots against your highest value keywords, and giving them the lion's share of the click "value" on those keywords. Then imagine not making it easy for advertisers to opt out of that "traffic." Then imagine editing your advertisers accounts without their permission to alter ad text and keywords, and only informing some of them about the changes sometime after they take place...with 1 in 5 rejecting the changes!

So inefficient and sloppy. They can call that account optimization, but only in an Orwellian sense. Why not give advertisers the tools to do optimization themselves?

Google offers about a half-dozen public keyword tools, makes it easy to filter out bad traffic, has way more volume, offers enterprise level analytics for free, and does not edit your keywords and ad copy against your permission. Is it any wonder Yahoo! managed to lose hundreds of millions of dollars last quarter, while Google keeps exceeding market expectations - even during a recession?

I just hope that when Yahoo! gets bought out by Microsoft that they keep Site Explorer around for us SEOs, and don't do us as poorly as they did their advertisers. ;)

[update: Danny Sullivan also covered this issue.]

The SEO Process Chart

It is no secret to readers here that SEO is an ongoing process, but I was playing with SmartDraw and created an SEO process circle.

One of the problems many people have with SEO is that they think that they will use SEO to get their site in front of thousands of relevant people, but that model only works if they are...

  • using pay per click marketing (buying the traffic)
  • using black hat SEO (which may provide only short term results
  • using an old trusted domain that already has many signals of quality built up

Amongst the hundreds or thousands of participants in your market, some of them enjoy an older site, more social relationships, more links, a more well known brand, a larger traffic stream due to their site already being trusted, and other traffic streams like RSS readers and email list members, etc.

All of those advantages for existing webmasters act as headwinds for a new webmaster (at least until you get established). You typically have to create some number of social interactions to leave the trail of signals of quality to make Google want to trust a site enough to put it in front of a large traffic stream, especially if you are starting a brand new site and are trying to operate within Google's guidelines. As Bob Massa says "search engines follow people."

If you like the above entry, you might also like the SEO Flowchart. :)

Who Do You Recommend for Web Design?

Good design makes quality content look and feel better. Design can help improve conversion rate, makes a site more linkable, and sometimes a site generates additional links and mentions just for having a great aesthetic design.

I frequently get asked how we can run a wide array of websites with only a few high-quality part time employees. One of our secrets is staying away from the stuff we are no good at - like web design. I could show you my attempts at design, but you would think less of me if I did. ;)

Rather than going the DIY route, I have been getting quality custom website designs from Wildfire Marketing Group for many of our newer sites, and they look great. I liked their services enough to work a deal with them to get SEO Book training subscribers $100 off their designs, which start out at $765 for a basic design and $975 for a design + a Wordpress theme. Their services page is here, and the coupon code is here.

A couple other people I would also recommend for design work without hesitation are Sophie Wegat and Chris Pearson. Though Chris Pearson is working on Thesis and no longer is available for hire. Luckily we have a 20% off Thesis coupon too.

Should You Have Multiple Websites?

Or just one?

Let's take look at a web strategy that has a number of SEO and benefits: the hub and spoke strategy. A hub and spoke strategy is when you create one authoritative domain (the hub), and then hang various related websites off that domain (the spokes).

If you don't yet have an authority site, it's probably best to focus on that one site. However, once you've built an authority hub, it can be a good idea to specialize in a number of niches using multiple, smaller sites.

Let's look at a few reasons why, in the context of dominating a niche.

Economics

Economic theory holds that division of labor increases profitability.

During the early days of the web, it was easy to make money by being a generalist. However, as the web got deeper and richer, it became difficult to maintain a generalist position unless you had significant resources.

Specialization, by way of niches, allows for greater targeting, and this targeting can increase value. Leads and advertising become more valuable, because the target audience can be reached more efficiently.

The hub and spoke approach is this theory in microcosm. The hub is the generalist authority, whilst the spokes allow for niche specialization.

We'll see how this dove-tails with SEO shortly.

Domain Knowledge

If you were to create a series of sites on different topics, it might take a significant period of time to know each area well. However, if you create niche topics within your own area of expertise, you should be able to create new sites very quickly.

Why would you create new sites? Why not just stick with one?

Let's say your main site is fairly broad in it's appeal. However, you've discovered some lucrative niche keyword areas within that broad topic area. By creating spoke sites, you can focus on these keyword areas, and dig deeper, without compromising the general appeal of your main site.

An example might be a hub site that is aimed at community education, whilst spoke sites might cover private tuition, corporate learning materials, and education facility hire.

This segmentation can be done in a number of ways. You could aggressively target one search engines algorithm and/or audience (MSN) with one spoke, whilst targeting another search engine on another spoke. One site might be aimed at do-it-yourself people, whilst another site is aimed at a person looking to hire a professional. Both sites cover the same topic, but require a different approach in terms of language, structure, offer and tone.

Likewise, you may use spoke sites for brand reasons. When Google bought YouTube they wisely kept the YouTube name, as the brand appealed to users. Google Video - not so much. There is a general perception that YouTube does video, and Google is a search company, and never the twain shall meet.

Google knew better than to force the issue.

Legitimate Links

A hub site on education that links out to pharmaceutical affiliates could easily get hit by Google. The relationship between the two areas is questionable. However, if you link out to your spoke sites, that cover related niches, your link pattern will be much more acceptable.

From an SEO standpoint, it can be difficult to get links to purely commercial sites. If you have a hub site that already has link authority - or is created specifically to attract links - then you can pass this authority to your more specialized spokes. Once the spokes become more popular, you can either pass that authority along to yet more specialized sites (one way), or even promote your hub site (reciprocal). Either way, the link graph makes sense.

Each site doesn't need to be directly profitable. You can use one site to attract links, and pass this authority on to your monetarized domains. One can subsidize the production of the other.

Fame

If you've already built up name recognition in your niche, you'll find it easier to get links and press attention for your new projects.

Status is important because if no one knows who you are, they probably don't care about the content so much. Let's say Danny Sullivan or Matt Cutts writes something, it will instantly get attention because of who they are and the trust relationship they have with their audience. If you're new to the SEO space, no matter how profound your content is, it could easily get over-looked.

This is why it can be more difficult building multiple areas across unrelated niches. You may need to establish yourself in each new area, which can be a lot more difficult than leveraging your name recognition in your existing niche, then going granular.

Enhanced Monetarization Opportunities

We've looked at how you can target the most profitable areas aggressively using a hub and spoke strategy, without affecting the main brand.

Other advantages include economies of scale. As your network grows, you have more ad inventory to sell people. The inventory can be segmented, as opposed to the advertiser having to accept a one-size-fits-all approach of a generalist site. Similarly, you may be able to demand higher affiliate payouts, because you can precisely target offers.

Aaron covers this toipic in greater depth in the video "Why You Should Dominate A Niche".

Google's .edu Domain Love: Department of Economics ≠ Mortgage, or Does It?

Some recent Google shifts have caused a lot of .edu websites to rank for competitive keywords like mortgage and credit card. Here is a screenshot of the top 100 search results for "mortgage" with 57 .edu results and 15 .gov results. And here is a similar credit card screenshot.

Note that few of these pages have any relevant on-page content. Is this a case of Google-bombing? Or did Google dial up the .edu bonus too far?

Does Google want to return all the irrelevant pages? Or does it not matter if they are deep enough in the result set? Will having mystery meat results on pages 2 through 100 hurt Google's brand? Or does everyone just click on the first page?

We discussed this a bit more in the forums: new Google results

Where Do New SEOs Go Wrong When They Set Learning Priorities?

Another question we received recently from the SEOBook.com community was:

What qualities are common in Aaron Wall, DaveN, Bob Massa, Jason Duke, SugarRae, et al, that new SEOs can adopt, to come closer to people like these in expertise. Where do most new SEOs go wrong when they set learning priorities?

I've asked these people to provide their views, which I'll get to shortly.

It's a great question, because the avalanche of SEO information that confronts the beginner can be overwhelming. How do you know what information is important? What aspects do you really need to spend you time on, and what information do you need to reject? What are the qualities that make for a good SEO?

Let's take a look...

Learning SEO

Most people stumble into being an SEO.

An awareness of SEO usually comes about when a person launches a site, only to find that the site doesn't magically appear #1.

Soon after, the webmaster will likely find themselves knee deep in SEO forums and blogs, where everyone has a viewpoint, and often those viewpoints contradict each other. Contradiction is rife in SEO. To understand why, we need to understand the history of search engines.

The first step in setting learning priorities for SEO is to.....

1. Understand The History & Context Of SEO

My own foray into SEO began with Infoseek.

Infoseek was one of the early search engines. Infoseek introduced a feature around 1996 , whereby they would crawl a site and update their index immediately. This feature made it easy for webmasters to game the algorithm.

I had just launched a small, commercial site. I thought all I had to do was publish a site, and the search engine would do it's job, and put me at number one! Unsurprisingly, that didn't happen.

So, I tried to figure out why Infoseek didn't think my site was great. I could see that there were sites ranking above mine, so there was clearly something about those sites that Infoseek did like. I looked at the code of the high ranking sites. Did that have something to do with it? To test that idea, I cut and pasted it their code into my own code and republished my site. Viola, I was at number 2!

So far, so good.

But why wasn't I number one? The sites that were ranking highly tended to have long pages on the same topic, so I added more text to my pages. Soon enough, with a little trial and error, I was number one. Predictably, Infoseek soon pulled this feature when they saw what was happening.

I was clearly not alone in my underhanded trickery.

At the time, I thought my cut n paste trick was an amusing hack, but I wasn't earning my bread and butter from the internet. I was working in the computer industry, and unaware of "SEO". I soon forgot about it.

A few years later, a whole cottage industry had sprung up around SEO. The search technology had become a lot more sophisticated. My dubious copy n' paste hack no longer worked, and the search engines were locked in a war against webmasters who were trying to game their ranking criteria.

There is an inherent conflict between the business model of the search engine, and that of the SEO. The SEO wants their site to rank, the search engine wants to rank a page a searcher will find useful.

That isn't necessarily the same thing.

Therefore, the search engines are notoriously secret about their ranking formulas. SEOs try and reverse engineer the formulas, or just guess the factors involved, which is why you'll see so many contradictory viewpoints.

So who do you listen to? What information is relevant?

2. Technical Know-How

Dave Naylor had this to say about doing too much at once:

Common qualities that's simple we notice the little things and understand the larger impact that they will have in long term,

And where do you most new SEOs go wrong when they set learning priorities?

From the new SEO's on the block that I chat too, they seem to run at a million miles an hour trying 100 different things at once, they need to slow get a decent data set of information and slowly pick though it and test small things at a time, and work out thing like why is it when I search for The FT in Google it returns Grand Theft Auto ?

Most people new to SEO place a lot of emphasis on the technical aspects. It's natural to seek out the secret recipe of high rankings. Whilst most forums obsess over these issues, much of what you'll read is irrelevant fluff. These days, SEO is more about a holistic process, rather than an end unto itself.

Start with a solid, credible source - like SEOBook's course for example ;) The cost of a well researched course is nothing compared to the time you may spend heading in the wrong direction.

Most people will benefit by applying the 80/20 rule. To rank in Google, you need to be on-topic, you need to be crawlable, and you need to have inbound links.

You could spend a lifetime trying to figure out the other 20%. Unfortunately, the formula is in Google's hands, and even then, only known to a few. It is reasonable to assume Google tweaks the dials often, especially once a common exploit makes the rounds. Take Dave's advice and take it one step at a time. Focus on the key aspects first - relevance, crawlability and linking - then methodically test and evaluate in order to expand your knowledge.

Bob Massa on not sweating the small stuff:

I honestly think the only way anyone can go wrong, new to online promotion or a seasoned veteran, is to not look too hard for tricks and magic beans from those who make their names posting those so-called tricks, in forums.

I believe anyone can be successful at online marketing or even traffic generation and search engine placement specifically, if they just stop looking for ways to trick machines and instead look for ways to connect with humans.

search engines are just computer programs and algorithms written by humans. The engine is only a tool intended to aide humans do things faster and easier that are important to their lives. I think machines can help with connecting humans BUT the humans are the target, the goal, the end that machines can provide the means to.

I think one thing that is common among the list of people you mentioned is that they all realize, understand and accept that concept.

3. Strategy & Goals

The opportunity in SEO lies in the fact that Google must have content, around which it places advertising. If you rank high, you get "free" clicks.

Of course, nothing in this world is free, and SEO is no exception. There is significant time cost involved in getting lucrative rankings. And that cost comes with a reasonable degree of risk. Google has no obligation to show you at position x, and your competitors will always try and eat your lunch.

Strategy is the most important aspect, and one you should spend a lot of your time on. Why are you trying to rank? Are there better things you could be doing i.e. building up a community? Do you have an on-going publishing model? How is your brochure-web site ever going to attract links? Are you building enough link juice to ensure your entire 500K page affiliate site gets indexed?

Check out my post on strategy and goal setting. The key is to take a holistic approach.

I think some of the general principals that apply to most of them are that they are: smart, curious, hard working, blunt, honest, and sharing. They also view SEO as a tool to help them achieve other goals, rather than having SEO be the end goal.

Where a lot of people go wrong with SEO is that they try to think in concrete numbers based on a limited perspective built off a limited set of data. Some things may happen sometimes, but there are very few universal truths to the shifting field of SEO beyond preparing for change. And the certain lasting truths do not provide much competitive advantage...that is built through curiosity, testing, hard work, and creativity - Aaron Wall.

4. Measurement

It's surprising how little time is spent talking about measurement, because without it, SEOs are flying blind.

One common metric is rank. It's not a very good metric, because it doesn't tell you very much, other than you've won the ranking game.

But so what?

What if that rank doesn't help you achieve your goals? What if every person who clicks on your link ends up buying from the guy who is advertising on Adwords instead?

This is why measurement, aligned with your goals, is important. If you track SEO efforts through to a goal, and most of those goals tend to involve making money, then you'll be head and shoulders above most of the forum hacks and pretenders. It doesn't matter what tracking software you use. Become an expert and tracking and metrics.

Summary

  • 1. Understand the history and context of SEO
  • 2. Learn your chops from a reputable source
  • 3. Clearly define your strategy and goals
  • 4. Become a metrics and measurement guru

How Your Competitors Can Help You

Are you thinking of building a new site?

Before you do, it pays to take a look at your competitors. By choosing the right sites to compete against, you can gain significant advantage.

Firstly, you need to position your offering relative to your competitors.

1. What Problem Do you Solve?

Making money is mostly about solving problems. Write down the problem you're going to solve. Be specific.

For example:

  • Provide auto repair training to amateurs
  • Sell bomb detectors to airlines
  • Sell ice to Eskimos

For this article, we'll use the idea "sell ice to Eskimos". No doubt you've already spotted the problem with this rather lousy business model, but let's have a look at what a bad idea looks like within this evaluation process.

2. Who Is Your Audience?

You may have noticed I included the prospective audience in my examples above. Know what you're selling, and to whom.

Demographics, in other words.

Who are you customers? What do they want? What type of language do they use? Build up a profile.

In our example, our customers are Eskimos. Eskimos live around the North Pole region, mainly in Siberia, Alaska, Canada, and Greenland. Internet access is obviously going to be an issue, not to mention language barriers, which is about the point the idea should die.

Yet, surprisingly, many prospective web businesses never address this simple question. Various Web 2.0 businesses clearly didn't ask questions 1 & 2. Presumably they jumped straight to the "how can we get a few million dollars in VC?" question instead.

3. Where Are Your Customers Hiding?

You need to get in front of your audience.

SEOs know a lot about keyword research, so have a huge advantage over others when it comes to finding out who their competitors are, and where the opportunity lies.

You're probably familiar with keyword research tools and competitive research tools such as:

Find out the keyword terms your potential audience use, conduct searches, and make a note of the big players under those keyword terms. Keep in mind that language people use on search engines is always changing. There are more queries that are longer and more specific. These give you valuable insights into how to position your offering.

What questions are people asking? What problems are they trying to solve? What are the many different ways they describe that problem? What keyword areas are your competitors missing? What value can you provide that they do not? Have your competitors missed lucrative keyword areas?

4. What Is The Nature Of The Market?

You should look for rapidly growing markets. You want to avoid established, declining markets, unless you can provide a new layer of value that is difficult for competitors to emulate.

Take a look at the type of sites you intend to compete against. Are they big companies? Are they hobby blogs and thin affiliate sites? It's going to be much easier for your new site to compete against the thin affiliates and hobby projects than it is to compete against the establishment.

One of the common stumbling blocks at this point is solving a non-problem. The "ice to Eskimos" market is not dominated by established players, or hobby blogs for that matter, but there's a good reason for that - Eskimos don't have a "lack of ice" problem. Beware of the Web 2.0 trap - solving the non-problem.

5. What Related Markets Exist?

If the market you were thinking of entering is competitive, are there any closely related markets you can enter? You can find these areas by looking for patterns in the keyword research results.

Let's try "fitness". Notice any patterns here?

You might notice there are numerous searches for fitness locations i.e. a gym, a center, a club. So, instead of targeting fitness in terms of health, which would see you up against established health organizations and generalist publications, you might want to target the fitness center section of the market e.g. a comparison of gyms and centers. Such a niche could possibly be more lucrative, as there is a clear money making opportunity as people need to pay to join these facilities.

Which brings us onto...

6. Is there Potential To Make Money?

Just because a lot of people are doing something, doesn't mean it is worth doing.

Some areas are difficult to monetarize. Science, for example. And social discussions. Some area's are saturated, making it very difficult to find a new value layer to add. SEO, for example.

How easy would it be for one of the major players to copy your value proposition? Look for areas that have a clear path to monetarization and that aren't dominated by major players, or saturated by sites with little to distinguish them.

Good luck in your hunt for a lucrative niche :)

If you want to see a video presentation on how to evaluate competitors, Aaron has more in the members section.

Majestic SEO - Interview of Alex Chudnovsky

Majestic SEO.

I have been fond of the depth of Majestic SEO's data and the speed with which you can download millions of backlinks for a website. While not as hyped as similar offerings, Majestic SEO is a cool SEO service worth trying out, and their credit based system allows you to try it out pretty cheaply (unless you are trying to get all the backlinks for a site as big as Wikipedia)...as the credits depend on the number of inbound linking domains.

They give you data on your own domain for free, and share a nice amount of data about third party sites for free. For instance, anyone can look up the most well linked to pages on SEO Book free of charge

What made you decide to create Majestic SEO?

We arrived to it naturally - our main aim with the Majestic-12 Distributed Search Engine project is to create a viable competitor to Google. We use volunteers around the world to help us crawl and index the web data. This project was started in late 2004 and about 2 years later it became clear that we need to be as relevant as Google and in order to do that we have to master power of backlinks and anchor text. As time went on many hundreds of terabytes of data were crawled it also became clear that we need to earn money as well in order to sustain our project. It took well over a year to actually achieve the level that we felt confident with to release it publicly in early 2008.

What were the hardest parts about getting it up and running?

The most difficult part was to avoid the temptation to simplify the problem and focus on a small subset of data that is much smaller than that indexed by Google. It was felt that it would be a mistake as you can't really be sure that you have the same view of the web unless you are close to Google's scale.

Once it was decided to follow the hard path a lot of technical scalability problems had to be solved as well, and then deal with the financial aspect of storing insane amount of data using sane amount of hardware.

You use a distributed crawl, much like Grub did. What were some of the key points to get people to want to contribute to the project? How many servers are you running?

The people that joined our project did so because they felt that Google is quickly becoming a monopoly (this was back in 2004) and a viable alternative was necessary. We have over 100 regulars in our project that run distributed crawler and analyser on well over 150 distributed clients: all this allows us to crawl at sustained rate of around 500 Mbits.

Since we recently moved closer to commercial world with Majestic-SEO it was decided that our project participants will benefit from our success by virtue of share ownership - essentially project members are partners. It needs to be stressed here that our members did not join the project for financial reasons.

How often do you crawl? How often do you crawl pages that have not been updated recently?

We crawl every day around 200 mln urls. At the moment our main focus is to grow our database in order to catch up with Google (see analysis here), however we have dedicated some of our capacity to recrawls, in fact in February we should have new version of automatic recrawls of important pages (high ACRanked) released and this will allow to see competitor backlink building activity pretty quickly. Our beta daily updates feature shows new backlinks found in previous day for registered or purchased domains, this gives a chance to see new backlinks before we do full index update (around every 2 months time).

What is AC Rank? How does it compare to Google's PageRank?

ACRank is a very simple measure of how important a web page is based on number of unique domains linking to it. More information can be found here: http://www.majesticseo.com/glossary.php#ACRank

This measure is not as good as PageRank because it does not yet "flow" between pages. We are going to have much improved version of ACRank released soon.

Do you have any new features planned?

Can't stop thinking about them ;)

You allow people to export an amazing amount of data, but mostly in a spreadsheet basis on a per site basis. Have you thought about creating a web based or desktop interface where people can do advanced analysis?

We offer a web based interface to all this data with ability to quickly export it using CSV format.

For example, what if I wanted to know pages (or sites) that were linking to SearchEngineWatch.com AND SearchEngineLand.com but NOT linking to SeoBook.com AND have a minimum AC rank of 3 AND are not using nofollow. Doing something like that would be quite powerful, and given that you have already done the complex crawl I imagine adding a couple more filters on top should be doable. Another such feature that would be cool would be adding an Optilink-link anchor text analysis feature which allows users to break down the anchor text percentages.

We do have powerful options that enable our customers to slice and dice data in many ways, such as excluding backlinks marked as nofollow or only showing such backlinks, this applies to single domain analysis however, but something like what you describe in your example of interdomain linking will be possible soon.

Have customers shared with you creative ways to use Majestic SEO that you have not yet thought of?

We get good customer feedback and often implement customer requested features to make data analysis easier. As for new creative ways our customers prefer to keep them to themselves, but once you look at the data you might see one or two good strategies on how to use it. ;)

How big of an issue is duplicate content while crawling the web? How do you detect and filter duplicate content?

It is a very big issue for search engines (and thus us) as many pages are duplicate or near duplicate of each other, with very small changes that make it hard to detect them. We do not currently detect such pages (we crawl
pretty much everything) though we have a good idea how to do it and will implement it soon. Our reports tend to show data ordered by importance of the backlink, so often it is not an issue though it depends on backlinking profile of a particular site.

A lot of links are rented/bought, and many of these sources get filtered by Google. Does your link graph take into account any such editorial actions? If not, what advice would you give Majestic SEO users when describing desirable links vs undesirable ones?

At the moment our tools report factual information on where backlinks were found, we do not currently flag links as paid or not. This is something that humans are good with and computer algorithms ain't - that's why Google hates such paid links so much. We do have some ideas however on how to detect topically relevant backlinks (paid would usually come from irrelevant sites) - it's coming soon and might actually turn up to be a ground breaking feature!

Microsoft has done research on BrowseRank, which is a system of using usage data to augment or replace link data. Do you feel such a system is viable? If search engines incorporate usage data will links still be the backbone of relevancy algorithms?

BrowseRank is a very interesting concept, though we are yet to see practical implementation on large scale web engine. I don't think such system obsoletes link data at all, in fact it is based on link data just like PageRank only it allows to detect the most relevant outgoing links on a page, essentially such votes should be given more weight in PageRank-like analysis. For example imagine that this very interview page is analysed using BrowseRank and it finds that the following cleverly crafted link to Majestic-SEO homepage is clicked a lot, then such link could be judged as the real vote that this page gives out!

This approach would help identify more important parts of on page content as well so that keyword matches within this content block could get higher score in ranking algorithms. So I actually think there is a lot of mileage in BrowseRank concept, but it would be a mistake to think that it will completely replace need for link data analysis. I am pretty sure Google uses something like this already - Google Toolbar stats would give them all they need to know.

The great irony in my view is that Microsoft lacks good web graph data to apply their browsing concept, this is probably why they are so desperate to buy Yahoo search operations who are much better than it comes to backlinks analysis, though Google are the real masters. Majestic-SEO is trying to slot itself just behind Google and who knows what happens after it ;)

I look up a competing site and see that a competitor has 150,000 more links than I do and feel that it would take years to catch up. Would you suggest I look into other keywords & markets, or what tricks and ideas do you have for how to compete using fewer links, or what strategies do you find effective for building bulk links?

First of all: don't panic! :)

Secondly use the SEO Toolbar that will query our Majestic-SEO database to show number of referring domains - it may well be very few.

Thirdly consider investing into detailed stats we have on this domain: this will tell you anchor text used, actual backlinks that you can analyse by their importance (we measure it using ACRank). Once you see real data a lot of things can become clear: for example you can see that your competitor has got lots of backlinks pointing just to homepage or spread around the site. Seeing actual anchor text is really an eye opener - it can show which keywords site was optimised for, this will allow you to make a good decision whether you can catch up or not. Chances are you may find that your competitor is weak for some keywords, this is where keywords research tool like Wordtracker is invaluable.

And finally consider that a few good relevant backlinks are likely to be worth more than many irrelevant ones: it is those backlinks that you want to get and knowing where your competitor got them should help you create a well targeted strategy.

You allow people to download a free link report for their site. How does this report compare to other link sources (Yahoo! Site Explorer, Alexa links, Google link:, and links in Google Webmaster Central)?

We give free access to verified websites, this is a great way to try our system and you might see the backlinks that you won't find elsewhere because our index is so large and we show you all backlinks (rather than top 1000) that we've got: this will include backlinks from "bad neighbourhoods" (this is not yet automatically marked by our system, but visual human analysis wins the day here) that you may not be shown in other sources.

We believe that our free access reports are the best in class, since it's free why not find out for yourself?

For analyzing third party sites you have a credit based system. How much does it cost to analyze an average site?

The price depends on how large (in terms of external referring domains) a particular website is. We have some sites that have hundreds of millions of backlinks, average would be very different depending on what you really after, the best option is just to run searches for domains that you interested in on our website, this will give you very interesting free information as well as price for full data access.

For a domain like Wikipedia I might only want the links to a specific page. Are you thinking about offering page level reports?

Yep I am thinking of it - I actually had requests like this, funnily Wikipedia being the main object of interest.

What is the maximum number of links can we download in 1 report?

Our web reporting system tries to focus on most valuable backlinks to avoid information overload, however we allow complete dataset download that will include all backlinks - some of our clients have retrieved data on domains with well over 100 mln backlinks! Using our powerful analysis options you can focus on backlinks for particular urls coming from particular pages and retrieve all qualifying data.

------

Thanks Alex. For more information on Majestic SEO please visit their site and look up your domain.

Pages