Now All Blogger Outreach Campaigns Will Be Considered Spam Too :)

Not that long ago I highlighted how infographics were largely being destroyed as a link building tool by some unscrupulous folks who were offering to pay people to host the infographics to their sites - in a sense making the word infographic seem & feel like spam, just like paid links. :D

After killing that source of links, the folks behind that work are proudly moving on to fake non-profits and awards for bloggers, launching multiple fake charity sites in the last couple months, and then mass emailing bloggers with fake awards and link buying offers, along with a touch of comment spamming. I mean, its not that many comments. :)

Any website with only 4 pages of content, that claims to be a non-profit, yet has no contact information available, while claiming to be 3 or 4 years old (even the domain name was only registered 2 months ago) is probably somewhat sketchy.

Hello,
I'd love to purchase a text link ad on one of your pages such as ____

The link would be going to a finance or MBA site.

My budget is $100 and can pay via PayPal. I can give you a call about the
details or email you more information, let me know.
--
Maggie Sands

As a marketer, you want anything you do to pass the sniff test. So if your stuff looks anything like this fake scammy crap garbage then you are not going to have much success with it. Largely because the folks who are sending out millions of emails are going to sterilize the market and turn the web cynical toward even more marketing techniques. So you need to make your marketing efforts that much more personalized, and it also helps to have a real presence in the field, that way bloggers won't dismiss you as just another scam, like they might those folks promoting the fake charity angle.

This is another reason why it also helps to create things in entirely unique formats. The gamers exploiting stuff burn out one opportunity after another, but most of their new & creative slants are simply extensions of things that worked for others. Getting out in front of the scammers on a new trend & format is far more profitable than following in their footsteps. But be aware that marketing ideas have a curve to them. An idea which starts off pure and is successful & profitable will end up coming under the eye of scammers at some point. And most forms of fraud are based around trying to look like the real thing, so eventually a format that was once profitable eventually loses its potency and you must move on.

The best forms of marketing which help you differentiate yourself from the scammers are those which build trust over time: branding, awareness, social interaction, etc. The person holding up a puppet might be able to compete with you here and there from time to time, but if you build something with depth and substance they will have a hard time competing on a sustainable basis.

The Yahoo! Story

Worth a click, and full of cautionary reminders :D

The Yahoo! Story.

Search Power Plays, And How To Avoid Getting Crushed

The little guy often loses.

As market niches get saturated, the winners are typically those with the deepest pockets.

Up until the last few years, the little guy has been able to prosper with SEO. The little guy didn't face much competition from big companies, because the big companies didn't get SEO. However, Google's current algorithmns and corporate strategy often have the side effect of benefiting large companies.

According to Google CEO Eric Schmidt, the Internet is a "cesspool" where false information thrives....Brands, he said, are the way to rise above the cesspool

There is a danger in reading too much into Schmidt's words, however this statement mirrors a lot of what happens in the search results. A big company or brand, with a crawlable site, will find it easy to dominate the search results. A big company will be linked to, discussed in the media, and have established keyword query volume - all factors which Google rewards. All these factors are becoming increasingly difficult for the small guy to emulate.

Factor in Google's ongoing moves to "own" verticals, and many more little guys will be crushed underfoot. It doesn't matter if your site is white hat, grey or black, if your site competes directly with a big company, or with Google - who are now a big company themselves - you'll almost surely lose.

This isn't just true in the SERPs, of course. It's also true in Adwords, which essentially rewards those with deep pockets. It's true in print. It's true across all media. It's true in politics, in money markets, and in life.

Power is like that.

Even if you don't face competition from big operators, you'll face competition from a million other little guys, especially if there is no barrier to entry. This is often the case on the web. Check out this article by Tim O Shea, founder of the short lived UK group buying site Snippa. Snippa was similar to Groupon.

Due to the number of players, commission levels are being eroded far from the 40-50% that Groupon achieves down to 0% just to get the deal (at Snippa our deals averaged around 10-20%). Merchants are getting numerous phone calls from prospective group buying companies and the conversation with many is more about the commission level charged rather than how they could offer a great discount for a group of new customers. This will continue until a clear leader emerges that can demonstrate a large customer base allowing them to negotiate better deals and commission levels. Many companies chasing the same deal is counter productive for the end customer.

Too many competitors errode margins to zero. Eventually, the biggest operator wins.

How To Protect Yourself And Win

When you're looking for a niche to get into, how do you evaluate it?

Do you look at the search volumes and look to position a site top ten for that search volume? An ok strategy, and one used by many in the SEO business.

However, lets take it a step further.

If you're thinking long term, you need to consider other factors, especially competitive threats. Ask: is this niche likely to be so lucrative that it will attract big companies? If so, then you may need a strategy to become one, or be bought out by one. You may win such a fight for a while, but the big company will invariably win in the end through greater reach and purchasing power.

Are you the cheapest, or are you the best?

Pick one.

The little guy is almost always better off aiming to be the best at what they do. Being the cheapest requires volume, and is very difficult to sustain. Many companies, both big and small, get locked in a downward spiral of price cutting. Again, you'll last being the cheapest until a bigger company turns up. Bigger companies can get price advantage through volume. If the internet equivalent of Wal Mart is your competition, you're in trouble if you compete on price.

Zappos was a small company, that eventually became a big company, not by competing on price, but by competing on service. They aimed to be the best at service. Had they competed on price, they wouldn't have got anywhere. The big shoe and clothing chains would have crushed them.

Is SEO your only strategy to dominate a niche? If so, then you're vulnerable to the whims of Google. Instead, think about ways you can develop a brand. I use the term brand in the widest possible sense. Being the best guy in the world to talk to about, say, the eating preferences of neon tetra fish - is a brand. Whatever it is you do, if you're not competing on price, aim to be the very best. If you have to carve a niche even finer, do it, at least until the costs outweigh the benefits.

Think about ways you can lock in customers/visitors and keep them coming back. If you only ever have search volume, then you rely on people who haven't seen you before. Encourage visitors to bookmark you, or sign up for a newsletter. Hook them in some way. Above all, be memorable. Being memorable will create search volume out of nothing (how many people searched for Zappos years ago? Or SEOBook? ). Building an audience may not be enough to fend off big companies, but it will help you fend off other small companies and new entrants, especially if they only rely on SEO.

Be the big guy in the little niche :)

Money is an Arbitrary Value System*

* If you think otherwise, do you care to debate the following video?

"Perhaps the only lessons learned by the banks in the crisis is that the rules don't apply to them." Ain't that the truth. The video is completely accurate:

Richard M. Bowen, former chief underwriter for Citigroup’s consumer-lending group, said he warned his superiors of concerns that some types of loans in securities didn’t conform with representations and warranties in 2006 and 2007.

“In mid-2006, I discovered that over 60 percent of these mortgages purchased and sold were defective,” Bowen testified on April 7 before the Financial Crisis Inquiry Commission created by Congress. “Defective mortgages increased during 2007 to over 80 percent of production.”

Read that bold sentence again.

The bankers KNEW they were committing fraud that would create trillions of Dollars in losses.

Rather than suing them for the appraisal fraud, accounting control fraud, securities fraud, mortgage fraud & foreclosure fraud, instead we have a government that assists them in the fraud by papering over their mistakes and forcing other institutions to waive the right to sue over the fraud.

Everyone else is forced to eat the socialized losses that mirror the "profits" generated by the banks in years past. We have a central bank holding rates at 0 while running the printing presses in overdrive. That is destroying pension accounts and the living standard of the elderly as their life savings is literally stolen from them to prop up a bunch of criminal thugs on Wall Street.

The goal is not to create value or stability, but rather to engineer instability, with sharp market moves which can be profited from on the way up AND on the way down:

Bubble, Crash, Bubble, Crash, Bubble ...

We will continue this cycle until we catch on. The problem isn't only that the Fed is treating the symptoms instead of the disease. Rather, by irresponsibly promoting reckless speculation, misallocation of capital, moral hazard (careless lending without repercussions), and illusory "wealth effects," the Fed has become the disease.

The average person is fed backwards looking misinformation by the media, which is sponsored by those selling the bag to others.

If you are in debt for consumption driven reasons it is worth noting that the game is rigged. These people have done, are doing, and will continue to act as criminals to rip you off. There is no reason to go out of your way to make yourself easy prey.

Are you anti-American? Do you have an axe to grind? Did you somehow fail at life? These are questions we are trained to think when anyone complains about scams embedded in the political economy.

Speaking for myself, I have never been as successful as I am today, and much of that success comes from the free market ideals that have been preached (and meeting my wonderful wife). The issue I have right now is that we still preach those free market ideals, but are doing nothing to follow them (and are actively undermining them). If I was born 5 years or 10 years later I might be a street beggar, simply because my opportunity to succeed had been marginalized by the market corruption of the bankers. Maybe I still would have done ok, but a lot of people are being driven to the margins and off a cliff. Many of them are even sadly conned into believing it is their own fault.

Certainly consumers are to blame for some stuff, but they had no idea how much fraud was embedded in the financial engineering going on beyond the headlines. They didn't know that the central banker believed that there was no reason police fraud. Most people are too busy to be macro-economic experts on the side, and yet if you made the mistake of trusting the bankers in some cases they may have destroyed your life savings or left you living a meager life as a debt serf.

Back when the FBI warned that there was an epidemic of mortgage fraud (in 2004), the US federal government shifted some of those resources to policing other areas like homeland "security." When states tried to protect their citizens against predatory loans the federal government used preemption to block them, allowing the fraud to continue. As Alan Greenspan states: there was fraud, it was intentional, and it was indeed a series of criminal acts.

None of that is a matter of debate.

And none of the banking executives are in jail.

The federal government sided with the criminals on the way up. And it is siding with the criminals on the way down. They would rather debase the currency and steal your savings than let their criminal buddies on Wall Street go bankrupt.

Society is nothing more than a system of laws and the culture they promote. Betray someone's trust and they become less trusting toward everyone else. That creates friction in the marketplace which shrinks the economy and living standards. Make heroes out of criminals & at some people a lot of people are going to say "screw it, the laws do not apply to me either." When that happens (as it will) then at some point there will be a sharp increase in violence.

This is not about promoting communism, or some other such label. What we have now in the capital markets is far worse than communism, as they have privatized profits AND socialized losses. If you believe in capitalism OR communism then what is happening now is broken.

And yet the people who just privatized the profits and socialized the losses are to be worshiped and followed. You should be thankful that the government bailed them out & you should just suck it up. The role of government is to protect the wealth of the opulent from the stupid majority. It has always been.

The stupid thing about it is that all (or at least most) of that pain & suffering is easily avoidable by letting the people who preach free market values to eat their losses and fail. By pushing those off onto everyone else, ultimately the government just creates uncertainty and makes people less trusting. And as society breaks down, the government won't be able to sort out the issues, as they already gave away all the money to a bunch of brigands on Wall Street.

They will dance while your family starves.

"I believe that banking institutions are more dangerous to our liberties than standing armies . . . If the American people ever allow private banks to control the issue of their currency, first by inflation, then by deflation, the banks and corporations that will grow up around [the banks] . . . will deprive the people of all property until their children wake-up homeless on the continent their fathers conquered . . . The issuing power should be taken from the banks and restored to the people, to whom it properly belongs." —Thomas Jefferson

Google Ranking Internal Pages Rather Than Home Pages

When you get to *really* competitive keywords the results typically tend to be fairly stable because the cost of entry into the game is so high & many of the top players keep building additional signals of quality. You might get minor fluctuations from time to time, but large fluctuations on highly competitive keywords are fairly rare.

Over the last day or 2 Google has done yet another algorithm change (the 3rd or 4th noticeable one in 2 weeks), where on some searches they are ranking an internal page over the homepage. It is almost as if the best mental model for the algorithm that is doing this is...

  • find the top SITES that deserve to rank well & rank them based on that criteria
  • however, rather than ranking THOSE PAGES, instead do internal site searches & back in other relevancy factors to look for other popular & relevant pages on those sites
  • test to see how well searchers respond to them

Here is a pretty overt example, where Google changed 2 of the listings for "SEO" to internal pages.

I have seen other examples, some where Google also highlighted a new information-less blog post with only a couple automated backlinks pointing at it. I don't want to "out" that site though, but this is the type of image Google was showing beneath that entry

Google could conceivably use this sort of process to further adjust the search results based on demographics, searcher location, recent searches, searcher interests, and so on. Add in the ability to send searchers down a known path optimized for profitability, the ability to select vertical databases on the fly and change the titles on the fly and it allows whoever has the most search market share to keep refining the results to make them more appealing to users at an ever increasing level of granularity & greater profitability.

I have no problems keeping up with the increasing complexity of search, but Google is setting up some serious barriers to entry for new players. It is hard to explain in a straightforward manner that page A might be ranking due to relevancy signals pointing into page B, but these are the SERPs through which we make a living. And it is only going to keep growing more complex. ;)

Depending on how far Google pushes with this, it can have major implications in terms of rank tracking, SEO strategy, site architecture & conversion optimization. More on that stuff in the community forums.

Google Places & Local Search: A How-To Guide

As covered in Aaron's localization post, Google has introduced a few subtle changes to local search.

Take a look at the new results for location searches:

Did we say subtle?

Really, we meant "whoa!"

Perhaps Google should be renamed Altaooogle. Or GoogVista. Out with simple search results pages, and in with a cluttered mess. Or a "spectacularly useful" search results page, depending on your point of view :)

I guess it keeps search visitors on Google for longer. There are new clustered results, that show references from around the web, such as reviews on Trip Advisor. Certainly nice to see scraping is back in vogue again.....

This change replaces the Google "7 Pack" i.e. the seven or more local results alongside the a map, which appeared as the first set of results against local searches.

Here's the official Google pitch:

Sounds wonderful. Fluffy bunnies all round.

This is a big change for local search. Let's take a look at how to take best advantage.

Create A Place Page

Google wants there to be a Place Page for every place on the planet. In order to claim a Place Page, you can sign in here.

Google is providing a clear incentive for the publication of information-rich Places Pages. The more complete the data, the more exposure you're likely to get. Even if you don't provide information, Google will likely scrape information from elsewhere on the net - such as reviews sites - and bring it together in the SERPs. So if you're lacking third-party reviews, especially good ones, now is the time to "encourage" a few more.

Pay Your $25

Huh?

Yes, Google, ever the "friend" of the little guy, has introduced charging on their Places tags. Tags used to be free. Now, for a fee of $25, per month, per listing, you can have a web URL tag. Nifty. Once wonders how long it will remain at $25, of course.

Gotta pay for those Google Jets somehow.

Get Ranking

Time to revise your local search strategy.

Claim your Place Page. Ensure you use a local address, zip codes and phone numbers, wherever appropriate. Tag photos with place names. List your website with local directories, YellowPages etc, and get a link back to your site. List anywhere that associates you with a geographic place i.e. a listing at the local chamber of commerce. If your audience is outside the US, use local domains i.e. co.uk. com.au, etc.

So, Should You Use Google Places To Advertise?

Like anything, it comes down to how much value you can extract. Run a Place Page, pay your fees (if you want the tag option), and see how it rolls. It costs little or nothing to post your details.

As SearchEngineLand points out, if a local businesses uses Google Places, they may not need a web site, a blog, a Facebook page, a mobile strategy, an Adwords campaign, SEO or all the other marketing strategies that very likely confused the hell out of them. That's certainly a cost saving, so it's reasonable to assume many small businesses will benefit. Google Places provides some interesting tools and data, and the learning curve isn't as steep as search marketing and other web publishing channels, which saves time, and therefore money.

On the downside, Google, as ever, has the control. Like Adwords became oriented around the big spenders, which has progressively rewarded those with the deepest pockets and squeezed the little guy out, the same thing may happen to local, especially if Google decides to eat the Yellow Pages markets' lunch. There's no guarantee you'll gain/maintain search exposure, especially when competition heats up, and Google are unlikely to leave money on the table.

Let us know your experience of Google Places, and the new treatment of local results.

Google Replacing Page Titles in Search Results With On-Page Headings

When Bing launched, one of the interesting things they did to make the organic search results appear more relevant was to use link anchor text to augment page titles (where relevant). This would mean if people searched for a phrase that was mostly in your title (but maybe your page title was missing a word or 2 from the search) then Bing might insert those words into the page title area of your listing if they were in some of the link anchor text pointing into your page.

Before being switched over to Bing, Yahoo! would sometimes display the H1 heading as the clickable link to your site (rather than the page title). Bing also uses on page headings to augment page titles.

Historically if Google has thought it would appear more relevant to searchers, sometimes they have shown a relevant machine generated piece of your page displaying those keywords in context rather than the meta description in the snippet, but typically Google has been far more conservative with the page titles. Sometimes Google would list the ODP title for the title of a page, most until recently they have generally typically just listed the page title as the clickable link to your site.

Recently Google has grown more experimental on this front, being willing to use link anchor text and on-page headings as part of the listing. In addition, if the page title is short, Google may add the site's name at the end of the title.

Here is an example in Google of the page title being replaced by part of an on-page heading & also showing the site's name being added to the end of the link

And here is the associated on-page heading for the above

I have also seen a few examples of the link anchor text being added to the page title in Google, however it was on a client project & the client would prefer that I didn't share his new site on an SEO blog with 10's of thousands of readers. :D

Last November Matt Cutts recently did a video on the topic of Google editing the page titles for relevancy & how it was a fairly new thing for Google. Even back then Google was quite conservative in editing the clickable link ... I think they have only grown more aggressive on that front in the past month or so.

Interview of Rich Skrenta

Rich Skrenta has been at the core of search longer than I have been in the SEO market. He is famous for launching sites like DMOZ and Topix. His most recent project is a search engine called blekko, and I recently had a chance to chat with him about blekko, the web, and marketing.

Blekko search engine.

Blekko just launched publicly today. Be sure to check out their search engine, all their SEO features, and the Blekko toolbar.

Most start ups fail. And yet you have multiple successes under your belt and are going at it again. If you could boil success down to a few points, what really separates what you have done from the statistics?

Paul Graham said that the most important thing for a startup is to not die each day. If you can keep existing, that's survival for a company. Generally I like to keep costs low and hire carefully. Also, the first idea doesn't always work. We had to pivot Topix several times to find the right model. For blekko, we just want to make a site that a segment of people will find useful. If we can do that we'll be happy.

It seems openness is a great marketing angle to use online. Why do you feel that it is so under-utilized by most companies?

It feels counter-intuitive to take all our your company IP and secrets and just put them all out there. Little companies also tend to be insecure and want to be appear to be larger and more successful. They want to put on a big company face to the world, but being honest and transparent about who they are and letting the public see "behind the curtain" can often win people over better than a facade of success.

From my perspective, it seems your approach to marketing is heavily reliant on organic, viral & word of mouth strategies. What is broken with the old model of marketing? Is its death happening slower or quicker than you expect?

The internet and social media have made word-of-mouth stronger and stronger, and in many ways they eclipse traditional marketing channels now. This started with blogging and has accelerated with Twitter and Facebook. Everybody is media now. You used to fly around and do a 2 week media tour to launch a product. The aperture to get in the trade press was small, there was a handful of reporters you had to go pitch. Now there are thousands of people who have audience for every trade niche, so it's easier to get the word out about something new. But it has to be genuinely interesting, or your message won't get pickup.

A lot of people who are good at programming make ugly designs. Likewise many people are either programmers or marketers. What formal training or experiences have you had that have allowed an engineer to become such a sophisticated marketer? What strengths do you have that allow you to bridge the disciplines so well?

We joke that we have always made ugly web sites. Fortunately I was able to hire a good designer for blekko and he's been doing a great job taking our early ugly versions and making them a lot more attractive and workable.

I read a lot of stuff about marketing and positioning that we're trying to apply at blekko. I'm a big fan of Trout & Ries. I loved Kathy Sierra's stuff when she was writing. There is some fantastic material also in Kellog on Branding. We also worked with some great positioning consultants that tested various ideas on focus groups to see what would resonate with users best as a message. Every product has a bunch of features, but you want to find the one to talk about that's going to stick in people's heads the best.

I noticed you baked many social elements into your marketing strategy (friend us on Facebook, follow us on Twitter) as well as baking many social elements into your product (personal slashtags, allowing people to share their slashtags, etc.). There is some talk on the web of apps or social stuff replacing search as the center of the web, however from a marketing perspective I see much higher traffic value in search traffic. Do you think that one day social and apps will largely replace global search? Or do you feel it will generally continue to play a secondary role to search?

Social media can drive tons of attention, awareness and traffic. But the search box is the best way to navigate to stuff you want. Now what will drive those results - if I type in "pizza", what should I get? The answer can be very different depending on whether the results are coming from the web, Yelp, or Facebook. So I guess my answer is that I still see search being the core way to navigate, but I think what gets searched is going to get a lot more structured and move away from simple keyword matches against unstructured web pages.

A good number of the social sites are doing redirects for security purposes & to some degree are cannibalizing the link graph. Do you feel that links from the social graph represent an important signal, or that most of that signal still gets represented well on the remaining link graph?

There is very definitely signal in social graph links - potentially more than in the web graph. In 2000, a hyperlink was a social vote. Most links were created by humans and represented an editorial vote. That's no longer true - the web today is inundated with bulk-generated links. To the extent that humans can be separated from bots, there's more true signal in social graphs. The challenge is to get enough coverage to rank everything you need to rank. Delicious had great search results for the corpus of links they knew about, but it wasn't nearly big enough to be comprehensive. Facebook and Twitter are certainly a lot bigger, it will be interesting to see if they start to apply their data to ranking and recommending material from outside of their own sites.

When Google was young Sergey Brin at an SEO conference stated that there was no such thing is spam, only bad relevancy algorithms. When I saw some of your talks announcing Blekko you mentioned that you never want to see eHow in your personal search results. Do you feel that spam is largely down to a personal opinion? If you had to draw a line in the sand between spam & not spam, how would you characterize the differences?

Search must serve an editorial function. You can call this editorial position "relevancy", but that's hiding behind the algorithm. Of course someone wrote the algorithm, and tinkered with it to make some sites come up and others not to come up.

The web has grown 100-fold since 2000. There is most definitely spam out there. Let's take a clear-cut example, like phama links being injected via exploits into unpatched WordPress blogs. Then there is gray-area stuff, like eHow.com. Some people like eHow. Some don't. That's why we let users develop their own /spam filters.

Eric Schmidt mentioned that sharing their ranking variables would be disclosing trade secrets that could harm Google. Yet you guys are sharing your web graph publicly. Are you worried about doing this impacting your relevancy in a negative way? Or do you feel the additional usage caused by that level of awareness will give you more inputs into your search relevancy algorithms?

When I first moved to Silicon Valley I worked in computer security. In security there's an idea that "security through obscurity" isn't very good. What this means is that if you have some new encryption algorithm, but don't let anyone see the details of how it works, it probably is full of holes. The only way to get a strong encryption algorithm is to publish all of the details about how it works and have public review. Once the researchers can't punch any more holes in your algorithm, only then is it good enough to trust.

We see search the same way. If this magic 200-variable equation is so sensitive that if it leaked out the results would be completely overrun with spam, well then the algorithm doesn't actually sound that strong to me. We'd rather work towards a place where there can be public review of the mechanisms driving ranking, and where many eyes can make the spam problem shallow.

Certainly the big search engines have hundreds of human raters that help identify spam and train their algorithms. These are contractors that are the knowledge workers behind the scenes. As a little startup, we asked ourselves how we could get many more people helping us to make our results better, and also be a lot more open about the process. Formerly we had experience running a big crowdsourced search site with the Open Directory, where we had 80,000 editors classifying urls. What if we could get 80,000 people to help us curate search verticals, identify spam, and train classifiers? That would be cool.

You had a blog post comparing pornographers to SEOs. Do you feel the SEO game is mostly adversarial? Or do you feel that paying attention to the SEO industry is a great way to quickly improve the quality of a search product? Or both? :)

I think my comparison noted that pornographers have often been early adopters of new technology. :-)

There is aggressive seo, and then there is what I call appropriate discoverability. Aggressive seo can go over the line - if someone hacks your server to add links, that's borderline criminal activity. But if you have great content and it's not showing up, that's a shame. After we sold topix to the newspapers, we spent some time evangelizing seo within their organizations. Think of all of the movie reviews and restaurant reviews the US newspaper sites collectively have. Wonderfully written material by well-paid professional journalists. But you don't see their content anywhere for a restaurant or movie search. That's a shame.

Recently Ask sorta rebranded away from search & towards more of a QnA format, and Yahoo! bowed out of search through a Bing partnership. Are the cost scales that drive such changes just a legitimate piece of the business model, or were those organizations highly inefficient? How were you able to bring a competitive product to market for so much less?

I was a fan of Ask's Teoma technology, and what Jim Lanzone had been doing with the site. And Yahoo was delivering very high quality results, and had interesting initiatives like the BOSS apis and SearchMonkey. This was all great stuff. I'm disappointed that they lost heart. Running a big company that has been around for a long time is not an easy job.

From an SEO perspective I think that Google tends to have a large index, but crawling so deeply likely allows a lot of junk into their index. Bing seems to be a bit more selective with their crawling strategy. How would you compare Blekko against the other major search engines in terms of depth? Do you feel that relevancy boosts offered through vertical search (via your Slashtags) allows you guys to provide a similar or better experience without needing as large of an index?

Our crawler tends to go into highly ranked sites more deeply than poorly ranked sites. We have a 3 billion page crawl, and so we need to choose the best content to include. This starts at crawl time - should we crawl this url or that url? There are a whole set of heuristics which drive what crawl budget an individual site gets.

The web keeps getting deeper and deeper - the challenge is how to return the good stuff and not sink. This is why we believe human curation needs to be brought back to search. Only by curating the best content in every vertical can the most relevant results be returned.

Amongst SEOs the issue of "brand" as a relevancy signal has been a topic of heated debates. How important do you feel brand is as a signal of relevancy & authority?

One of the things we look at is how natural the pattern of mentions of a site looks. Real brands tend to have a natural pattern of mentions on the web.

You had a blog post a few years back titled "PageRank wrecked the web." How do you feel about paid links? What editorial actions do you guys take when you find paid links?

If links have an economic value, they're going to be bought and sold. It's that simple. What happens in our ranker is that we classify different sources of signals, and then let the machine learning figure out what the signal is telling us. Is this a good source of anchortext? Or maybe a certain class of links even has a negative contribution to rank, if what the links are telling us doesn't correlate with the direction we want the ranker to be going.

How hard is it to detect paid links? What has been the most challenging part of launching a world class search engine?

The whole thing has been hard. Search has so many sub-components, and even things that sound trivial like DNS turn into big projects when you need to scale them up to billions of web pages.

---

Thanks Rich! Be sure to check out blekko. You can follow them on Twitter & read Rich's musings on the web, search, and marketing at Skrentablog.

Localization, Unique Data Sets & the Future of Search

Local is Huge

Google's US ad revenue is roughly 15 billion & the size of the US Yellow Pages market is roughly 14 billion. Most of that money is still in print, but that shift is only accelerating with Google's push into local.

Further, cell phones are location aware, can incorporate location into search suggest, and on the last quarterly conference call Google's Jonathan Rosenberg highlighted that mobile ads were already a billion Dollar market for Google.

Google has been working on localization for years, and as a top priority. When asked "Anything you’ve focused on more recently than freshness?" Amit Singal stated:

Localization. We were not local enough in multiple countries, especially in countries where there are multiple languages or in countries whose language is the same as the majority country.

So in Austria, where they speak German, they were getting many more German results because the German Web is bigger, the German linkage is bigger. Or in the U.K., they were getting American results, or in India or New Zealand. So we built a team around it and we have made great strides in localization. And we have had a lot of success internationally.

The Big Shift

I have been saving some notes on the push toward local for a while now, and with Google's launch of the new localized search results it is about time to do an overview. First here is Google's official announcement, and some great reviews from many top blogs.

Some of the localized results not only appear for things like Chicago pizza but also for single word searches in some cases, like pizza or flowers.

Promoting local businesses via the new formats has many strategic business benefits for Google

  • assuming they track user interactions, then eventually the relevancy is better for the end users
  • allows local businesses to begin to see more value from search, so they are more likely to invest into a search strategy
  • creates a direct relationship with business owners which can later be leveraged (in the past Google has marketed AdWords coupons to Google Analytics users)
  • if a nationwide brand can't dominate everywhere just because they are the brand, it means that they will have to pony up on the AdWords front if they want to keep 100% exposure
  • if Google manages to put more diversity into the local results then they can put more weight on domain authority on the global results (for instance, they have: looked at query chains, recommended brands in the search results, shown many results from the lead brand on a branded search query, listed the official site for searches for a brand + a location where that brand has no office, etc.)
  • it puts eye candy in the right rail that can make searchers more inclined to look over there
  • it makes SEO more complex & expensive
  • it allows Google to begin monetizing the organic results (rather than hiding them)
  • it puts in place an infrastructure which can be used in other markets outside of local

Data Data Data

Off the start it is hard to know what to make of this unless one draws historical parallels. At first one might be inclined to say the yellow page directories are screwed, but the transition could be a bit more subtle. The important thing to remember is that now that the results are in place, Google can test and collect data.

More data sources is typically better than better algorithms, and Google has highlighted that one of their richest sources of data is through tracking searcher behavior on their own websites.

Pardon Me, While I Steal Your Lunch

There are 2 strong ways to build a competitive advantage on the data front:

  • make your data better
  • starve competing business models to make them worse

Off the start yellow page sites might get a fair shake, but ultimately the direction they are headed in is being increasingly squeezed. In a mobile connected world with Google owning 97% search marketshare, while offering localized search auto-complete, ads that map to physical locations, and creating a mobile coupon offers network, the yellow page companies are a man without a country. Or perhaps a country without a plot of land. ;)

They are so desperate that they are cross licensing amongst leading competitors. But that just turns their data into more of a commodity.

Last December I cringed when I read David Swanson, the CEO of R.H. Donnelley, state: "People relate to us as a product company -- the yellow-pages -- but we don't get paid by people who use the yellow-pages, we get paid by small businesses for helping them create ad messages, build websites, and show up in search engine results. ... Most of the time today, you are not even realizing that you are interacting with us."

After seeing their high level of churn & reading the above comment, at that point I felt someone should have sent him the memo about the fate of thin affiliates on AdWords. Not to worry, truth would come out in time. ;)

Making things worse, not only is local heavily integrated into core search, with search suggest being localized, but Google is also dialing for Dollars offering flat rate map ads (with a free trial) and is testing fully automated flat rate local automated AdWords ads again.

Basic Economics

How does a business maximize yield? Externalize costs & internalize profits. Pretty straightforward. To do this effectively, Google wants to cut out as many middle men out of the game as possible. This means Google might decide to feed off your data while driving no traffic to your business, but rather driving you into bankruptcy.

Ultimately, what is being commoditized? Labor. More specifically:

  • the affiliate who took the risk to connect keywords and products
  • the labor that went into collecting & verifying local data
  • the labor that went into creating the editorial content on the web graph and the links which search engines rely on as their backbone.
  • the labor that went into manually creating local AdWords accounts, tracking their results, & optimizing them (which Google tracks & uses as the basis for their automated campaigns)
  • the labor that went into structuring content with the likes of micro-formats
  • the labor that went into policing and formatting user reviews
  • many other pieces of labor that the above labor ties into

Of course Google squirms out of any complaints by highlighting the seedy ends of the market and/or by highlighting how they only use such data "in aggregate" ... but if you are the one losing your job & having your labor used against you, "the aggregate" still blows as an excuse.

But if Google drives a business they are relying on into bankruptcy, won't that make their own search results worse?

Nope.

For 2 big reasons:

  • you are only judged on your *relative* performance against existing competitors
  • after Google drives some other players out of the marketplace and/or makes their data sets less complete, the end result is Google having the direct relationships with the advertisers and the most complete data set

The reason many Google changes come with limited monetization off the start is so that people won't question their motives.

Basically I think they look at it this way: "We don't care if we kill off a signal of relevancy because we will always be able to create more. If we poison the well for everyone else while giving ourselves a unique competitive advantage it is a double win. It is just like the murky gray area book deal which makes start up innovation prohibitively expensive while locking in a lasting competitive advantage for Google."

You would never hear Google state that sort of stuff publicly, but when you look at their private internal slides you see those sorts of thoughts are part of their strategy.

What is Spam?

The real Google guidelines should read something like this:

Fundamentally, the way to think about Google's perception of spam is that if Google can offer a similar quality service without much cost & without much effort then your site is spam.

Google doesn't come right out and say that (for anti-trust reasons), but they have mentioned the problem of search results in search results. And their remote rater documents did state this:

After typing a query, the search engine user sees a result page. You can think of the results on the result page as a list. Sometimes, the best results for "queries that ask for a list" are the best individual examples from that list. The page of search results itself is a nice list for the user.

...But This is Only Local...

After reading the above some SEOs might have a sigh of relief thinking "well at least this is only local."

To me that mindset is folly though.

Think back to the unveiling of Universal search. At first it was a limited beta test with some news sites, then Google bought Youtube, and then the search landscape changed...everyone wanted videos and all the other stuff all the time. :D

Anyone who thinks this rich content SERP which promotes Google is only about local is going to be sorely disappointed as it moves to:

  • travel search (Google doesn't need to sell airline tickets so long as they can show you who is cheapest & then book you on a high margin hotel)
  • any form of paid media (ebooks, music, magazines, newspapers, videos, anything taking micro-payments)
  • real estate
  • large lead generation markets (like insurance, mortgage, credit cards, .edu)
  • ecommerce search
  • perhaps eventually even markets like live ticketing for events

Google does query classification and can shape search traffic in ways that most people do not understand. If enough publishers provide the same sorts of data and use the same types of tags, they are creating new sets of navigation for Google to offer end users.

No need to navigate through a publisher's website until *after* you have passed the click toll booth.

Try #3 at Reviews

Google SearchWiki failed in large part because it confused users. Google launched SideWiki about a year ago, but my guess is it isn't fairing much better. When SideWiki launched Danny Sullivan wrote:

Sidewiki feels like another swing at something Google seems to desperately desires — a community of experts offering high quality comments. Google says that’s something that its cofounders Larry Page and Sergey Brin wanted more than a system for ranking web pages. They really wanted a system to annotate pages across the web.

The only way they are going to get that critical mass is by putting that stuff right in the search results. It starts with local (& scrape + mash in other areas like ecommerce), but you know what they want & they are nothing if not determined to get what they want! ;)

Long Term Implications

Scrape / mash / redirect may be within the legal limits of fair use, but it falls short in spirit. At some point publishers who recognize what is going on will align with better partners. We are already seeing an angry reaction to Google from within the travel vertical and from companies in the TV market.

Ultimately it is webmasters, web designers & web developers who market and promote search engines. If at some point it becomes consensus that Google is capturing more value than they create, or that perhaps Google search results have too much miscellaneous junk in them, they could push a lot more searchers over to search services which are more minimalistic + publisher friendly. Blekko launches Monday, and their approach to search is much like Google's early approach was. :)

Who Benefits From Following Google's Guidelines?

Some of you may have been hit by Google's 20 October algorithm change.

And some of you wouldn't have noticed any difference.

On 20 October, a number of sites got trashed. Rankings, and traffic, plummeted through the floor. The webmaster forums lit up. Aaron noticed it. I noticed it. Yet, other webmasters wondered what all the fuss was about.

As many of you know, there is not just one ranking algothimn. There are many algorithms. What affects one site may not affect another. Rather interestingly, Google's John Mu dipped into this thread on Google's support forum, offering these words of wisdom (HatTip: Barry)

It looks like the changes you're seeing here may be from an algorithmic change. As part of our recent algorithmic changes (which the outside world sometimes refers to as the "May Day update" because it happened primarily in May), our algorithms are assessing the site differently. This is a ranking change, not any sort of manual spam penalty, and not due to any technical issues with regards to crawling or indexing your content. You can hear more about this change in Matt's video: "

...and....

Various parts of our algorithms can apply to sites at different times, depending on what our algorithms find. While we initially rolled out this change earlier this year, the web changes, sites change, and with that, our algorithms will continually adapt to the current state on the web, on those sites. While it might be confusing to see these changes at the same time as this issue, they really aren't related, nor is this a general algorithm change (so if other sites have seen changes recently, it probably doesn't apply to them as well).

Matt's video, made four months ago, was talking about the algorithmic MayDay change. John Mu adds: "Various parts of our algorithms can apply to sites at different times" In other words, whatever happened in May may not affect your site in May, or June, or July, but might hit you many months later. This implies that your site may trip a threshold, and be judged quite differently than it was the day before.

This still doesn't completely explain why so many sites were hit on the same day, but then Google don't typically explain things in detail.

To complicate matters, there was an acknowledged indexing problem, affecting new content, particularly on blogs. Again, John appears to suggest this was a separate issue.

Forget About Search Engines, Just Publish

Now, all SEOs are used to algorithm changes. Nothing new. But this one has me genuinely perplexed, mainly because of the type of sites that got hit.

Time for some self-searching Q&A about one of my own sites:

Q: So, how many links did you buy?
A: None.
Q: Are you selling links?
A: Nope.
Q: Linking to "bad neighborhoods"?
A: Not that' I'm aware of.....
Q: Did you link-build in an aggressive manner?
A: No. I did no link building, whatsoever.
Q: Huh?
A: That's not a question.
Q: So you just published content?
A: Right.
Q: And people linked to your site, of their own accord?
A: Yep. I guess they liked it.
Q: Was your content heavily SEO'd?
A: No. In fact, I gave writers specific instructions not to do anything resembling "SEO copywriting". It ruins the flow for readers.
Q: All original content?
A: All original. Hand written. No machines involved anywhere.
Q: So this site conforms to Google's Webmaster Guidelines?
A: I'd say it lies well within them. "Be useful to end users", was the guiding principle.

Yet it got hit hard.

What's also interesting is the nature of the sites that replaced it. I checked keyword after keyword, and found script driven, aggressive black-hat, content-free sites in top positions. Not in all cases - there are certainly useful sites that deserve to be there, and deserve to appear above mine. Fair play. However, there were plenty of sites of - shall we say - dubious merit- occupying high positions.

Curious.

Be Useful. Perhaps

Now, I believe in publishing useful, unique content, and not paying too much attention to SEO, other than covering the basics. SEO is one strategy amongst many, and sites should, first and foremost, prove useful to people.

Clearly, no site is immune. You can stay within Google's Webmaster guidelines, and get taken out. I knew that anyway, but when the sites that don't follow the guidelines replace you...

....I'll admit - it grates.

Presumably, Google rewards the sites it likes with high rankings, and if we see a lot of aggressive sites filling the top page, should we therefore assume that aggressive sites are what Google actually wants?

I'd like to think not.

Perhaps they are just trying to mess with our heads?

Or they messed up?

Or the changes are still bedding in?

Or they really do want it this way?

I'm still watching, and considering. Perhaps the site will just pop back up in due course. Or perhaps I need to go back to the drawing board. I'll let you know how I get on.

If you've noticed something similar on your sites, chime in on the comments.

Pages