Get Listed is a cool tool for seeing how your website looks in local search, and to aid you in submitting your site to local search engines.

Wordtracker announced they will be launching a new version of their keyword tool soon.

FairShare helps you track scrapers publishing your content.

WebReader makes it easy for people to click a button that makes your blog speech enabled.

Wiep highlighted a new link building and public relations tool called BuzzStream.

SEOmoz announced they retooled their backlink anchor text analysis tool using their Linkscape data, which they also released an API for.

Majestic SEO created a neighboring sites search.

Wolfram Alpha is to launch in May as an answer engine. I predict that launch strategy will create branding issues (like not being able to outsource the blame for their poor algorithms onto spammers - like Google does) but if the tool is decent it might be a great tool to use when researching content generation ideas. The new CashKeywords toolbar also looks quite useful for researching content ideas.

In January Google started offering a new, more-advanced sitemap generator, and made Jaiku (a Twitter-like tool) open source.

Mozilla is testing collecting usage data to create open research, hopefully they don't pull an AOL with the data though.

Some developers are creating new tools to analyze Wikipedia.

One of the first books I read about the web which really helped me understand the culture of the web and the concept of the web as a social network was the Cluetrain Manifesto. In it, David Weinberger stated "hyperlinks subvert hierarchies," a concept that helps explain a lot of the chaos in the current world.

• We see how poorly our leaders act (or sometimes don't act, even when they should) and it lowers our confidence in leaders.
• Our memories are extended by linking together recent events with the not-so-distant past, done by millions of reporters (and recorders) around the world. Seemingly unrelated pieces are stitched together by our shared experiences.
• The financial industry collapsed quicker than experts expected. The mainstream media is collapsing quicker. And they can't help but out-gloom and doom each other. Job losses accelerate.

The staggering rate of change and seeing cracks in imperfect structures makes us more likely to question authority. Fear slows down economic activity but it also creates the conditions to help speed up change through creative destruction, as insolvent structures crumble and are replaced by thousands and millions of online experiments ran in parallel due to forced entrepreneurship. As Clay Shriky puts it:

That is what real revolutions are like. The old stuff gets broken faster than the new stuff is put in its place.

Each link creates a new opportunity, which in turn creates new opportunities, giving our social nervous system many senses. And the web is just getting started. Watch this Tim Berners-Lee video and try to predict the future of the web. You can't do it.

As the web grows (and grows smarter) two of the biggest risks are machines learning too much about us (through spying on our browsing habits) and proprietary databases that lock away pieces of our culture while surfacing other favorable pieces (the divisions could be nationalistic, idealistic, or commercially driven - like "brands" that we are apparently "hard wired" to). For both of those reasons, Google's market dominance scares me.

On the above video by Tim Berners-Lee, Ralph Tegtmeier wrote the following

As long as we don't seriously do something about protecting people from the very abuse of their personal data (more often than not linked without their express acquiescence), we're merrily lighting the fuse to a humungous collective powder keg. (And it's really not helpful at all shrugging such concerns off with pejorative epitheta such as "tin foil hat", "conspiracy theories" etc. as is so common across the board.

Let's never forget that all the major atrocities committed in "civilized" countries ever since the 19th century, ranging from genocide to mass destruction, ethnic cleansings, wars, the holocaust etc. were only as scalable as they eventually proved to be because of just that: "linked data" ruthlessly leveraged and deployed by those who could get their hands on it.

Think about how distributed (and targeted) ad based business models work in a republic / quasi-democracy. Buy ads that change the opinion of couple percent of people and you change the course of a country, and perhaps the course of civilization. Think about how well Google intends to know your flaws, and sell them off to the highest bidder in any medium they can:

users that spend a long time bartering instead of stealing in a game may suggests that they are interested in the best deals rather than the flashiest items so the system may show ads reflecting value. As another example, users that spend a lot of time exploring suggest that they maybe interested in vacations, so the system may show ads for vacations. As another example, users that spend a lot of time chatting instead of fighting or performing other activities in online games suggest that they like to chat, so the system may show ads for cell phones, ads for long distance plans, chat messengers, etc.
...
The dialogue could indicate that the player is aggressive, profane, polite, literate, illiterate, influenced by current culture or subculture, etc. Also decisions made by the players may provide more information such as whether the player is a risk taker, risk averse, aggressive, passive, intelligent, follower, leader, etc. This information may be used and analyzed in order to help select and deliver more relevant ads to users.

And while we are being profiled, pieces of our culture are being locked up via anti-competitive agreements. Richard Sarnoff, the chairman of the Association of American Publishers, noted how they were hoodwinked in a deal with the devil:

Sarnoff also speculated that … [l]egal hurdles may make it infeasible for any other firms to build a search engine comparable to Google Book Search.

Many power structures that are being killed off by the web are the walking wounded, making deals that are rational only when paired against death. And we are stuck living with the consequences of those decisions.

With Google being so profit driven they are leaving room for a pure search play, if only someone that got branding, marketing, and the web would step up. I hope Rich Skrenta (or anyone) provides real competition to Google soon, before spying is seen as respectable and too much of our culture gets locked up in exclusive deals.

Phorm, a UK company that partnered with BT to run secret trials to target ads based on usage data, was roasted by the media with article titles like Phorm’s All-Seeing Parasite Cookie.

Google, which has long stayed away from behavioral targeting due to privacy (and negative publicity) concerns, announced they are jumping into the behavioral ad targeting market:

Google will use data it collects about what Web sites users visit and what it knows about the content of those sites to sort its massive audience of users into groups such as hockey fans or travel enthusiasts. The data won't be drawn from users' search queries, but from text files known as cookies that Google installs on the Web browsers of users who visit pages where it serves ads.

DoubleClick, AdSense, Google Toolbar, Gmail, Youtube, Blogger, Google Groups, Google Checkout, Google Chrome, Google Analytics...there are lots of ways to track you, even if you do not want to be tracked. Google will allow users to opt out of such targeting, with yet another cookie, but if you clear cookies then you are back in the matrix again.

And while Google claims they are not using search queries in their current behavioral targing, Danny Sullivan wrote:

Google confirmed in a session I moderated at the Omniture Summit last month that they have tested behaviorial targeted ads using past search history data. Again, that doesn’t seem to be part of this release, but it could come in the future.

As discovered during early Google research titled The Anatomy of a Large-Scale Hypertextual Web Search Engine:

we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.

Indeed. Tim Berners-Lee, creator of the world wide web, spoke out against behavioral targeting:

"People use the web in a crisis, when wondering whether they have a sexually transmitted disease, or cancer, when wondering if they are homosexual and whether to talk about it … to discuss political views."
...
"The power of this information is so great that the commercial incentive for companies or individuals to misuse it will be huge," he said. "It is absolutely essential to have absolute clarity that it is illegal."

If Google continues down this path unquestioned, then in due time you may not be able to get health insurance because of a web page you viewed or a keyword you trusted Google enough to search for. Better luck next life!

when I got on the web one of the first mistakes I made was trying to go after the cheap traffic on the second and third tier networks. I arbitraged one of them and was pulling a 300% ROI selling them back their own traffic - but the wankers never paid me a cent.

It can be appealing to think of how to do things cheaper...and sure in the short term it might make sense to do something half-way to get it up and going and then to keep making incremental improvements. But people like Philip M. Parker have created automated technologies to write books. Cheaper is a hard way to compete in the content business.

If you are just fishing for nickels and do not intend to take the market head on then any of the following can take you out:

• algorithm updates
• remote quality rater or spam report
• self-sabotage through doing something a little too clever
• more established niche competitors accidentally or intentionally copying you
• large general competitors accidentally or intentionally copying you

With search many early successes will be longtail keywords, but eventually you want to go after the biggest and best that you can achieve. Why? Once you have status you enjoy cumulative advantage in everything else you do. Things that are somewhat remarkable become quite remarkable just because who is doing them.

It can take a long time to work yourself to the top of a hierarchy. Most people who succeed ignore the hierarchy and look for a way to dominate a related niche

Ideally, a new player wants to come in with a fresh approach that doesn’t necessarily threaten the existing hierarchy. This allows you to develop an audience by sharing with existing players, not necessarily competing with them.

What you’re looking to do is intensify the niche by doing something more, or differently (or maybe even better) than the existing players. You do this by first evaluating and understanding where the niche is currently, and position your content in a way that pushes the envelope.

Unless you are really well established there is a lot of uncertainty in what you do online. Each additional investment can seem like you are getting closer to the point of diminishing returns, wasting your time. But then surprisingly one day things go way better than expected, and things are received much better than expected. You get a dopamine rush and the sun shines a bit brighter. Network effects kick in and you have reached the point of increasing returns - where each $ invested returns 10s or 100s of dollars back. I think Seth refers to this concept as The Dip - its what separates market winners from people who wasted their time.

But you usually have to lose $50,000 to $100,000 in sweat equity to get to that point, at least on your first successful project. The good news is that once you have already done that work nobody (except for you) can really take it away from you. Even if Google or some other market maker does not like you then you still have other social leverage and exposure which can be used to help generate revenue, launch new websites and projects, or (God forbid) get a real job.

I posted the cartoon version of this post about a month ago.

I have been a follower of Jonathan Mendez's Optimize and Prophesize for a while, and recently interviewed him.

At SES in New York you are speaking on a panel titled "search becomes the display OS" - what does that mean, why is this shift happening?

The shift is part of the Darwinian evolution of the web. Many people have mistakenly viewed search as a channel when in reality it is a behavior. It is the way people use the web. This is clear as YouTube is now the #2 search engine, Facebook, eBay & Craigslist are in the top 10 search engines and Twitter is trying to position itself as a real-time search. Search is integral to the web experience.

From the display standpoint we need to keep in mind that this medium does not need ads to support it nor are ads part of the experience. Display advertising was built as a parallel platform - not weaved into the web like search but placed on top of it. Display has always had its own ecosystem of real estate, content and serving that is separate from the public web.

What we’ve witnessed with display’s lackluster performance and the inevitable crash of CPM rates is the idea of it being a stand-alone platform was wrong. Display needs to be an application that is integrated with web platform and the way people use the web. It should be based on user controls and rules based delivery of content. To truly be relevant and useful for people, publishers and advertisers, it must become a web service like search.

Search is currently at the center of the web. Do you foresee any technologies or services that might shift its position?

On the contrary I think it will become more entrenched and important since everyday millions of new pieces of content data keep getting added to the web and older content gets digitized. As I mentioned search is basic human behavior. We all go online with a goal in mind to either recover information and content or discover information and content. Those behaviors are primal. No technology or services will shift them.

How do you place value on a search impression?

The value is based on what you do with the information. Impressions are the ultimate arbiter of interest and demand. Of course if you go to Google trends often you will become somewhat worried about the collective psyche of this country. In all seriousness however, this is business intelligence. Quick story about impressions - a few years ago I was working with a big client and they were launching a new product. We had purchased the category kw for this product over a year and it hardly had any impressions. We strongly advised them against spending two million dollars to launch this product because there was no demand for it. They didn’t listen to the “search” guys. Within a year the CMO was fired because the product flopped. So in that instance I would say those couple hundred impressions were worth two million dollars.

One of the most powerful pieces of search is that the ad unit looks just like the content. What can publishers do to maximize ad integration without risking their perceived credibility?

In my experience you add credibility as a publisher if you provide helpful, useful and interesting content. There’s no reason that can’t be an ad. Most everyone I know has clicked on a Google ad. Sometimes it is preferable. This creates value to Google as a publisher. Ads that are helpful and interesting will add value to other publishers in the same manner because they are helping their visitors. People rarely forget who helped them in a useful way whether it be a website or “in real life.” In fact there is a large intangible value that is not even being captured when this happens. I think some people even refer to it as branding.

What can publishers and vendors learn from the dominance of search when thinking about how to build and brand their websites? What are some easy ways to make our user experiences more relevant?

Give people control over the delivery of content. The most successful online segmentation strategy is when a person tells you what they want -- self-segmentation. That is the beauty of search. The keyword is the ultimate expression of people’s goals. No website or advertiser knows more about what I want than I do! It explains why the best and most successful experiences on the web (Google, eBay, Craigslist, Yelp) have query fields and lots of text links and it is something I always keep in mind in doing page design. As far as branding I think that goes back to what we were just talking about, the site experience. Great experiences build brands and that is the same online as well as off. Keep in mind all of this should be tested and optimized. It is no accident that Google is the #1 brand in the world without spending a penny on advertising. From day one no one has tested online experience more than Google.

Many people have been promoting Twitter as a Google-killer in real-time search. Why are they wrong?

You mean besides the fact that Google made $21B in ad revenue last year has $8B in cash, owns half a million servers and Twitter search has probably 10 employees and no revenue?

There are some major problems with RTS. First let’s start with the way people search. This has been studied and very clearly defined over the years by brilliant people like Andrei Broder, Daniel Rose and others. I recently took the query classifications they defined and applied it to RTS (http://www.optimizeandprophesize.com/jonathan_mendezs_blog/2009/02/misguided-notions-a-study-of-the-value-creation-in-realtime-search.html). I came to the conclusion that with optimal RTS - which is a huge challenge as I’ll get to - that less than 20% of all queries would benefit in anyway from RTS.

As far as the technical challenges spamming would be very hard to filter in real-time. Also authority as we know from PageRank is a fundamental driver of relevance. How do you define authority in real time? If you do not rank results than is it just a noisy stream? I’ve come to the conclusion that if it RTS becomes anything useful it will be a search vertical, like travel. Helpful for certain things but nowhere near a primary search tool. It is still a great addition to the web but not something Google needs to be concerned with. In fact I think Google is in the position to provide RTS for the entire web which is much more useful than RTS for a single app.

How slow and painful will the transition of ad dollars from offline to online be? What will be the catalyst that allows ad agencies to push search and online aggressively?

Very slow, but this shouldn’t be painful. We know the attention is online so dollars will continue to increase but I think a $25 billion dollar online industry is pretty good right now. It’s grown much faster the past few years than even the most bullish forecasts from five years ago. The catalyst will be innovation and the businesses themselves that must demand performance. Bill Gross the inventor of PPC said it best, “the true value of the Internet is in its accountability…performance guarantees have to be the model for paying for media.” As soon as we embrace performance for all advertising, even so called brand advertising, we will prove our value and grow our industry. Google stands as proof of concept for this. But the battle over performance will be long and bloody. In just the past couple of weeks we’ve had groups like the IAB and the AAAA speaking out against performance and metrics. This type of rhetoric and their fear of accountability are actually helping to slow down the transition.

How many newspaper companies do you see lasting through this economic downturn?

Not too many. Besides the fact that their authority over the past years has waned with bloggers and false reporting the real problem is that newspapers are not an efficient means of information compared to everything else we have today. What percentage of the paper is relevant or interesting to you? 5%? 15%? Yet you are paying for the entire paper when you buy it. Doesn’t make sense. We used to have town criers too, but then newspapers came along. I don’t think most people will miss them. Times have changed. Maybe we’ve just come full circle – people getting their news from other people they trust is the best way to disseminate information. Who trusts the papers?

What will the online vs offline divide look like in 2 years? 5 years? 10 years?

I’m not so sure we’ll have a divide in 5 or 10 years. The kids graduating high school this year were 8 years old when Google was started. I see kids 4 and 5 years old naturally manipulating iPhones. Many of us have persistent web connection and we like it - we feel uncomfortable without it! Of course it’s nice to get off the grid sometimes but what is happening with digital technology is the great story of our age. Everything is becoming digital, addressable and connected via the web. All of us lucky enough to be working here will reap the rewards of that in the coming years because the growth of digital will far outpace the amount of talent in the workforce. We should have bigger paychecks in 5 years!

Many people focus on one particular segment of the market, whereas you seem to have a well-rounded knowledge of SEO, PPC, user experience, and conversion strategies. How did you find the time to tie all these different disciplines together?

Well, I’ve been at it 11 years so that accounts for the time. It is corny but I love the web and I am passionate about trying to make it more relevant to everything we do. Looking back my career path from Site>Email>SEO>UX>SEM>LPO>Display, it seems like a very natural progression to me. Basically, with one stop for UX I have just been a marketer trying to stay ahead of the advances in marketing technology. Also, I love learning how people use the web and all the disciplines I have worked in are fundamentally rooted in the same thing -- understanding people’s goals and optimizing the delivery and presentation of information to meet those goals. As an industry we tend to divide the web into vectors but we often lose sight that the web experience for people is linear. The more holistic understanding we have generally the better our results.