The New York Times has an interesting article about Cisco's roll in the Great Firewall of China + more:

Cisco, the maker of Internet routing gear, customized its technology to help China track members of the Falun Gong spiritual movement, according to a federal lawsuit filed last week by members of the movement.

The lawsuit, which relies on internal sales materials, also said that Cisco had tried to market its equipment to the Chinese government by using inflammatory language that stemmed from the Maoist Cultural Revolution.

And that from a company which promotes itself using the label "the human network."

And how did Cisco react when the above information became public? "When evidence of the company’s activities in China became public in 2008 through a leaked PowerPoint presentation, Cisco disassociated itself from the marketing materials, stating that they were the work of a low-level employee."

That is what big brands do. The PR team steps in and says "Oops it was a rogue marketer/trader/monkey/employee who was smoking crack at work and they have now been fired. We were ignorant of our actions but we really care about people. We promise to not (get caught) doing it again!" TM

As Google pushes to make the web more corporate, it is worth taking a step back and considering what that means for "the human network."

Google likes to pretend that something is good just because it is a big brand, but many big brands have big ad budgets *precisely* because their business model contains hidden costs. For instance: bad faith insurance which takes your money as long as you pay & then disappears the minute something goes wrong.

The legal system granted large corporations more rights than human beings. Not because they are any better, but because they are more corrupt. I bet many Google engineers are disappointed to see Google following suit & taking the easy way out. Spy & personalize. And when in doubt, brand, brand, brand. ;)

With the vast potential of the web should we settle for making it as corrupt (or more corrupt) than the real world?

The following song is brought to you by the Facebook "like" spy button. ;)

Facebook is a Sleazy Organization

Facebook recently hired the PR firm Burson-Marsteller to plant a Google smear campaign in the media:

Somebody, it seems, hired Burson-Marsteller, a top public-relations firm, to pitch anti-Google stories to newspapers, urging them to investigate claims that Google was invading people’s privacy. Burson even offered to help an influential blogger write a Google-bashing op-ed, which it promised it could place in outlets like The Washington Post, Politico, and The Huffington Post.

And why would Facebook run such a campaign?

Confronted with evidence, a Facebook spokesman last night confirmed that Facebook hired Burson, citing two reasons: First, because it believes Google is doing some things in social networking that raise privacy concerns; second, and perhaps more important, because Facebook resents Google’s attempts to use Facebook data in its own social-networking service.

So now Facebook is trying to position itself as an advocate of consumer privacy rights?

Seriously?

• Is this the same Facebook that had a class-action lawsuit filed against it for Beacon?
• Is this the same Facebook which stood hand-in-hand with Google AGAINST user privacy?
• Is this the same Facebook where the CEO spoke of his users with the flowery language of dumb f*cks "trust me"?

Yes. Yes. Yes.

The bottom line is this: Facebook is a sleazy organization.

Google is a Sleazy Organization

The above Facebook complaint sounds like the same complaints that came from the old media powers which Google used high power lawyers to steamroll over.

How can Facebook be surprised with Google entering a new vertical by not respecting the property rights of existing market participants? It has been Google's approach to virtually everything:

• trademark (selling branded keywords for sketchy uses with 50,000 AdWords advertisers selling knock-off goods)
• copyright (paying scrapers to steal your content & wrap it in AdSense)
• video (Youtube & Google Video)
• ebooks (illegal scanning)
• etc.

So far Google has only fell flat on their face once: when they challenged the pharmaceutical corporations:

Google Inc. is close to settling a U.S. criminal investigation into allegations it made hundreds of millions of dollars by accepting ads from online pharmacies that break U.S. laws, according to people familiar with the matter.

The pharma corporations are powerful & are in bed with the government. In spite of repeated felonious behavior in marketing their drugs for illegal off label use (which has literally murdered millions of people) these companies can have the government step in and protect their property rights, by having the government enforce unto others the same laws that these same pharma corps regularly break (literally killing millions of people).

Maybe Google is Philosophically Opposed to Property Rights?

Yes, but only when convenient!

Everyone *but* Google should be open.

While Google tramples on the property rights of everyone else, the first sniff of someone operating anything like they do drives Google into black-ops mode & they conduct a smear campaign. Google launched Buzz without warning, but when their feared Facebook was collecting more personal information than they could Google went into black ops PR mode warning against security issues in Facebook.

Remember that bogus "Bing is copying our results" stuff Google engineers did earlier this year? Google later rolled out their content farm update & many of the sites which were torched by Google are now getting more traffic from Bing. What does that tell us? If Bing was putting *any* significant weight at all on Google rankings & traffic then why didn't that carry any weight when Google torched a bunch of websites?

Here is the Google traffic profile for a site that was torched by Panda

And that same website's Bing traffic

Google traffic fell through the floor, while Bing traffic kept climbing. Some sites that were hit by Panda are getting more visits from Bing or Yahoo! Search than from Google.

Conclusion? Once again Google distorts media to promote itself & its business interests, while bogusly smearing competitors with fabricated trash.

Part of why Microsoft's search marketshare is less than Google's is that Microsoft is willing to block sleazy traffic partners, unlike Google. But Google's treatment of their partners is inconsistent. Using "inside voices" Googlers openly explain in plain English how they treat their partners: "we are using compatibility as a club to make them do things we want" - Google's Dan Morrill.

Big Companies Hate Honest Market Innovation

Large companies are largely counter to honest innovation in the marketplace. They are comfortable atop the perch and want to lock down innovation to maintain their current dominance.

Sure the big banks welcomed CDOs, MERS, etc. ... but those were welcomed precisely because they were part of an elaborate scheme of dishonesty and fraud. But the same society which brings us CDOs built on fraud (that ultimately cost you your job, your house, your retirement savings, the value of the currency, etc.) is also a society where dirty corporate whores push to force smaller market competitors to be entirely transparent.

This stuff is literally everywhere. Consider this: Major Record Labels Forced to Pay $45M USD for Pirating Music. Once again, property rights are only important when they are forcing their own rights, but they are willing to walk on the rights of others. Consider the actions of MarkMonitor, yet another seedy Google partner:

I have for years been telling you even if you have no interest in the new gTLD’s you had to pay close attention to the process as whatever rules come out of that process will be attempted to be applied to all existing TLD’s including .com, .net and .org.

This is especially troubling because as you know the new gTLD process has not even been approved yet since the .Net contract is up for renewal, trademark groups are going to push for this new system to take away domains, be imposed on .net

The very domain name of the front organization that is pushing to remove domain privacy is registered using a private registration. ipconstituency.org uses Domain Discreet!

Read this piece on Google & Skyhook and ask yourself if Google is actually open & is promoting or suppressing market innovation.

Small Businesses Typically Can't Act Sleazy

Try getting customer service from Google & you will quickly find yourself in a hall of mirrors. Compare that to the customer service you get from a small company. Sure some small companies may decide they have no interest in supporting freetards, but if you are actually a paying customer you will usually be treated well by small companies because word of mouth marketing is the most important lead channel for many small businesses.

When a consumer or small business owner gets caught (acting like a big business, and) doing something illegal they go to jail. When a big business repeatedly commits serious crimes the wost thing that could possibly happen is a shake up of management. A company has no soul. A corporation can't go to jail.

This is precisely why Google's corporate-first approach to relevancy is bad.

Soon after the Facebook/Google story broke a friend of mine told me they put “facebook smear of google” in Google & they got:

• Image result = Globe and Mail
• Number one result = Huffington Post
• Number two result = TechCrunch (top websites are both AOL properties)
• Number three result = Get more results from the past 24 hours
• Number four result = The Daily Beast – better known as the site that broke the story.
• All other results are a retelling and mashup of the original.

The big publishers complained that smaller sites were stealing their stories. Google made secret arrangements with the Online Publishers Association & now the big companies get to rank at the top of the search results for stories that they stole from smaller outlets.

While small players are desperately fighting against each other for scraps off the table, the pawns have been driven out of the search ecosystem.

All webmasters are equal but some webmasters brands are more equal than others

Society hierarchy has been restored.

Don't be evil, just be corporate.

How to Handle Duplicate Content

Here is a fun webmaster help video from March 10th of 2010, answering the following question:

"If Google crawls 1,000 pages/day, Googlebot crawling many dupe content pages may slow down indexing of a large site. In that scenario, do you recommend blocking dupes using robots.txt or is using META ROBOTS NOINDEX,NOFOLLOW a better alternative?"

The answer kinda jumps around a bit, but here is a quote:

I believe if you were to talk to our crawl and index team, they would normally say "look, let us crawl all the content, we'll figure out what parts of the site are dupe (so which sub-tree are dupes) and we'll combine that together.

Whereas if you block something with robots.txt we can't ever crawl it, so we can't ever see that its a dupe. And then you can have the full page coming up, and then sometimes you'll see these uncrawled URLs where we saw the URL but we weren't able to crawl them and see that its a dupe.
...
I would really try to let Google crawl the pages & see if we can figure out the dupes on our own.

Trust in GoogleBot

The key point here is that before you consider discarding any of your waste you should give GoogleBot a chance to see if they can just figure it out on their end. Then, without updating said advice, Google rolled out the Panda update & torched 10,000's of webmasters for following what was up to then a Google best practice. Only after months of significant pain did Google formally suggest on their blog that you should now block them from indexing such low value pages.

Matt's video also suggested some of the other work around options webmasters could do (like re-architecting their site or using parameter handling in Webmaster Tools), but made it sound like Google getting it right by default was anything but an anomaly. What such advice didn't take into account was the future.

What Does a Search Engineer Do?

The problem with Google is that no matter what they trust, it gets abused. Which is why they keep trying to fold more signals into search & why they are willing to make drastic changes that often seem both arbitrary & unjust.

Search engineers are well skilled at public relations. A big part of what search engineers do is managing the market through FUD. If you can get someone else to do your work for you for free then that is way more profitable than trying to sort everything out on your end.

Search engineers are great at writing code. A lot of what the search engineers do is reactionary. Some things get out of control and are so obvious that FUD won't work, so they need to stomp on them with new algorithms. Most search engine signals are created through tracking people, so they usually follow people. Even when it seems like they are trying to change the game drastically, a lot of that data still comes from following people.

What to Do as an SEO?

The ignorant SEO waits until they are told by Google to do something & starts following "best practices" after most of the potential profits have been commoditized, both by algorithmic changes & a market that has become less receptive to a marketing approach which has since lost its novelty.

The *really* ignorant SEO only listens to official Google advice & trusts some of the older advice even after it has become both stale & inaccurate. As recently as 2 years ago I saw a published author in the SEO space handing out a tip on Twitter to use the Google toolbar as your primary backlink checking tool. Sad!

The search guidelines are very much a living breathing document. If search engines are to remain relevant they must change with the web. Those blazing new paths & changing the landscape of internet marketing often operate in ways that are not yet commonplace & thus not yet covered by guidelines that are based on last year's ecosystem. Individual campaigns fail often, because they are trying something new or different. Off of each individual marketing campaign the expected outcome is failure. However they generally win the war. Those who follow behind remain in their footprints (unless they operate in less competitive markets).

The savvy SEO is a trail blazer who is pushing & probing to test some of the boundaries. They are equally a person who watches the evolution of the web through the lens of history, attempting to predict where search may lead. If you can predict where search is going you are not as likely to get caught with your pants down as the person who waits around for Google telling them what to do next. It may still happen in some cases, but it is less common & you are more likely to be able to adjust quickly if you are looking at the web through Google's perspective (rather than through the perspective they suggest you use).

Google's Noble Respect for Copyright

Google has a history of challenging the law & building a business through wildcatting in a gray hat/black hat manner.

• They repeatedly broke the law with their ebook scanning project. Their ebook store is already open in spite of a judge requiring them to rework their agreements.
• They bought Youtube, a den of video piracy & then spent $100 million on legal bills after the fact. When they were competing with Youtube they suggested that they could force copyright holders to pay Google for lost ad revenues if they didn't give Google access to the premium content. :D
• They sold ads against trademarks where it was generally viewed as illegal and awaited the court's decisions after the fact.
• They tried doing an illegal search tie-up with Yahoo & only withdrew after they were warned that it would be challenged. They later slid through a similar deal with Yahoo Japan that was approved.
• They "accidentally" collected personally identifiable information while getting router information & scanning streets (and we later learn via internal emails in court documents how important some of this "accidental" data collection was to them).
• They pushed Buzz onto Gmail users and paid the fine.
• Google torched UK finance comparison sites for buying links. Then Google bought one of the few they didn't torch (in spite of its spammy links). After getting flamed on an SEO blog they penalized that site, but then it was ranking again 2 weeks later *without* cleaning up any of the spammy links.
• When the Panda update torched one of your sites Google AdSense was probably already paying someone else to steal it & outrank you. Google itself scrapes user reviews & then replaces the original source with Google Places pages. The only way to opt out of that Google scrape is to opt out of Google search traffic.
• Google promotes open in others, but then with their own products it is all or nothing bundling: "we are using compatibility as a club to make them do things we want." - Google's Dan Morrill
• For years Google recommended warez and keygens and serials to searchers, all while building up a stable of over 50,000 advertisers pedaling counterfeit goods. That only stopped when the US government applied pressure, and then Google painted themselves as the good guys for fighting piracy.
• Google is reportedly about to launch their music service, once again without permission of the copyright holders they are abusing.

Those were examples of how Google interpreted "the guidelines" in modern societies.

Google doesn't wait for permission.

What are you doing right now?

Are you sitting around hoping that GoogleBot sorts everything out?

If so, grab a newspaper & pull out the "help wanted" section. You're going to need it!

If you want to win in Google's ecosystem you must behave like Google does, rather than behaving how they claim to & tell you to.

Google's Amit Singhal offered more "clarity" into Google's approach with the Panda update. However I am not convinced that any clarity was actually added, and I think a lot of the questions they ask are to a degree even a bit wrong-headed.

Would you trust the information presented in this article?

Sounds like this is all about promoting perceived authority.

Counter points:

• Did you (or anyone you know) trust this outlet to give you a complete worldview at any point in the last couple decades & end up bankrupt, utterly decimated, or destitute because you followed its advice (on say internet stocks or the housing bubble or using excessive credit because "this time may actually be different")?
• Did your allegiance to this particular media outlet cause you to be more likely to be unbelievably ignorant & ill informed about world-shaping anti-facts used to push things like wars based on fraud? Are there extra trillion Dollars of debt your children must pay interest on & over 100,000 people dead because a news outlet lied to you? And the US government (the largest seat of power & authority in the world today) was also complicit in ensuring American citizens were ignorant going into Iraq.
• A lot of news sites are given additional distribution through services like Google news, which start them from a position of authority (because if you go to search to find something & Google promotes their news vertical right away, then sites in that news vertical will rank highly instantly & accrue backlinks from that early exposure). The education system itself is partly a propaganda tool to teach you to trust an obey authority. If the banking crisis taught us nothing else it should have taught us that many authorities are not worthy of our trust as they act in self interested ways at the expense of the whole.

Is this article written by an expert or enthusiast who knows the topic well, or is it more shallow in nature?

Mainstream media sites saw a $1 billion Dollar lift in annual ad revenue from the Panda update. Most mainstream media articles are *not* written by true subject matter experts, but rather by devout generalists who grab a couple quotes to fill out the shallow piece & make it feel more informed.

A lot of the "official" quotes are from officials who represent industry trade organizations. That means those folks support the interests of folks in that trade, even if/when that trade is working against the interest of the common man.

The problem is, you don't get to see who is a whore until *after* they already ____ed you. See for example David Lereah: "Ahhh, so he admits to being nothing more than a paid shill whose mouth was available for a price. How does that job description vary from the Trannies who hang out by the West Side Highway? In my book, not by very much. A whore is a whore is a whore."

Does the site have duplicate, overlapping, or redundant articles on the same or similar topics with slightly different keyword variations?

*Cough* Google Video vs Youtube vs Vevo.

Aren't most AP articles by their definition redundant duplication?

How are some of Google's late-to-the-party services like their ebook store or their places pages justified if we seek to minimize redundancy?

Would you be comfortable giving your credit card information to this site?

Some sites aim to sell, while others aim to tell.

If a passionate hobbyist desires to share but isn't selling something (and thus uses a quirky site design or a more personal formatting structure) should they be dinged for putting their passion ahead of getting an unneeded SSL certification & paying firms like TRUSTe, McAfee & VeriSign?

Interestingly, sites which display some trust symbols are *more* likely to scam consumers. Being a con man requires abusing trust and confidence. Some of the top brands do just that, over and over again.

Sites which don't go out of their way to sell you something are more likely to be built on passion.

Does this article have spelling, stylistic, or factual errors?

"Correct spelling, indeed, is one of the arts that are far more esteemed by school ma'ams than by practical men, neck-deep in the heat and agony of the world." - Henry Louis Mencken

Further, the error of omission is one that is constantly made in the mainstream media, which is precisely why you have to read fringe rags like the Rolling Stone to get an honest look at how bankers are robbing the country blind. Of course you will read the same article in the mainstream media in 6 or 7 years, after the statue of limitations runs out. And they will sell it as "new" news, even though the story at that point is nearly a decade old.

Are the topics driven by genuine interests of readers of the site, or does the site generate content by attempting to guess what might rank well in search engines?

I can tell you sure as hell that the auto-generated spam stub pages on the mainstream media sites (driven by services like DayLife or Truveo) which scrape blogs like mine are not driven by passion. You can't program a bot to have "passion."

Does the article provide original content or information, original reporting, original research, or original analysis?

Hard to disagree with this point. However, it is worth noting that the mainstream media is notorious for stealing stories.

Further, I have had a client featured in a well read trade magazine where they wrote an entire article on the client. They were unwilling to link to the client's site (even though the client was the only source & entire purpose for the article) because they said they felt it would be too promotional. How warped is it that they will do a photo shoot at your house & make you the feature of an article, yet they are afraid to link because that might be seen as being too promotional!

Does the page provide substantial value when compared to other pages in search results?

This is actually a bit of a bait and switch styled topic. Let me explain. In an ideal world every single page would be great.

But when some brands are above regulation, Google keeps screwing up source attribution & Google creates no-value-add scraper pages like their places pages, if you ensure that every page you make is unique & value add then if you operate at any scale you will likely go bankrupt in the search game (unless you have significant non-search distribution).

Most articles individually are failures that do not pay for themselves. It is the rare success that helps carry the failures. You do not know which is which in advance, but you hope that with some level of effort and scale you are marginally profitable out the other end.

This is how literally all forms of publishing work: online, music, movies, books, etc.

In terms of a money loser, take for instance this article. I am already rather well known, have a wide following, spent hours writing that article, and ultimately it garnered 1 comment & 0 inbound links (once you back out scraper sites, automated links, and links with nofollow on them).

Making things worse, you not only compete against others who will copy anything of yours that is successful, but if Google does decide to whack your site with a penalty then a scraper site (which Google paid with AdSense money) that steals your content will outrank you for your own work. How exactly do you provide a unique substantial value add when Google is paying others to steal & republish your work wholesale?

Things like source attribution issues, brand bias, and Google competing against publishers with scraper pages have a very real and significant impact on profit margins. A good sustainable company is generally lucky to have 20% profit margins. When Google introduced their places pages that scraped TripAdvisor Google instantly redirected 10% of TripAdvisor's search traffic.

Ultimately the above issue with content is not down to cost or effort, but if what you are doing is profitable. If it is not, then it is simply unsustainable.

And even when you are profitable, you can count on Google helping others subvert that position.

On the topic of value add, I have even seen people buying AdSense ads to redistribute 3rd party works, where the only value "add" was lowering the retail price!

How much quality control is done on content?

A lot of the high ranking and much hyped social media networks like MySpace, Friendster, Twitter & Facebook are almost exclusively spam. A couple days ago I deleted over 75% of my Facebook "friends" because I was sick of getting daily email updates about how some dirtbag wanted to promote some autowealth MLM blaster unlimited downstream product on my wall.

That is not to say that everyone I deleted did anything wrong (most of them are likely good people) but there was no opportunity cost to spamming. The spammers who automate drive everything toward the tragedy of the commons. A paywall is perhaps the single best filter for quality, but if you use a paywall expect to deal with a lot of freetard rage & expect Google to pay some folks to steal it.

Google polices the web, but anything goes in their ad programs.

You can see how ridiculous the double standard is by simply considering that Google let their counterfeiting advertisers count grow to 50,000 strong before finally axing them when the US government pressured Google. Bizarrely, Google had the audacity to position themselves as good doers who were cracking down on spammers, when in fact they were taking their own longtime business partners out to the wood shed!

Does the article describe both sides of a story?

Mainstream media sources often like to share "both sides of a story" to seem unbiased. But the truth is that media by its very nature is biased toward the interest of advertisers & away from consumers. See, for example, either Manufacturing Consent or the BGH lawsuit.

Further, some well known corporations (LIKE GOOGLE) blackball media outlets that question them in certain ways. Google would never give exclusives to SEOBook & the sites that they do give exclusives to would lose the relationship if they were as blunt as we are.

Is the site a recognized authority on its topic?

Lots of recognized authorities have conflicting funding sources - something that was well highlighted in early Google research, and has been consistently exposed (years or decades after the fact) in the medical space.

Many firms which can "move the market" regularly trade against the advice they give to retail schleps.

Honesty is more important than authority, but then being bland & honest is not quite as remarkable (or profitable) as putting on a coat of spin.

Is the content mass-produced by or outsourced to a large number of creators, or spread across a large network of sites, so that individual pages or sites don’t get as much attention or care?

How would Google's efforts stand up when graded against this suggestion? Why does Google have Google Video, Youtube & Vevo?

Further, most market leaders do have large networks and multiple branded sites for purposes of branding, segmentation, and double dipping in the marketplace. Remember when Bankrate (which already owned Bankrate, Nationwide Card Services, Credit Card Search Engine, Bankaholic, etc.) bought out CreditCardsGuide.com & it got temporarily penalized for the spammy links it had? Well it ranks again & of course since then they have also bought out CreditCards.com. You see this sort of behavior amongst almost any big brand: from Amazon.com to Zappos. (Oh wait, Amazon.com now owns Zappos!)

Was the article edited well, or does it appear sloppy or hastily produced?

A lot of the best content comes from people who are subject matter experts. But those people may have only mastered their subject & may be new to: writing, website design, online publishing, etc.

For a health related query, would you trust information from this site?

Let's put it this way: the media people consume is in part responsible for the current state of health in the US where there is an obesity epidemic. Further, a lot of the leading health authority sites (like WebMD) run special advert sections in their site where it looks just like content but you have to read the small print to see it is an ad.

Going one step further on this front, it is worth mentioning that a number of the large pharmaceutical corporations have repeatedly sold drugs for off label purposes & yet none of their packaging is required to highlight those ill deed they did that have literally killed millions of people.

And let's not forget Google's take on educating misinforming the public about these drugs.

Would you recognize this site as an authoritative source when mentioned by name?

A lot of "authoritative" sites are simply sites with large ad budgets.

Quick, tell me which company advertises a clever gecko with a British accent. Other than as a mascot (& perhaps alliteration), how relevant is that gecko (or the accent) to their business? Not at all. But they do spend nearly a billion Dollars a year on ads.

Does this article provide a complete or comprehensive description of the topic?

Most articles that do are money losers.

Especially true while Google is funding so much no-cost automated web scraping.

Does this article contain insightful analysis or interesting information that is beyond obvious?

Held to your own standard, how would Google Places pages hold up?

Is this the sort of page you’d want to bookmark, share with a friend, or recommend?

A lot of the stuff which is shared is shared precisely because it is ill-informed, controversial, or shares someone's pre-existing biases.

Does this article have an excessive amount of ads that distract from or interfere with the main content?

In the Manufacturing Consent DVD a big media guy highlighted that they like to have a 60/40 split between ads and content. Google already pushes online publishers to do less, even if Google does the exact opposite:

Would you expect to see this article in a printed magazine, encyclopedia or book?

Aren't print magazines where a lot of the bait and switch headlines came from?

Are the articles short, unsubstantial, or otherwise lacking in helpful specifics?

Do you mean something similar to most magazine articles? Or do you mean the content farms that Google funded & then used as a justification to torch 10,000's of small businesses?

Held to your own standard, how would Google Places pages hold up?

(I know. I know. If what you do looks like what Google does then you MUST be a spammer.)

Are the pages produced with great care and attention to detail vs. less attention to detail?

This basically excludes almost any user generated content site, with the exception of Youtube.

Would users complain when they see pages from this site?

I'll complain about something I just saw. ;)

While searching for a link for a blog post I was writing today, the #1 Google result (not voted up by social circle stuff) was a Tweet linking to a Hootsweet framed page linking to a music industry site which posts RSS feed content and linked to a BusinessInsider article that referenced the TechCrunch article I was looking for.

If we want to get rid of unneeded duplication & noise then why is Google tying their bonus system to promoting more social media noise? After Amazon.com has done a great job with Kindle why is there a need for Google's ebook marketplace? After Yelp has created a strong community review site (with real editorial expenses) why is there a need for Google Places to scrape & displace its reviews?

-----

tl;dr

If you look at what actually happens in reality (rather than what folks claim to support in their "ideals") it is anarchy. The bankers stole what they could and moved on. The pharmaceutical corporations create fear-driven propaganda about the dangers of drug re-importation, all the while pushing drugs for off label purposes. Google pays people to steal your content, then tells you to suck it up & it is your fault you are not a big brand.

Anarchy is here.

The only difference is that it is dressed up in suits and fancy language, where people perceive anarchists as like ripped jeans, megadeth shirt wearing, pyro's.

Before Google's Panda update an effective SEO strategy was to "make a page for everything." If you are Wikipedia that strategy may still work, but for most websites that approach is a high risk & low return approach. Clustering like keywords together and using that to help set up your site's information architecture is a lower risk and higher return strategy.

Given Google's new approach to search (where dead weight can harm your good pages) organization is more important than ever.

Let's say you have a big list of keywords which is not well organized & you want a quick and dirty way to organize it. Here are a dozen different tips and tools to help you organize your keywords.

Ad Group Filter

We created an ad group organizer tool which aims to create a footprint for keywords by putting muti-word keywords in alphabetical order & stemming the keywords. The output is TSV, so you can copy it and paste it into a spreadsheet for further analysis. It also allows you to use stop words to filter the list. This can not only be used for organizing keywords for paid search, but also to help organize them for your SEO efforts down to a per-page level.

RKG Duck

RKG Duck is a Perl clipboard extension which was the inspiration for our above tool. This tool works well in spreadsheets, but it takes some level of programming sophistication to get working.

Wordstream Keyword Grouper

Wordstream's keyword grouper allows you to see niche keyword groups at a fairly granular level, with them emailing the results to you.

SpyFu Keyword Groupie

Spyfu's Keyword Groupie allows you to look up a competing site's keywords & see them organized by root word. In addition to listing keywords by root word, they also give you the option of viewing the top 100, 500, or 1000 keywords for a site.

The big benefit with SpyFu is that you can view data on a competing site & use their performance as a bit of a filter for you, but the downside is that at times it can be a bit slower than the above tools. That is to be expected though, because it is searching through a database of records related to sites, rather than just applying a filter or returning results.

They allow you to browse keywords to drill down in areas of interest. Another nice feature is that they show you keywords they feel you are missing out on by putting them in bold, so this tool is great both for looking up competing sites & for looking up your own site to see what you missed.

Google AdWords Desktop

There are a couple different ways to use the Google AdWords editor to group keywords. Here is a semi-automated way (where you still have a bit of human editorial in the process to manually filter to find themes & create groups)

You could do similar to the above with Microsoft Excel's table filters. And if you are combining multiple data sources / tables in Excel this AbleBits merge tool is handy.

Google also offers an automated option inside Google AdWords editor to help organize keywords into fairly tight groups.

First create a new ad campaign (and its settings can be a bit arbitrary off the start as you are mainly using this to help automate data sorting). You only need to create 1 ad group in that ad campaign and then bulk upload a group of keywords into it.

Next use the keyword grouper tool, as shown here.

In the keyword grouper you can use the "generate common terms" option to automatically create keyword groupings. Note that in the right box you can add stop words & other words that you don't want them to cluster keyword groupings around.

Google then spits out a result set you can use, with the keywords clustered into tight groups. Note that sometimes they footprint geo-local keywords similarly even if they are for multiple different areas, but outside of that it is a pretty nice tool considering the amount of work it does in what amounts to a 2-minute process.

Rank Checker

Put your site in rank checker and see how you rank for your target keyword list. If your site is nowhere to be found for a keyword then that may indicate a need to create more content pages around those new topics. If your pages already rank well then see how well they are optimized. A small amount of link building & on-page SEO can go a long way if you were already ranking for a keyword that you were not intentionally targeting.

If your site is brand new & has no authority (or you are researching a new market) you can search for the rankings of a popular website in your niche and see where they rank. Export the ranking data and you can sort the Excel spreadsheet by URL, which should help you cluster your keywords around a similar strategy that top ranked websites are using.

You can pull data on competing sites from competitive research tools like SEM Rush, Compete.com, Keyword Spy, SpyFu, and Alexa to help get an overview of some of the top keywords competing sites are ranking for.

Crawl Their Site

Do you have a well optimized competitor? You can crawl their site using tools like Xenu Link Sleuth or Screaming Frog & then export the data to a spreadsheet, using that as a baseline to start your information architecture strategy from. Xenu is free & Screaming Frog's SEO Spider is free for up to a 500 URL site.

Keyword List Cleaner

If you have a big and dirty keyword list where some of the words have multiple meanings you can try to filter the list down by using negative keywords on a keyword list cleaner.

Google Related Searches

Whenever you search on Google not only does their search box recommend tightly related keywords (which are good for late state optimization of on-page content), but in the left column they have a link to "related searches" which organizes related keywords. Within these lists of keywords you can click further into to drill down deeper.

Some folks scrape that data in bulk as well, but if you do that then you are back to having to organize it again. ;)

Google AdWords Keyword Tool

Many paid keyword tools like Wordtracker have advanced filtering & organization options, but I mainly wanted to show free options in the post. Google AdWords keyword tool has multiple helpful ways to organize data.

I tried to highlight key areas & options in the above image, but it sorta feels like I highlighted everything, as there are so many amazing options baked into it. You can get keyword data based on selecting a category, a site, or entering a root keyword. They allow further filtering by match type, tight or loose keyword groupings, location, and so on. Sometimes the data can be a bit inaccurate, but nonetheless it is a great starting point as it really is an amazing feature-rich tool.

Microsoft adCenter Plug-in for Excel

Earlier I mentioned how Excel tables have a bunch of handy filters in them. Taking that to the next level, try the adCenter Excel plug in (review here), offering you quick access to Microsoft's keyword data by root keyword, general topical category, ad campaign association, and so on.

Your Web Analytics

There are at least 4 amazing benefits to using your site's keyword data

• This is the stuff that actually applies directly to your site. Rather than being some sort of academic exercise or a bunch of "what if" sort of stuff where there is a big margin for error, you have the data related to the actual business impact of these keywords.
• Since your site is already ranking for these keywords you already have momentum behind them. Pushing a #5 to #2 is typically far easier than going from nowhere to #5. And it is not only easier, but it is also more profitable.
• This data is organized by page already, and (since you know your site) you should be able to quickly tell if pages that are ranking should be further optimized for a keyword or if the user intent for that keyword is different and it deserves a different page.
• If you have been tracking your site for an extended period of time you should know not only what pages are ranking, but also why. Sure Google aims to make this a bit more complex, but that is precisely why looking at data on your own site is so helpful: you already know so much about it.

The same types of benefits can be had by using a (phrase matched, broad matched, or modified broad match / with negative keywords) AdWords ad campaign to do keyword research. You are not only testing the search volume of the keywords, but also how your site performs for them.

Visualize It

WordTracker's Strategizer (review here) is a premium SEO-oriented extension of web analytics data, which helps make the data relationships easier to visualize. Concentrate is another paid application built on data wrangling & visualization front.

Free keyword cloud tools like Wordle & tools like Many Eyes can also be valuable for helping you see word relationships for a page and convey concepts to management. You can probably guess which page the following analytics-driven word cloud is for without even visiting it. ;)

You could also put a URL in a keyword density tool, a page comparison tool, or a word cloud tool to view a page's on-page content that way. If it isn't too self-referential, ...