Google Insights for Search

Google recently added search volume estimates to their keyword tool. They also recently launched Google Trends, Google Hot Trends, Google Trends for Websites, and the Google Ad Planner. And now Google hits the competitive research market with yet another product - Google Insights for Search

Insights for Search shows the following search data

  • relative keyword search trends for keywords (and A/B comparisons between keywords)
  • top related keywords and hottest rising related keywords
  • category based top keywords and category based hottest rising searches (and overall top 10 rising searches)
  • category based keyword search volume trends, and the relative growth of a keyword compared to its category
  • countries, states, and cities where a keyword query is popular
  • you can also mash up many data points, like celebrity searches in New York in the last 30 days

Keywords are weighted such that their top volume day is anchored at 100, and other days are represented as a relative percentage of that search volume.

Just like their Google Trends tool, by default Google Insights for Search defaults to the broad matched version of a keyword, so a word like credit will show more volume than credit cards, even though credit cards gets more search volume (because terms like credit cards and credit reports count as search volume for the word credit). Credit vs credit cards pictured below:

This type of tool can also be used to see how related some generic concepts are to more specific related concepts, and how much news coverage and marketplace changes move the relative importance of different keywords in a marketplace. Public relations experts will be able to use graphs like the following to say "hey our brand is catching up with the market leader."

Rather than brand lift and PR lift being an abstract concept, we can compare brands in real time and see which markets resonate with our brands and marketing. When marketing is working *really* well you can consider boosting early success in the most receptive markets via offline advertising, social interaction, and live events. If I wanted to hold an offline seminar guess what state is most receptive to my brand? The one I live in. That's pretty cool. :)

And the problem with such tools? It is easy for me to lose days or weeks playing with them. What are your favorite search data tools? What creative ways do you use them?

Google Beta Testing Showing Related Phrases Near Documents in Search Results

While using Opera I noticed the following Google test which places related phrases near some documents in the search results

When I entered our above link building page into the Google AdWords Keyword Tool they showed mostly phrases related to the broader category of SEO and did not list the niche link related phrases, which indicates Google is still holding back quite a bit of data from advertisers that they are willing to share with searchers for free.

Google Knol - Google's Latest Attack on Copyright

Knol Off to a Quick Start

One day after Knol publicly launched Wil Reynolds noticed that a Knol page was already ranking. Danny Sullivan did a further test showing that 33% of his test set of Knol pages were ranking in the first page of search results. Danny was also surprised that his Knol was ranking #28 after 1 day. After citing it on his blog now that Knol page ranks #1 in Google!

Google's House Advantage

From the above data (and the aggressive promotion of YouTube content after the roll out of universal search) it is fair to state that house content is favored by the Google algorithm.

Another Knol Test

Maybe we are being a bit biased and/or are rushing to judgement? Maybe a more scientific effort would compare how Knol content ranks to other content when it is essentially duplicate content? I did not want to mention that I was testing that when I created my SEO Basics Knol, but the content was essentially a duplicate of my Work.com Guide to Learning SEO (that was also syndicated to Business.com). Even Google shows this directly on the Knol page

Google Knows its Duplicate Content

Is Google the Most Authoritative Publisher?

Given that Google knows that Business.com is a many year old high authority directory and that the Business.com page with my content on it is a PageRank 5, which does Google prefer to rank? Searching for a string of text on the page I found that the Knol page ranks in the search results.

If I override some of Google's duplicate content filters (by adding &filter=0 to the search string) then I see that 2 copies of the Knol page outrank the Business.com page that was filtered out earlier.

Some may call this the Query Deserves Freshness algorithm, but one might equally decide to call it the copyright work deserves to be stolen algorithm. Google knows the content is duplicate (as proven by the notification they put on their page), and yet they prefer to rank their own house content over the originally published source.

Hijacking Your Rankings via Knol - Google Knoljacking

Where this becomes a big issue is if a person...

  • posts your content to Knol
  • and buys/rents/begs/steals/spams/borrows a couple decent inbound links

they can get you filtered out of the search results - even if your site is an authority site. Bad news for anyone publishing copyright work online.

Google Knol Undermines the Creative Commons Spirit

Some new publishers decide to license their work via Creative Commons (hoping to be paid back based on the links economy), but Google wants no part in that! All outbound links on Knol are nofollow, so even if a person wants to give you credit for your work Google makes it impossible to do so.

Google Voids YOUR Copyright

Why do I get enraged by this sort of activity? I remember when one of my sites was voted against, and Google paid someone to steal it and wrap it in AdSense. The person who stole my content outranked me for my own content because a Google engineer thought that was reasonable and fair.

To this day someone publishes seobookopen.blogspot.com, a site dedicated to stealing and sharing an old version of my ebook. As the opening post on that blog explains

www.seobook.com very famous book from Aaron Wall its really good but paying $79 its really sucks so yesterday, I think why not to share this book to my friends etc openly in text by decompling Acrobat files

Can a casual mention get it removed? Nope. Can flagging it as spam and highlighting that it is stolen copyright content get it removed? Nope. I need to file a DMCA request to get it removed. (Or maybe they will remove it out of embarrassment after I hit publish on this post...we shall see!)

Google Pays Thieves

Google doesn't like lonely cheating housewives, unless it favors their business objectives.

Help Improve My Google SEO Knol & Win a Free 3 Month SEO Training Trial

Squidoo, Mahalo, eHow, EzineArticles, etc etc etc just got validation for their business models and competition for the ad network that helps them monetize their sites. Google today launched their Knol project:

The key principle behind Knol is authorship. Every knol will have an author (or group of authors) who put their name behind their content. It's their knol, their voice, their opinion. We expect that there will be multiple knols on the same subject, and we think that is good.

With Knol, we are introducing a new method for authors to work together that we call "moderated collaboration." With this feature, any reader can make suggested edits to a knol which the author may then choose to accept, reject, or modify before these contributions become visible to the public. This allows authors to accept suggestions from everyone in the world while remaining in control of their content. After all, their name is associated with it!

Wired has an article about the designing of Knol, and Danny Sullivan offers more Knol background, including that it is a product from the search team (which could greatly hint at future search integration plans).

Want a free 3 month trial of our SEO training program and community?

Comment on my SEO Knol leaving tips on how I could improve it and a way to contact you, and I will pick a winner next Wednesday.

SEO Friendly Flash?

Matt Cutts on Using Search Usage Data to Fight Spam

A couple weeks back we mentioned that Google's Peter Norvig stated that Google does not use search usage data directly in their relevancy algorithms. Yesterday Matt Cutts made a post on the official Google blog stating that Google does look at search logs / usage data to determine how large spam attacks are and how well new anti-spam measures are doing

Data from search logs is one tool we use to fight webspam and return cleaner and more relevant results. Logs data such as IP address and cookie information make it possible to create and use metrics that measure the different aspects of our search quality (such as index size and coverage, results "freshness," and spam).

Whenever we create a new metric, it's essential to be able to go over our logs data and compute new spam metrics using previous queries or results. We use our search logs to go "back in time" and see how well Google did on queries from months before. When we create a metric that measures a new type of spam more accurately, we not only start tracking our spam success going forward, but we also use logs data to see how we were doing on that type of spam in previous months and years.

Google Website Trends & The Death of Privacy

Google launched a new version of Google Trends which includes website traffic estimates, and highlights....

  • visitor locations
  • top keywords
  • related sites

You need to be logged into a Google account to get a numerical scale on the traffic graph, but you can still get the general trend graphs and other data without logging in. Google also allows you to compare up to 5 sites at any given time

Search Engine Land reported that Google Trends...

This tool bases its data off Google search data, aggregated opt-in anonymous Google Analytics data, opt-in consumer panel data, and other third-party market research.

Those who wondered why using Google Analytics is a bad idea just got a taste of why it is a bad idea! Sure the data is only aggregated anonymously, but with Google owning 70% of the search market and having access to your analytics data you never know how deep they could decide to go through the data for competitive purposes. Is your site a keyword source for AdWords? How much of your data will appear on Google Trends?

You can't export the data yet and it only shows data for high traffic sites, but it is a great free tool for marketers.

  • If sites do not show any data then they are probably either low traffic sites or penalized.
  • The related sites feature lets you know how related your audience is to other sites, which is useful for determining how familiar their audience is or if you are reaching a new audience when buying ads.
  • The top keywords shows you what competing sites are getting traffic from, and if those terms tend to be more brand oriented or more generic in nature...which is useful when thinking about the stability of a website you may want to purchase.

Fred Wilson did a review of Google Trends, comparing Google Analytics to Compete.com and comScore across 3 high traffic websites.

Since we are investors in these three companies and know what their internal numbers are saying, I can safely say that ComScore and Compete are lower but directionally correct. Google is like Alexa in that they don't report absolute numbers but even if they did, they are not directionally correct on this particular set of companies.

The fact that Google is even compared to the analytics firms with years of experience and algorithmic tuning shows how easily they could take a leading position in this market if they wanted it. Google will improve their accuracy, and at any point in time they could chose to expand the top 10 keywords to the top 100 or top 1,000.

The AP threatened to sue a blogger for quoting small passages, and at the same time the mainstream media is trying to redefine copyright for their own benefit. Eventually much of the mainstream media will start looking like eHow and Mahalo. Your content compiled and slightly rewritten by a third party. Your keyword list is their money list. Thousands of people are competing against you while you read this sentence. As your data leaks it is going to be tough to stay competitive unless you are often the topic of conversation. Public Relations is the only PR that matters.

I am probably a bit less pessimistic than Michael is at the moment, but firms are using G.P.S. data to create "reality mining" for offline analytics, predicting everything from traffic patterns, where to buy an ad, and where to place another store. A recent NYT article highlighted some such services:

Tony Jebara, also 34, the chief scientist and another co-founder of Sense, said, “We can predict tourism, we can tell you how confident consumers are, we can tell retailers about, say, their competitors, who’s coming in from particular neighborhoods.”

The idea of staying competitive through obscurity is obsolete. So you may as well be a loud mouth, encourage users to be loud mouths, and build a big brand that helps protect your plot from competitive market forces.

Peter Norvig - Google Does Not Directly Use Search Usage Data in Relevancy Algorithms

Anand Rajaraman recently spoke with Peter Norvig, who revealed that:

  • their best machine learning algorithms is already as good as, and sometimes better than their current hand roled relevancy algorithms
  • but they still prefer to use their hand roled algorithms because of hubris, and they feel that machine learning algorithms may be more inclined to have catastrophic errors on searches that do not look much like those in the training set

I think a third piece (that you will never hear Google employees admit to) is that as the web's structure changes Google feels they have use FUD to police the web and help ensure Google has revenue entry points into important markets. In their 2007 Google search quality rater guidelines they used a typical Commission Junction link as an example of a sneaky redirect. It is doubtful that Google would ever do that with AdSense code or a Performics link (since they own those).

In the follow up post about his chat with Peter Norvig, Anand highlighted how Google measures relevancy. In the post he stated why Google prefers internal review data relative to using direct usage data:

Peter confirmed that Google does collect such [usage] data, and has scads of it stashed away on their clusters. However -- and here's the shocker -- these metrics are not very sensitive to new ranking models! When Google tries new ranking models, these metrics sometimes move, sometimes not, and never by much. In fact Google does not use such real usage data to tune their search ranking algorithm.

Exposure from top rankings already creates a self-reinforcing effect because of the power of defaults. Further tying in search usage data directly into relevancy might not add much benefit to searchers, especially as more people click on the first search result. Anand further explained why direct usage data is not used to refine Google's relevancy algorithms:

The first is that we have all been trained to trust Google and click on the first result no matter what. So ranking models that make slight changes in ranking may not produce significant swings in the measured usage data. The second, more interesting, factor is that users don't know what they're missing.

Google AdWords Price Fixing

In Google's commentary about their ad deal with Yahoo! they wrote:

This does not let Google raise prices for advertisers. Google does not set the prices manually for ads; rather, advertisers themselves determine prices through an ongoing competitive auction. We have found over years of research that an auction is by far the most efficient way to price search advertising and have no intention of changing that.

Aspects of that statement are categorically untrue, perhaps even lies. In many competitive markets with lots of participants the ad market may set ad price minimums, but Google...

  1. publicly talks about how they tweak the number of ads they display to maximize revenues
  2. uses quality scores that allow them to give friendly businesses discounts
  3. Google not only favors its own ads, but also creates custom ad units that only it can buy

Abitrary Pricing Floors

Google has articles in the media talking about how they tweak dials to optimize revenues. While many competitors have increased the number of ads they show, Google has been showing ads across a smaller portion of their search queries, as shown via this comScore data.

If you do not pay Google enough they simply will not show your ads, even if there are no competitors. I have ads where I am the only bidder and I get a 17% clickthrough rate - and yet there is a 17 cent price on those clicks, rather than a true market floor. Bid too low and your ads simply do not show up - even if you are bidding against nobody.

Preferential Pricing

Getting your account Google slapped is a well known phrase amongst many affiliate marketers. One day your ads are going great, and then the next day every keyword has a minimum bid of $5 or $10 per click.

On the flip side of that, many click arbitrage based business models are only profitable *because* a publisher gained access to a high authority trusted Google partner which allowed cheaper ad prices for the same keywords & ad units.

Google has went as far as publishing information about the types of business models that they do not like. Unlike acceptable business models like reverse billing fraud and infidelity, selling ebooks on sites with ads might merit a low landing page quality score.

Google Only Ad Units

Google products are advertised aggressively across Google's content network. Given that internal Google product benefit from brand awareness, bidding with funny money, and cheaper ad prices (since they don't have to give Google a cut) others with similar business models can not compete.

When Google recently entered the mortgage lead market they gave themselves an ad title of 49 characters, and a dropdown that is not available to other advertisers.

Google Does Not Like Flash Welcome Screens, Will the Google Browser?

Philipp Lessen pointed out that Google now offers a link in the search results to skip intro on flash intro pages.

Philipp also mentioned rumors of a Google Browser from February.

Google also launched a major offline advertising campaign in Moscow, hoping to gain market share on Russian search leader Yandex.

Pages