Google Getting Gunshy on Traffic Referral Deals?
Incredimail reported its 4th quarter earnings today and the stock got killed thanks largely to the revelation that Google had recently approached it and indicated that they were considering significant changes to their policies on how their "traffic partners" are allowed to reset their users homepage and default search provider settings. You may recall Incredimail from my recent post on companies that highly dependent on Google for their economic livelihood. As that post pointed out, Incredimail gets the vast majority of their revenues from one of Google's "traffic partner" programs so any change to that program is potentially a very big deal.
What exact "traffic partner" program is at issue? Anyone who has downloaded a piece of free or shareware software in the few years is familiar with this racket. You download seemingly free software and the software tries to change your default search provider and default home page, usually to Google, but sometimes to Bing or even Ask. Some installers are very explicit about this change and require you to approve the change at least once during the install, while others make the change part of the default install and clearly try to make the change fly by unnoticed by the user Once these defaults are changed, the "traffic partner" that installed the software gets a Google's revenues for every search that flows to Google.
These kinds of "traffic partner" search deals have become very important for many software providers, especially "free" and "shareware" software providers. Their importance has led to an epidemic of "upgrades" to free software that require full re-installs, not because the re-installs are needed technically, but because the re-installs provide another opportunity for the traffic partner to switch people's default search provider and default home page.
Google is likely considering making changes for one of two reasons:
- They believe that this kind of default switching could expose them to regulatory and legal action from the US and EU. For example, Microsoft was recently forced by the EU to offer a "browser choice" screen for new Windows installs, effectively preventing Microsoft from making IE the default install. I wouldn't be surprised if Microsoft is complaining that they are being unfairly singled out because Google does the same thing in search. Or perhaps Google is hearing that if they don't clean up their partners' act the regulators may step in and clean it up for them.
- Since Google's search share is already over 70%, they may have done the math and figured it really doesn't make sense to pay for traffic, because most of the searches are bound to flow their way anyway.
Perhaps it's a little bit of both, but either way, for anyone who depends on these kind of traffic programs this is potentially a very big deal, especially if what's motivating Google is reason #1, because that means Bing and Ask will be out of the market in due time as well. I'm not sure how this will turn out, maybe Google will just force firms to have a more explicit "opt-in" process (which will no doubt significantly reduce uptake rates), but it's worth keeping a close eye on because the stakes for a lot companies are huge.
March 25, 2010 | Permalink
The Algorithm Myth And Why Google Will Be Hated
Google’s search algorithm, The Algorithm, is arguably the single most important computational routine in the world right now. As I outlined in my last post, for many companies The Algorithm literally controls their entire economic destiny and as Google’s traffic and market share grows, so too grows the power of The Algorithm. Traditionally, concentrating so much power in hands of one company has provoked great populist opposition, and while unease with Google’s power is clearly rising, this unease is nowhere near the level of wrath directed at the dominant companies of earlier generations such as Standard Oil, IBM, or Microsoft. Ironically, it is the heart of Google’s power, The Algorithm, which has in many ways enabled Google to greatly mitigate the build-up of popular resentment.
The Algorithm, as we have been told many times, is an objective, unprejudiced, egalitarian and ultimately democratic computational routine. It does not play favorites or king maker nor harbor grudges or biases, it simply reflects reality. In short, it does no evil. This positioning of The Algorithm provides Google with a compelling defense mechanism against anyone that would question its motives or methods. To question Google’s search results, is in effect to question objective reality.
With all this in mind it was very interesting to read a great article in Wired recently that describes in breathless detail the process by which Google makes changes to The Algorithm, changes that are likely to number more than 500 this year alone. Wired dutifully casts Google’s engineer’s as the High Priests of Algorithmic purity working secretly and securely to enhance the end-user search experience. The Wired article is a triumph for Google’s PR machine because it maintains and enhances the Algorithm’s mythic status even though the truth is that The Algorithm myth is dying before our eyes, an unavoidable victim of it’s all encompassing success.
Dying you say? How in the world could The Algorithm, the very engine of Google, be dying at the same time that Google appears to be on the cusp of total search domination? Look no further than the text of Wired’s article. While many of the hundreds, if not thousands, of changes that Google will make to The Algorithm (and the Algorithm’s lesser know supporting cast of crawlers and indexers) will indeed be focused on making it easier for the end-user to find the proverbial needle in the haystack, what’s left unsaid in this article is that many, perhaps even the majority of these changes will be made as part of war; a war against those who are actively gaming The Algorithm by manipulating the very reality it attempts to reflect.
Google vs. SEO
And just who is this that is “gaming” The Algorithm? Just about everyone on planet earth. There is a large and rapidly growing Search Engine Optimization (SEO) industry focused exclusively on this very task. Within this industry there are the “white hats”, those who use a stable of Google-approved or at least Google-condoned “site optimization” techniques, and then there are the black hats, those who actively seek to game or trick Google’s indexers. Most reputable companies try to adhere to “white hat” techniques, but with the stakes so high, the economic rewards so huge and the playing field constantly evolving, there is a very slippery slope towards gray and ultimately black techniques.
While Google lightly encourages White Hat SEO (it makes Google’s crawlers more efficient and its results more accurate and complete) it abhors the Black Hats. The Black Hats are seen are seen as public enemy #1 (perhaps more so than click fraudsters) because they corrupt the platonic integrity of Google’s results and weaken the overall user experience thus undermining the very foundations of Google’s success. When discovered, Black Hats are often dealt swift and summary justice from Google. Not only are page ranks reduced but in many cases pages, sites, and even entire IP address ranges receive what has become the information age’s ultimate punishment: banishment from the Google index. But such justice doesn’t solve really the problem, because the Black Hats merely decamp to another URL, another IP and start all over again.War Over The Algorithm
The primary focus of the Black Hats is The Algorithm. By reverse engineering how it works, they seek to create ways to trick it into elevating specific pages within the Google index with the ultimate goal of achieving the Holy Grail of SEO: becoming the #1 organic search result. The war goes back and forth in volleys of ever increasing complexity. The Algorithm prizes term repetition, the Black Hats repeat a term hundreds of times on a page. The Algorithm values home page links, the Black Hats create paid link exchanges. The Algorithm elevates back links, Black Hats create multi-layered networks of “back link farms” for hire. It has gotten to such an extreme that it is rumored that there are thousands, some say millions, of “robo-users” on Google now, computers designed to give false feedback to Google by clicking through on specific search results just enough to raise their index rank without tipping off Google’s counter measures. In this way, SEO has gone from just “optimizing” content on a single site, to networks of sites, to building an entire closed loop of sites, networks, and virtual users.
Google isn’t blind to this behavior. They know what the Black Hat’s are doing and they are fighting them, hard. As Wired ably chronicles, their search engineers are constantly making hundreds of changes to The Algorithm, many with the explicit or implicit goal of filtering out the “noise” introduced by SEO. While these high-tech efforts to fight Black hat SEO within the context of The Algorithm are reasonably well known, what is less well known is that increasingly Google is manually tweaking output from The Algorithm to ensure “search quality”. Literally hundreds people around the world spend every day typing searches into Google and making sure that the correct results appear. Some of this is to ensure search results reflect local culture (e.g. “football” results in Europe are focused on soccer while “football” results in the US are focused on the NFL), but much of it also focused on making sure that the Black Hats are not winning the war. Google is also rumored to manually review the search results for particularly high profile or high value search results and to be manually intervening when the results diverge from the Google’s platonic ideal.The Mother of All Negative Feedback Loops
The great irony in all of this is that Google’s efforts to defeat the Black Hats and to ensure “search quality”, arguably very laudable goals, are ultimately undermining the credibility and sustainability of The Algorithm. The more Google “tweaks” The Algorithm in the name of “search quality” the less The Algorithm reflects the objective reality of the web. A pristine mathematical routine becomes, by necessity, a mish-mashed labyrinth of math routines, rules based logic and arbitrary exceptions/exclusions. Google is in essence caught in the mother of all negative feedback loops. Its success is driving more and more SEO which makes reality a noisier and nosier mess which necessitates that the Algorithm depart further and further from objective reality. Google’s efforts, laudable though they may be, to create the platonic ideal of Internet “reality” are thus doomed to become Google’s editorialized viewpoint on what should constitute Internet reality as opposed to what really does.The End of the Algorithm Myth
In this way it is almost a done deal that The Algorithm will lose its mythic status as an objective and unprejudiced computational routine. It will merely become Google’s point of view on what should constitute reality and once it does that, every single change Google makes to The Algorithm, every manual tweak, will be open to question: “Why did you make this change?”, “Who benefits from this change?”, “What ‘reality’ is this supposed to reflect?”. The Algorithm’s loss of credibility will thus expose Google to charges of bias, of playing favorites, of being, well, evil.
What’s more, it’s unavoidable that in Google’s war against the Black hat SEOs there will be collateral damage. After all, if you are making hundreds of changes to The Algorithm a year and doing daily quality checks, someone is bound to either make a mistake or make a decision that while hurting the Black Hats also hurts some White Hats. Mix this collateral damage with the editorialized Algorithm and with Google’s staggering economic influence and you get a readymade recipe for intense fear and resentment of Google on part of website owners around the world.Today’s Hero, Tomorrow’s Villain?
In the 1980s, Microsoft was the darling of American industry. Founded by a college drop-out, it took on the world’s most powerful technology company, IBM, and in the process made mass market computing cheap for everyone. Just a decade later, the popular perception of Microsoft had changed dramatically. It was now an unscrupulous monopolist who abused its position of power to extort monopolistic rents and used its vast wealth to unfairly crush competition in adjacent markets by literally giving away things such as the browser and music player.
During the first decade of the 21st century, Google has become the new darling of American industry. It has taken on Microsoft and Yahoo! and won. The best and brightest want to work there. With the inevitable decline of The Algorithm myth, Google will lose one of its most important assets, its reputation as a objective broker of the democratic truth. How it responds to this loss will determine whether or not it faces a similar fate as Microsoft. At one level, there may be nothing Google can do to change its fate. There is bound to be a reaction to the accumulation of vast power and wealth no matter how innately good or evil its owner might be. However Google will have choices. Will it continue to develop its Algorithm in secrecy even as its objectivity clearly wanes, potentially fostering conspiracy theories and resentment? Or will it publicize the Algorithm hoping that such transparency builds trust and credibility? Or will it open-source the Algorithm and seek to disavow ownership of it, and its associated liabilities, entirely?Personally I have no idea what they will do. The good news for Google is that they have a bunch of incredibly talented people at the company. I know many of them and they are some of the brightest and least evil people you will ever meet. But I also know that the Algorithm myth will die and its death, along with Google’s immense power, will pose one of the great management challenges of this decade. The betting man in me says more people will hate Google than love it in 10 years, but 10 years is a long time and fate is what you make of it.
March 24, 2010 | Permalink
The Google Dependency Index: A List of Public Internet Companies That Must Kiss Google's, er, Ring
Wall Street has lots of stock indexes. Everyone knows the NASDAQ and Dow Jones Industrials, but there are hundreds of other indexes for almost every sector and capitalization. With that in mind, I offer the Google Dependency Index, which is composed of a list of public companies that essentially find themselves completely at the mercy of Google. I put this list together mostly as an exercise to quantify just how important Google was to the direct financial performance of other public Internet companies and I have to say that after going through the exercise it has convinced me that Google A) is actually even more powerful than people perceive it to be B) there will inevitably be a backlash against this power.
I should note that this is not an exhaustive list because to a certain degree almost every public Internet company is dependent on Google to some extent, but this list contains examples of companies that arguably have the most significant exposure to Google. So without further ado here is the Google Dependency Index:
Sub-Sector: Direct Dependents (Companies that actually get cash from Google)
America Online (AOL): Ever wonder how much Google pays AOL to be its default search engine? Well in 2009, Google paid AOL $556M or 17% of its total revenues for the privilege. Given that this referral revenue comes with very little costs, it’s a safe bet to assume that the share of AOL profits attributable to Google are probably are least 30% and perhaps higher.
Answers.com (ANSW): Question: How much money you could make if you started a wildly popular web site and only monetized it only using Google Adsense? Answer: you would make about $21M in gross revenues, or at least that’s what Answers.com did in 2009. Answers, according to Quantcast, is the 15th most visited site in United States with 55M monthly unique visitors, most than craigslist or Bing or MySpace. In 2009, Google generated 89% of Answers.com’s traffic and 88% of its revenues.
Infospace (INSP): Infospace has a collection of “meta-search” web sites, the most prominent of which is Dogpile.com. These meta-search sites split any ad revenues from click-throughs with the search sites. 95% of Infospace’s $207M in 2009 revenues came from either Google or Yahoo in return for search traffic and while Infospace doesn’t disclose the split between the two, it’s a safe bet that with Google’s ever increasing market share, Google accounts for an ever increasing share of that 95%.
Incredimail (MAIL): Incredimail is a software developer that used to make it’s money by selling share-ware applications such as a POP3 e-mail client and a universal chat client. It still makes those programs, only now it primarily monetizes those programs not through license fees, but by replacing people’s default search provider with Google and by integrating Google search into its programs. The results? Fully 70% of its $21.9M in revenues during the first 9 months of 2009 came from Google and that percentage is increasing quickly.
Vertro (VTRO): Vertro has a similar model to Incredimail in that they develop a software program which generates revenues by directing search queries back to Google. In Vertro’s case they have a “universal toolbar” called ALOT. The search box on this toolbar points to Google. As a result, fully 90% of Vetro’s $19.6M of revenues in first nine months of 2009 came from Google.
Sub-Sector Indirect Dependents (Companies that depend on Google for traffic, but not direct revenues):
As mentioned before, almost every Internet company is an indirect dependent of Google to some extent given how important search traffic is to revenue generation, but here a few examples of companies that are particularly dependent:
E-Health (EHTH): Quick question: Who has the #1 organic search position on Google for “health insurance” and the #1 paid search position? Why eHealth does. #1 Organic results for high value ecommerce transactions are worth a fortune and these days you can literally build a company on top of them. eHealth doesn’t disclose just how much of its new business is referred to it by Google, but it does note “We depend upon Internet search engines to attract a significant portion of the consumers who visit our web site” in its 10K.
Internet Brands (INET): Internet brands runs a collection of web sites, many of them ecommerce lead gen sites, such as Cars.com. Internet Brands doesn’t publicly disclose how much of their traffic is referred by Google, but it’s enough that their lawyers forced them to include a risk factor which says that search providers drive a “significant amount” of their traffic.
Local.com (LOCM): During the first nine months of 2009, Local.com spent $19M on paid search, $13M of that on Google, and that spending generated 56% of its overall traffic. Local.com’s traffic arbitrage expenses account for almost 45% of its total expenses.
ShutterFly (SFLY): Shutterfly maintains high organic search rankings for its key products including photo printing and photo cards. These high organic results most likely drive a huge amount of its traffic.US Auto parts (PRTS): US Auto parts runs a large network of auto part supply web sites. One of the expressed goals of this “network” is to have multiple PRTS owned sites show up on the front page of organic search results and thereby increase the probability that PRTS will get a click-through one way or another. PRTS doesn’t disclose how important Google’s organic and paid search traffic is to its business, but it is the 2nd risk factor listed in their 10K.
So there you have it, there are at least 10 public companies that basically owe their continued good fortune to Google one way or another and thus clearly belong in the Google Dependency Index. This dependency not only underscores just how important Google is to the overall Internet economy, but it sets the stage for the inevitable backlash against Google because some of these companies will inevitably cry foul should they lose a significant portion of their much prized Google traffic.
Google’s response to such cries will likely be, “You can’t blame us for the changes, our unbiased algorithm did it”, but as I will outline in my next post, Google’s algorithm is rapidly and inevitably becoming anything but unbiased and this transformation will ultimately make Google more evil than good in most companies’ minds.
If you have a suggestion for other public Internet companies that should be included in the index, leave a comment with the ticker and an explanation as I'd be interested to see what nominations other people have.
This is not investment advice, just some observations about how damn powerful Google has really become. Please read my disclaimer at the bottom of this page.