Burnham's Beat

Search Applications : Search Startups Are Dead, Long Live Search Startups

A couple of months ago I had the pleasure of moderating a panel at TIECon on the Search Industry.  Peter Norvig,  Google’s Director of Research, made one comment in particular that stood out in my mind at the time.  In response to a question about the prospects for the myriad of search start-ups looking for funding Peter basically said, and I am paraphrasing somewhat, that search start-ups, in the vein of Google, Yahoo Ask, etc. are dead.  Not because search isn’t a great place to be or because they can’t create innovative technologies, but because the investment required to build and operate an Internet-scale, high performance crawling, indexing, and query serving farm were now so great that only the largest Internet companies had a chance of competing.

Priced Out of the Market
While the comment might strike some as self-serving, the fact of the matter is that it is true.  Any start-up trying to displace Google, Yahoo, or even MSN or Ask (or for that matter any VC trying to fund them) should just get in their car (or hop on a plane) and go look at Google’s new server farms at The Dales in Oregon.  And if that doesn’t convince them they should head up the Columbia river a bit and check out Microsoft and Yahoo’s digs.   The costs to compete in core search , are now simply to high.

Start-up Salvation: Search as Infrastructure
So where does this leave search start-ups?  For many of them it will undoubtedly leave them in the Web 2.0 Deadpool, but some of them may thrive if they adapt to the new reality. This new reality is that search innovation will increasingly be about applications and not about the core infrastructure.  In fact, there’s a good chance that much of the core infrastructure will be available as a service to search-based applications.  Amazon is pioneering this “search as a service” with its opening up of the Alexa crawling and indexing APIs.  While in the blog world, Technorati already provides APIs for developers to integrate their blog search results directly into applications.   Even Google appears to be taking tentative steps in this direction with Google Co-Op which enables users to limit Google’s Search engine to a customized set of sites and keywords.

One can imagine a world in the not to distant future in which an application designer can easily leverage the billions of dollars being spent by Google, Yahoo et. al., by having programmatic access to what is essentially a custom crawl list and a highly filtered index.  In this way search engines, in some respects, may become an infrastructure layer not too dissimilar from the telecommunications networks and internet standards that they themselves are built upon.

For start-ups, having core search become just another part of the Internet’s infrastructure is actually great news.  This frees them from the huge capital costs required to build a competitive core search platform and instead lets them focus on building a great consumer/enterprise application.  The possibilities here are endless and will undoubtedly result in far more innovations than if search engines remained closed systems.

Seeing the Forest For the Trees
While there will undoubtedly be opportunities for start-ups to extend and improve core search (which is where the vast majority of effort has been extended to date), some of the most interesting opportunities will come not from trying to improve the accuracy and context of a single query, but from looking at aggregate information about search indexes, results, and queries across time.  In other words, the marriage of this search infrastructure, with persistent queries and advanced analytics will likely create an entirely new class of applications that generate insights and create value not by finding the specific piece of information someone is looking for, but by analyzing the ebb and flow of information across the web.   It is here, in this new world of search applications that more than a few start-ups are likely to find a happy home.

In my next post I will talk about some of the more promising search applications that may develop in the next few years.

Launching Inductive Capital ... Finally

Well it's October 1st, the beginning of Q4 2006, and I am happy to announce that I have finally launched Inductive Capital, a investment fund targeting public technology companies.  This may seem like a bit of a departure for many who know me primarily from my days as a VC, but those of you who know me from my days as a Wall Street analyst will probably not be all that surprised.

Before I go any further, I should point out that the reason I haven't blogged since mid-April is that as soon as I took the first serious steps to put this fund in place my lawyers advised me that I should stop blogging in order to avoid any chance of running afoul of a number of investment regulations.  (Incidentally, my lawyers are probably the biggest most respected hedge fund lawyers on the West Coast and believe it or not they did not have a single client that had a blog, so needless to say they were a bit hesitant to let me blog with abandon.)  Since Inductive Capital is a hedge fund, and therefore considered to be perpetually in registration, I still face many legal and regulatory restrictions which unfortunately will keep me from writing about the activities of the fund.  I also need to be especially careful to avoid writing anything that could be construed as fund raising for the fund or trying to provide investment advice, so just so we are clear this post is intended as nothing more than update to everyone who reads my blog on what I am doing now.

I want to be equally clear that I intend to continue this blog from now on and pretty much continue to write about what I have been writing about in the past.  I suspect I my have some commentary on the hedge fund industry from time to time, but given that my new fund will continue to be deeply embedded in the high-tech venture capital space I will continue to write about venture capital and interesting new technology trends.  About the only thing I did in the past that I won't be doing now is publishing lists of stock picks, such as my Virtual Stock Portfolio, or offering any kind of investment advice.

What exactly am I doing?  Well I am running a long/short, low net exposure investment fund, otherwise know as a hedge fund.  The investment thesis behind the fund is something I am particularly passionate about and something that is directly based on my experience as both a VC and a Wall Street analyst.  To put it briefly, the fund has been designed to collect information and insights from the private technology market and then try to apply those insights to help make investments in the public market.   Helping me to collect those insights and information is a formal advisory committee composed mostly of VCs and a few other members of the VC-funded ecosystem.  My advisory committee is composed of a great group of folks, all of whom I deeply respect, and I am also lucky to have a great group of investors many of whom have tremendous experience in both the private and public markets.

I named the fund Inductive Capital primarily to reflect the fact that almost all investment decisions are based on inductive logic and this is especially true of a fund that seeks to apply private market insights to the public markets because you must gain multiple data points from a variety of angles and them filter them based on judgement and experience before you can have any confidence that your reasons for making a particular investment are reasonably sound.

Why did I decide to do a public fund instead of staying in venture? Well it honestly was a tough decision, but I think in the short term there are number of very interesting trends that created a real opportunity for a fund such as Inductive and I was compelled to take advantage of that window of opportunity.   Over the long term I am pretty convinced that the VC world and hedge fund world will increasingly overlap so hopefully my absence from the venture capital world will prove only temporary.

Anyway, enough about me and my new firm.  I look forward in the coming weeks to doing some posts on topics I have covered in the past as well as a few new ones and I hope that everyone who has read this blog in the past will continue the dialoge with me as I really enjoy the interactions.