Google Base + Vertical Search + RSS = Death of Walled Gardens

A few weeks ago, I promised to write a follow-up on my post about Google Base that detailed how the launch of Google Base might affect the Internet’s so-called “Walled Gardens” (content sites that charge users and/or suppliers for access to their databases).  One month and a long cruise later here it is...

When I think of Walled Gardens on the Internet I am reminded of one of the many priceless scenes from Monty Python’s Holy Grail in which the King of “Swamp Castle” explains that he has built three successive castles in the same place only to see the swamp subsequently swallow each castle.  The king ends his speech by declaring that the 4th castle was the strongest in the land despite all historical experience to the contrary.

Much like the delusional King in the Holy Grail, many of the Internet’s biggest and most profitable “Walled Gardens”, sites such as Monster.com, Realtor.com, Match.com, and even EBay, appear to be in denial about the ultimate destiny of their sites, which is, that they are bound to be subsumed by the larger Internet.

History Repeats Itself
Not that this should come as a big surprise. Just look at the original “Walled Gardens”: Compuserv, Prodigy and AOL.  In their heyday these powerful “online services” controlled every piece of content on their networks and extracted princely sums, often from both content providers and customers alike.  With the advent of the Internet all three of these players tried to hold onto their Walled Garden strategy only to see the growth and diversity of the Internet overwhelm their own suddenly meager offerings in just a few short years.

The New Old Thing
Flash forward to today.  A set of “new and improved” Walled Gardens have been built only now they are known as “paid listings” sites.  Want to find a date?  Better be prepared to pay Match.com a monthly fee.  Want to sell a house?  Better list it though a realtor with access to Realtor.com.  Want to hire someone?  Better pay Monster.com some coin.   You’re a small business and need a heavily trafficked distribution channel?  Be prepared to pay EBay a hefty fee.

The primary value-add that most paid listings sites offer is that they aggregate, structure, and index similar content into one coherent “site”.   Early on in the Internet’s evolution these sites were literally the only place on the web that one could go for this kind of information.  The sites that successfully built scale, brand, and network effects usually ended up the “winners” in their respective categories.  Once they became winners, they were able to charge a healthy premium and thereby, in most cases, become highly profitable businesses.

What A Difference 10 Years Makes
The problem for these Walled Gardens, is that much like the online services before them, they have been built on a quickly shifting and deeply flawed foundation.  The Internet of 2005 is a far different environment than the Internet of 1995.  In 1995, the average user couldn’t spell “Internet”, let alone figure out set up and run their own Internet site.  Even if they did set up their own site, the lack of any mechanisms for other users to easily find a site made operating one’s own site a pointless exercise as it was like building a billboard on deserted island in the middle of the Pacific.

In 2005, things are a bit different.  Not only has the average Internet user become much more sophisticated, but several trends are rapidly coalescing to deliver what could be a “knock out” blow to the Walled Gardens.  These trends include:

1. Self-publishing:  Thanks to dramatically lower hosting costs and greatly improved software, today most businesses and a rapidly increasing number of individuals have their own Internet sites.  Businesses in particular have quickly taken to publishing all kinds of information on their sites.  For example, most companies have a section of their site where they list job openings and in the real estate industry almost every agent has their own web site with detailed descriptions of their current listings.  Of critical importance is that none of this content is behind a “wall”. It is typically out there for anyone who happens to stop by to see.  The net effect of all this self-publishing is that there is now a ton of openly accessible “primary content” just sitting out there waiting to be indexed and manipulated by anyone who chooses to.
2. Pervasive Search: Sophisticated index search has become the glue that ties the entire Internet together.  With search, no site is an island (or a Walled Garden) unless it chooses to be.  Search has in essence leveled the field when it comes to distribution and brand.  You can spend $1M on an ad in the Super Bowl, but it won’t change your search results.
3. RSS:  RSS is a content syndication standard which makes it incredibly easy for people to subscribe to content from a particular site.  Want to be notified every time someone updates their personal profile on their blog, or every time a company adds a new job opening, or every time a realtor gets a new listing?  Subscribe to their RSS feed.  What RSS does is that it provides an automated way for the web to “feed” new information directly to interested people and computers.

Collectively these trends are transforming the Internet from a disparate, sparsely populated frontier with a few major cities into a teeming highly integrated metropolis. It’s as if everyone on the Internet went from living on a deserted island to all living together on the same block in Manhattan.

A World Of Hurt
In such a close-knit world the foundation of paid listings sites is starting to look more than a little shaky.   About the only thing that is still missing is the ability to relate listings within a taxonomy or consistent hierarchy.   However that barrier is also rapidly eroding.   The first visible signs of such erosion came when a set of “vertical” search start-ups put 2 and 2 together and realized that thanks to self-publishing and pervasive search, they could create highly focused “vertical” search engines that trolled self-published Internet sites for specific information and then republished this information as one aggregated site.  (See Jeff Clavier's blog for more in-depth coverage of vertical search.) Seemingly overnight these start-ups have been able to build databases that in many ways rival those that the Walled Gardens have taken years to build.  For example, Trulia and Home Pages are searching individual realtor sites to build up databases of real estate listings that in many areas are bigger and more detailed than the ones available on the venerable Realtor.com.  Simply Hired and Indeed and doing something similar in the job listings space by indexing job listings from company sites and other “open" gardens.

That start-ups have been able to become so big so fast is a very bad omen for the existing Walled Gardens, but it is nothing compared to the world of hurt that is represented by Google Base (and the undoubtedly soon to follow copy cats from Yahoo and Microsoft).  What Google Base represents is an attempt to marry the world’s best search infrastructure and technology with a highly structured and highly automated way of gathering, storing, and presenting listings information.

As I mentioned in my previous post, Google base is essentially the world’s largest XML database.  If you take the time to read through the XML schema you will see that Google has essentially already built all of the components that it needs to enter the vertical search space in a big way; all it needs to do now is refine a few algorithms and flip a switch.  If and when it does so, it will undoubtedly not only the have the largest collection of listings in all major categories overnight but will also have arguably the best distribution channel for those listings on the Internet.  This is not good news if you are currently charging to either to display or to access similar listings (or if you are a vertical search start-up).

It is apparent that Google would prefer to have users flow information to Google Base via automated RSS feeds, and that is still probably the long term goal (services to help users do this have already been created).  However, if they decide to “prime the pump” by doing a little indexing of their own, watch out, because then Google can start automatically feeding millions of listings/day into Google Base.   Shortly thereafter, there will be population explosion of X.Google.com domains, as in Jobs.Google.com, houses.google.com, cars.google.com, etc., etc., etc. as Google launches “listing sites”, which will really be nothing more than a thin skin on top of Google Base and their indexing/crawling engine; think of it as Google News on steroids.

Uh Oh
In fact, if you haven’t checked out Google Base in the last few weeks since the launch announcement you should probably take another look.  In just a few short weeks of beta testing users have voluntarily submitted well over a million items to the database despite the lack of user friendly tools to do so.  Once at the site, take each of the major Walled Garden Sites you can think of and match them up against each of the major categories that Google Base is currently listing.  As a Wall Garden Executive might say: “Uh Oh!” 

With Google Base fully in place (and ultimately similar services from Yahoo, Microsoft, and Amazon), why in world would anyone pay to have their listings displayed or pay to have to access to a database of listings?  After all, if you publish the listing on your own site Google will automatically index it and then list it within Google Base within the next few days and if you want to make sure they get it immediately, you can just submit if directly to Google Base or register your RSS feed with them (a feature I'll bet they are likely to add).   Instead of charging you (or its end users) for the privilege, Google will make money off of the advertising it sells around the listings. Perhaps you may even be able to pay a fee to have your particular listing “advertised” in a preferential position.

Whatever the case, the end result is clear: the Walled Garden sites are in for a world of hurt and unfortunately for them it will probably come much more quickly and severely than it did to Compuserv, Prodigy and AOL.  That’s because the online service providers at least controlled people’s dial-up access to the Internet and thus were initially protected by very high switching costs. No such protections will be afforded to the listings site though.  Since most of them sell short term services with little or no customer lock-in, the effect of services such as Google Base and its associated off-spring, will likely have a much more immediate and painful impact.

That’s not to say that all of the paid listings sites are headed for imminent Armageddon, but it is to say that they shouldn’t renew their lease on Swamp Castle because it is clearly going to get swallowed up yet more time.

I will follow up on this with a post on with a ranking of the “most vulnerable” Walled Garden business and what, if anything, they can do to avoid sinking into oblivion.