The Right Search Tool

What does all of this research and development add up to? We’re entering what Danny Sullivan, editor in chief of Website Search Engine Land , refers to as the Search 3.0 era.

Sullivan describes Search 1.0 as being the late-90s period of multiple search engines -- AltaVista, Excite, HotBot, Infoseek, Lycos, WebCrawler -- that used simple text-search criteria to find and rank pages. Search for “nerf ball” and you’d get a lot of pages that use the terms “nerf” and “ball” in close proximity to each other. As marketers decoded those Web crawlers, tactics to game the system proliferated, and the usefulness of such blunt-force search tools declined.

That gave rise to Search 2.0 -- a.k.a. the Age of Google. Moving beyond text search, Google’s PageRank system uses a sophisticated algorithm to rank pages according to a set of “off-the-page” criteria, particularly the number and quality of links to a given Web page. That’s the primary form of Web search today, and it works fairly well for an extraordinary number of people.

EVEN GOOGLE HAS ITS LIMITS
Google has been offering its next generation of Web search since May 2007, when it introduced its Universal Search capability to return content from videos, images, news, maps, and books alongside conventional links to Web pages.

At the time, Marissa Mayer, Google’s VP of search products and user experience, described it as “integrating its siloed search engines.” Yet most of us remain firmly rooted in our silos. “We’ve barely scratched the surface with universal search,” Mayer said in her The future of search blog post Sept. 10, where she promised a new interface and user experience related to new media in the coming months.

Marissa Mayer

Google’s VP of Search Products and User Experience looks ahead

Just to begin universal search was an engineering effort that took more than 100 engineers longer than two years, letting Google run a search query across multiple search indexes without multiplying the resulting computational load in a geometric progression.

In her blog post, Mayer cited four big areas search would improve upon in the coming years, of which Media is one, including universal search across types of media. Another is Mode, such as using simpler mobile devices by speaking instead of typing, or even by entering a picture instead of words. The other two are Personalization, including location-based and social connections, and Language, meaning translations across them.

Google isn’t the only company seeking to add features to its search results. It has unmatched resources, but that doesn’t guarantee it will deliver the breakthrough most relevant to business users, acknowledges Peter Norvig, Google’s director of research.

Take universal search. Google serves perhaps the ultimate mass market. It’s in the business of satisfying a high percentage of general search queries, not creating highly specialized tools for data-intensive forms of search such as video, images, and other content. Norvig uses the example of military analysis. A pure-play military research firm with clients searching for photos of nuclear missile sites can afford to spend a lot of time fine-tuning a specialized search app by hand. Even Google, with its 10,000 engineers, can’t lavish that kind of time on a single-application search tool, so it has to “learn by example,” as Norvig puts it. Here are some pictures, here are some words in a caption: Can you associate certain words with certain kinds of pictures and learn to generalize?

“There may be companies that can really focus on one thing, and do a better job,” says Norvig. “That’s up to them.”

— Richard Martin

Now, though, there’s growing realization of the limitations of Google’s PageRank system and its one-search-fits-all results page. Thus the fledgling Search 3.0 movement: the blending of vertical or specialized search results along with the horizontal across-the-Web results normally provided by Google or Microsoft’s LiveSearch.

Google has its own take (see sidebar), but the most interesting action is in startups developing tools that do things the big guys don’t. “That’s where the opportunity is going to be,” says IDC analyst Susan Feldman.

Take Silobreaker, a contextual search engine that “brings meaning and context to Web content,” according to Kristofer Mansson, CEO of the search and relational-analysis provider, who spoke at the Demo conference in Palm Desert, Calif., in January.

Silobreaker is particularly useful for tracking news makers and current-events topics. It provides search results with value-added elements including context extraction (how a person or topic fits with other people, institutions, or categories), mapping, trend tracking (graphing numbers of mentions in the world press), and relationship mapping. The engine draws content on global issues, science, technology, and business from 10,000 news, blog, research, and multimedia sources.

While Silobreaker is being introduced as a free Web tool, Mansson sees opportunity for big companies and government entities, including intelligence agencies. Mats Bjore, the company’s director of business development, helped set up the open-source intelligence efforts for the Swedish Armed Forces.

The coolest part of Silobreaker is its relational mapping, which displays in graphic form the web of people and topics in close proximity to the search term. By manipulating the nodes in the map, you can see how the points of interest relate to one another; clicking on the midpoint between two nodes reveals an article or other content that explains how the two are related.

Businesses could use Silobreaker to track market trends, map connections among news events and products or industries, and trace links between competing executives, their previous companies, and industry developments. Mansson foresees partnering with content publishers or leading companies in different industries to create highly targeted tools that draw in premium content as well as general Web pages, whether the topic is fuel cells, derivative trading instruments, or nanotechnology.

“When general news content is not enough, we could add on research and premium content, together with a domain expert, to create a must-have service for that specific sector,” Mansson said.

Mahalo.com Inc. , launched the same month as Google’s Universal Search, is a back-to-the-future engine that uses human editors to sift through automated search results to separate the wheat from the chaff. Founded by former tech blogger Jason Calacanis, Mahalo employs Wiki-like, community-generated content to improve search results. A search for Pablo Picasso, for example, brings up a quick Picasso biography, a Top 7 Web page ranking, plus links to Picasso works online and related videos.

Employing editors to sift the vast Web, though, seems a Herculean task, even if its goal is to supplement Google, not replace it. Though its investors include CBS Corp. (NYSE: CBS), News Corp. (NYSE: NWS), and Sequoia Capital, Mahalo isn’t destined to be the businessperson’s first stop.

Next Page: Media-Centric Search