The race for the ultimate online search engine continues to unfold as engineers, math geniuses, and computer scientists work to create more efficient and accurate search tools. But a new crop of search services are taking a much different approach: combining great technology with the power of human intelligence.
In recent years, the hottest Web companies have put endless efforts into creating innovative search technologies and complex algorithms -- the computer programs that index the Web and seek to match your search terms to the most relevant sites. Google is perhaps the greatest example, and its algorithm is helping the company try to achieve an awe-inspiring mission: to organize the world's Web information.
Yet, in the past year, it has become clear that even these complex algorithms have their limits. Faced with a tsunami of new Web pages, user-generated content, and black-hat search engine optimizers constantly trying to "game" the algorithm, it has become increasingly difficult for algorithm-based searches to effectively sort through the clutter on the Web. As a result, users waste time searching, and often become frustrated because they never find what they are looking for.
A powerful trend is re-emerging to answer the frustration of online search engine users. We are getting back to the basics of "people-powered search." Now, a growing number of human search guides are becoming available to assist with queries and help determine the most relevant search results. Acting as intelligent filters, human search guides are armed with even better search tools and new collaboration technologies that make it easy for lots of people to contribute at the same time.
The human-powered search engine concept has evolved from the time Yahoo was using human surfers to index the Web back in 1994. Like most companies competing with Google, Yahoo had to give up the human touch because of the explosive growth of content available on the Web. Now, the goal isn't to keep up with the Web, but to use human intelligence to select the best content, filter the junk, and deliver only the most relevant content to other searchers.
Algorithms get us part of the way there. They instantly scour vast amounts of information, tell us if a page contains our search term, how popular it is, and how many other sites link to it. What they can't do -- what only a human guide can do -- is use critical thinking and experience to better assess which content is trustworthy, relevant, and useful.
New services are emerging that combine these powerful technologies with the wisdom, expertise, and skills of people. They are called "human-powered" sites -- they start with algorithm-based search but add a layer of human intelligence. Companies like OrganizedWisdom Health (where I serve as CEO), ChaCha, and Mahalo are human-powered search engines that have taken on this model, hiring freelance and full-time guides to deliver their own expertise and help users make sense of their search results.
These guides come from all over and often include bloggers, librarians, researchers, stay-at-home parents, and students. They're usually paid based on their productivity and the quality of their work. Fore example, at OrganizedWisdom, we pay guides $10 to $15 for every WisdomCard, or search results page, they create on a particular health topic.
Consider the value of human-guided search for the health sector. Good information can have a huge impact on people's lives, but conversely, confusing or misleading information can be detrimental. Spam sites lead users to unproven, sometimes dangerous treatments, and clutter search results pages, causing many users to give up before locating the information they need. In today's managed-care industry, doctors can't afford to take the time to answer all our questions. Good information is available on the Internet -- we just need to cut through the astounding amount of spam and clutter in the healthcare space to get it. Human-powered search is helping do this.
The algorithms aren't going away, of course, nor should they. But in the coming years we will see Web search and other services increasingly run by this combination of human judgment and technology-driven search. It's not humans versus machines, but humans and machines, driving innovation and solving today's greatest online challenges.
I think you are perfectly right. I was reading an on-line article a few days ago about the Semantic search engine and was really impressed. Rather than using human intelligence, equipping search engines with semantic intelligence would by far improved the efficiency and capabilities of today's search engines.Though the ideal search engine may be beyond our reach,at least semantic search technology will bring us closer.Of course it has it own drawbacks in the fact that a searcher has to to search for information in a semantic way instead of using single or double key words but at least we can expect to get better search results than what we are getting now.
I agree that human knowledge and
experience are two of the main contributors in a good database search.
Since there is immense amount of information available on the Internet which
some of those are not reliable?
To judge if a web content is reliable able or
not, it depends on many factors such as author's qualification, data currency,
and the purpose of the content. A search engine is programmed to search for the
key words that had been put in the page title and the content of the page.
Hence, any good web developer is able to make a web site result in a top hit of
the search engine despite its real content. On the other hand, the internet
search guides that are manual are far more convenience to users who use a very
human search content such as: “Where should I go to eat for the coming
Christmas?” and it will help users to get to their information quicker since
the information is already been categorized and reviewed by someone else.
Having said that, human power is one of the most expensive resources; therefore,
if everyone is moving to the subject directories search. We might see more
advertisements on our search page or more and more database will be no longer
free.
Generally speaking, the subject directories and the search indexes should
be combined and balanced out in a way that less human will be required to
intervene but certain pattern of good content can be learned or recognized by
machines.
OK, when it comes to scalability, Google is king! But there is way too much information on the Web for human-powered search engines to thrive for long. I believe semantic search technology is a more viable process that will have a greater influence on the next-generation Web.
"The so-called semantic web will be based on search technology that will enable people to type in their questions using everyday language. Developers of next generation search engines say their technology will “understand” the language within those queries—much more precisely than Internet giant Google’s technology.
The new search engines will then scour databases that have already “read” documents from across the web to find the best matches. Specific answers can be found for queries such as “Which is the best oil company in the world?” or differentiate between “movies by women,” and “movies about women.”
If you want to experience a very good semantic search engine, take a look at Hakia.com. According to the article, it received positive reviews for its ability to quickly find very specific information.
Okay, for those of you who don't know what Cha-Cha
is; it's a site designed for your grandmother.Because Grandma doesn’t get Google.
Cha-cha uses real people, who they pay minimum wage, to answer your queries
over an AIM-style box. And guess what your guides usually use to search? Google.
The outcome? Your guide will find the same information that you could
in a longer period of time and with fewer results.
So, yeah, this site is useful for the internet illiterate but that is a
quickly shrinking group of people.Plus,
the global trend is to create technology that allows customers to do it themselves;
grocery store checkout, airline check-in etc.But we already have that
access with Google.Why would we reverse
the process?
You seem to suggest that we need this because content quality is degenerating on the net
and additionally that this increased clutter causes “users to
give up before locating the information they need” because much of the
information, even when found, is misleading or just plain wrong.You’re right.The internet is full of lots of different types of crap .
This became pretty clear my freshmen year in
college when I entered my symptoms into a WebMD-like site and was informed that
I had Multiple Sclerosis.I didn’t. So, the truth is - I actually like your idea (organizedwisdom.com)
as it seeks to use professionals to validate specific medical sites.That’s smart.And that could be useful in other areas too; online encyclopedias (as used by Wikipedia), How-To sites (like howthingswork.com), recipe
sites, amongst many others.But it’s different
than providing a guide – it’s providing a site that posts good sites.
Though, there may still be work to do.Because even with a network of human filters, there is no way to
guarantee that this info is of quality. (As evidenced by some of the Wiki debate ).
Hello,Steven! Thank you for the interesting article.
Well, first of all, speaking about search in general, people often accuse "bad searching engines", at that time, people just don't formulate the key words properly. there's a good web-site about searching engins that explains a lot
And of course, the competence of the information on the web-sites leaves much to be desired. But I am afraid that not everybody understands that.People get used to trust pseudoscintific information-there as a great discussion about it in public and academic discurses-so I think, people need to be enlightened to use only reliable sources of information,from this it follows the next question: what are reliable sources:))?
The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
The options for finding great health information on the Internet are endless. But as with any good situation, there is a downside. The Web also harbors sinkholes of pseudoscience, shady activities, and, in the worst cases, outright fraud, all published in the name of “health.” Nowadays, you can buy powerful drugs online without a prescription -- some that could kill you if you use them incorrectly.
While Google introduces its new Chrome OS (which I'm hearing will be widely available in one year? Did I mishear that?), IBM announced 10 new products today to help companies using IBM System z mainframe technology.
Smarter Collaboration: How to Thrive in a Challenging Business Environment Market conditions are changing faster than ever, and organizations need to improve their agility and adaptability in order to provide better service and improve processes. The ability to work with customers, business partners, and employees as effectively as possible - while at the same time holding down costs - is a key to success. READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE! REGISTER HERE
Wanted! Site Moderators Internet Evolution is looking for a handful of readers to help moderate the message boards on our site as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?
To save this item to your list of favorite Internet Evolution content so you can find it later in your Profile page, click the "Save It" button next to the item.
The problem with telepresence is that it's not universally accepted, because video calling isn't. While we can all do video calling, we also apparently worry too much about how we look. If we want HD telepresence in our future, we have to dress down, mess up our hair, and dive into our online life.
The US loses about $20 billion a year on pirated software, movies, and music. But public policy can help stem the tide of digital theft. For example, France has recently passed a 'three strikes and you’re out' law, whereby if after two warning letters an individual continues to download pirated software then his Internet access will be cut off. US policy makers should consider adopting similar policies.
Meatballs aren't an obvious source of monetization for the Internet, but Cloudy With a Chance of Meatballs may well be just that, and might also represent a new dynamic in the complex world of Internet content delivery.
Evidence shows that you can tweet too much. Sites and services like Twitter and Facebook are a good place to reach your audience, but think quality over quantity.
A digital content market is emerging. Only two things are known about it: the first is that at some point the Internet will primarily become a paid network. The second known factor is that there are innumerable variables in the digital content market that have yet to be worked out. It’s not known, for example, exactly how users will pay for content (micropayments, subscriptions, bartering of farm animals, other).
Routesy is an iPhone application that uses the phone’s GPS to let the user know where and when the next train or bus is coming. The application’s developer, Steven Peterson, talks about why a mobile application makes sense, especially given that this transportation information is already available on the Web.
Techies have been going crazy over the pending release of Nokia's N900 cellular phone, which incorporates a newly revised touch-screen operating system. Reiter's got one. Is the craziness justified?
Bad news! By eliminating the world’s digital divide we’re likely to create a new divide: the information divide, where we end up creating a two-tier Internet where access to 'quality' content is controlled and charged for by mega-corporations, and the gulf between information haves and have-nots is entirely dependent on how much money they have. This is, of course, an almost exact inversion of the current situation on the Internet – where access is expensive and content is free.
Industry initiatives and government stimulus funds are giving enterprise software vendors a great opportunity to help build out and manage smart grid technologies.
The problem with telepresence is that it's not universally accepted, because video calling isn't. While we can all do video calling, we also apparently worry too much about how we look. If we want HD telepresence in our future, we have to dress down, mess up our hair, and dive into our online life.
The US loses about $20 billion a year on pirated software, movies, and music. But public policy can help stem the tide of digital theft. For example, France has recently passed a 'three strikes and you’re out' law, whereby if after two warning letters an individual continues to download pirated software then his Internet access will be cut off. US policy makers should consider adopting similar policies.
Financial management planning does not need to include Voodoo economics, but it does help to tap into the knowledge base of your team through some sort of real-time system. We explore your options.
When Reiter gets incensed over incompetent Verizon FiOS order-taking and support, he broadcasts it via Twitter. Did it do any good? How should your company offer Twitter support? Watch this for all the answers.
The successor to the BlackBerry Bold 9000 – the Bold 9700 – will be available soon in the US. Is it worth upgrading? Reiter's got one, and offers advice.