Search the Web More Efficiently: Tips, Techniques and Strategies (Part I)

Search the Web More Efficiently: Tips, Techniques and Strategies (Part I)

By Daniel Bazac - November 6, 2003
Copyright © 2003. All Rights Reserved.

Studies show that after email, searching the Web is the most
popular activity on the Internet. Searching is easy; finding
what you're looking for can sometimes be difficult.
Hopefully the advice below will make your next Web search a
breeze.

Do you really need the Web?

Before using the Web to search for information, you'll have
to ask yourself if the Web is the most appropriate medium to
use to find your information. You can find a florist shop in
your neighborhood faster by using the local, printed Yellow
Pages instead of using the Web. And sometimes a library can
give you better, more comprehensive answers than the Web.

However, in most of the cases, the best and fastest way to
find information is... a Web search.

Obviously, the first thing you need to search the Web is a
computer with Internet access.

Before really starting your search, you'll have to decide
which browser you are going to use. As a reminder, a
browser, according to WhatIs.com is a program "that
provides a way to look at and interact with all the
information on the World Wide Web." You can select a
popular browser such as Explorer, Netscape, Mozilla or Opera
or you can use an alternative browser. My favorite: Avant
Browser. (I have NO connection with them)
Keep in mind that some browsers are faster or have
more options. You can download these browsers from their
companies' web sites.

Tools for searching the Web

There are many search tools available: search engines,
subject directories / virtual libraries, invisible (deep)
web databases, meta search engines, etc.

A search engine is a keyword searchable database of Internet
files that uses a software program to continually scour the
Web. The resulting information is then indexed and stored in
its database.

My favorite search engines are:

* Google™ [ http://www.google.com/ ]
* AlltheWeb [ http://www.alltheweb.com/ ]
* MSN [ http://www.msn.com/ ]
* Teoma [ http://www.teoma.com/ ]
* AltaVista [ http://www.altavista.com/ ]
* WiseNut [ http://www.wisenut.com/ ]

A subject directory (web directory) is a searchable
collection of Web pages gathered, selected and organized by
human editors into hierarchically subject categories. A
virtual library is a web directory that includes highly
selective links, chosen mostly by librarians.

Web directories cover a much smaller proportion of the Web
but using them will bring you more highly relevant results.
The largest web directories index a few million pages
compared with the billions of pages indexed by some major
search engines.

Remember that the web directories - like the search engines
- do not search the Web directly. Instead, they search their
own databases of indexed Web pages. Also, be aware that
directories might not be up-to-date. Some search engines are
in fact hybrid search tools because they are both search
engines and web directories. (Google™, for example, has a
search engine and a directory, powered by Open Directory
Project)

Some widely used web directories are:

* Google™ Directory [ http://directory.google.com/ ]
* Open Directory Project (ODP) [ http://www.dmoz.org/ ]
* Yahoo! [ http://www.yahoo.com/ ]
* Zeal [ http://www.zeal.com/ ]
* JoeAnt [ http://www.joeant.com/ ]
* Gimpsy [ http://www.gimpsy.com/ ]

Popular virtual libraries include:

* Librarians' Index to the Internet [ http://www.lii.org/ ]
* Internet Public Library [ http://www.ipl.org/ ]
* The WWW Virtual Library [ http://www.vlib.org/ ]
* Internet Scout Project [ http://www.scout.wisc.edu/Archives/ ]
* BUBL Link [ http://www.bubl.ac.uk/ ]

The so-called invisible (deep) web is a collection of online
information stored in live databases accessible on the Web
but not indexed by traditional search engines. Examples of
excellent invisible web databases are:

* ProFusion [ http://www.profusion.com/ ]
* Invisible-web.net [ http://www.invisible-web.net/ ]
* Complete Planet [ http://www.completeplanet.com/ ]
* Resource Discovery Network [ http://www.rdn.ac.uk/ ]
* direct search (Gary Price) [ http://www.freepint.com/gary/direct.htm ]

A meta search engine (also known as metacrawler or
multithreaded engine) is a search tool that sends your query
simultaneously to several search engines, web directories
and sometimes to the so-called invisible (deep) web. After
collecting the results, the meta search engine removes the
duplicate links and - according to its algorithm - will
combine and rank the results into a single merged list.

Because most of the meta search engines take only the top 10
or 20 from each search engine, you can expect excellent
results, "la crème de la crème."

But be aware that because some search engines and web
directories do not support advanced searching techniques -
such as quotation marks to enclose phrases or Boolean
operators - no results from those search engines will appear
in the meta search engines' results list when those
techniques are used.

Remember, meta search engines do not maintain their own
databases and therefore cannot accept web site submissions.

The best meta search engines are:

* ez2Find [ http://www.ez2find.com/ ]
* Vivisimo [ http://www.vivisimo.com/ ]
* InfoGrid [ http://www.infogrid.com/ ]
* Infonetware [ http://www.infonetware.com/ ]
* iBoogie [ http://www.iboogie.com/ ]

A special kind of meta search engine is the search utility
(also called desktop search programs or client-side search
software). Unlike the web-based meta search engines listed
above, search utilities are software programs that you
download to your computer. The most popular are:

* Copernic [ http://www.copernic.com/en/index.html ]
* Arrow Search [ http://www.rt-software.co.uk/arrow_search/ ]
* SearchRocket [ http://www.searchrocket.com/ ]
* WebFerret [ http://www.ferretsoft.com/index.html ]
* ProtoSearch [ http://www.npccenterprises.com/products.shtml ]

Meta search engines are excellent tools, but they do not
eliminate the need for search engines.

For more about meta search engines, see my article: The Meta
Search Engines: A Web Searcher's Best Friends.

Please see Part II.