METACRAWLERS
~~~~~~~~~~~~
Webster's
defines the prefix, meta, as meaning "more comprehensive:
transcending
-- used with the name of a discipline to designate a new
but
related discipline designed to deal critically with the original
one."
Generically relating this definition to Internet search tools,
one
can envision the metacrawler as a comprehensive source for
searching
many engines at once. Unlike search engines, metacrawlers
don't
crawl the web themselves to build listings. Instead, they allow
searches
to be sent to several search engines all at once. The
results
are then blended together onto one page. Upon investigation,
you
will find that the number of search engines that an individual
metacrawler
uses is, on average, between four and eight. Every
metacrawler
that we experienced also used the directory, Yahoo!, as a
part
of the search.
Think
about metacrawlers from the perspective of the auto mechanic.
For
lack of better terminology, metacrawlers have the body of a normal
automobile
with the power of many different racecar engines underneath
the
hood. This type of search tool is versatile and time-friendly
because
it allows you to search multiple search engines
simultaneously.
But you cannot submit your web site to Metacrawlers.
You
can be found in a metacrawler search only by being correctly
submitted
in all of the search engines.
Why
would search engines allow metacrawlers to use their technology?
Metacrawlers
were originally non-profit creations of the computer
departments
of a few universities. Since metacrawlers presented no
direct
competition in the marketplace, search engines knowingly and
openly
provided the use of their technology behind the scenes to
encourage
the educational growth in this field, and to get some free
visibility
for themselves. As the Web has grown, so have
metacrawlers.
Once low-traffic, non-profit, unknown sites, have
become
traffic-heavy, revenue-generating players in the search
industry.
Accordingly, search engines have begun to enter into
licensing
agreements or other business arrangements, rather than
freely
provide, the technology that is underneath the metacrawler
surface.
In other words, without search engines, the metacrawler
could
not exist.
How
does the relevancy work on metacrawlers?
Each
of the metacrawlers creates its relevancy based on the
preferences
of those who designed the search tool itself. Most
metacrawlers
admit that they created their search tool as a response
to
not getting "focused results." Metacrawlers return the most
relevant
searches at all of the major search engines that it utilizes
at
once rather than returning every possible instance of your keyword
search.
That is why you will only see a few returns from each search
engine
rather than all of the returns that you would normally
experience
if you made a search at the search engine itself. Beyond
personal
preferences, relevancy ranking at metacrawlers is based on
how
many search engines a link was found at and how those sites ranked
it.
Keywords that are located in the title and/or description will
also
rank higher in relevancy.
Searching
with a metacrawler allows you to take advantage of all of
the
different approaches to locating pages which contain all of the
closest
references to your keyword search from each service. In
effect,
you are receiving the most polished results from each of the
major
search engines and Yahoo!.
Visit
http://www.stpt.com and search for "metasearch" under the power
search
feature to locate a few metacrawlers. You may find the
PowerSearch
Assistant Internet floating utility helpful. At
http://www.WebPromote.com/directory
you can strategically place your
business
into the major search tools so that Metacrawlers will give
you
visibility.