What are Search Engines?
Search Engine has become a "generic" term used to
refer to both crawler-based (Google) and directory-based (Yahoo!) site search
companies. The two types of searches produce listing in significantly different
ways and therefore can produce widely varying results. Search engines create
listings automatically by crawling website and compiling information about that
site into an index. When "searching" one of these indices, results
are presented giving emphasis on specific words. The methodology that
determines specific results from this search and delivery is known as an
algorithm. And these algorithms are proprietary and confidential to each search
engine company.
Crawling a Site
Search Engines that use bots or spiders to crawl a site create
references automatically. People typing in "key?€? words find those words
in the sites list of referenced words and the crawler-based sites serve those
pages where those words are referenced.
These kinds of sites like to see site content changed on a
frequent basis as changes are seen as an indication of new and revised
information. A site that uses bots or spiders to crawl a site eventually finds
these changes and that can affect your ranking in their listings. How you lay
out your pages matters. Page titles, your page content, links to your site, and
other items are all important in determining your rank relevance to the words
being referenced.
Crawler-Based Search Engines Dissected
The first of 3 major components to crawler-based search engines
is the spider (or bot) known as the crawler. A bot visits a site, reads the
main words while eliminating all connector words and follows all links it can
find to other pages within the site. If you change one of your web pages,
search engines eventually find those changes, which can affect how you are
listed. Bots return to sites on a schedule known only to the search engine
company, but more frequently to sites that continually update content.
The words found by the bots are placed in a database called an
index or catalog ?€" the second component of a crawler-based search
engine. This is where your keywords for each page indexed by a search engine
will be searched and produces a results page and the order in which that page
will be displayed. If a page is changed, then this index is updated with that
new information the next time a bot visits the site.
New sites and new pages can take some time before a spider finds
and adds the page to the index, and depending on how the site is constructed
and whether or not a spider can "read?€? a page or links to a page, the
page may never be found, read, or indexed. And pages are not immediately added
to the index as soon as they have been crawled. It may take some time to get
from the read stage to the index. Until the content is added to the index, the
content is not available to the search engines.
Search software may be the most important component to the user
in producing relevant results from the indexed content. The software is a
program that searches through billions of pages of indexed content finding
matches and determining relevance to the terms searched. Each search engine
handles this task differently and that is why each may produce varying results.
You may rank near the top on one and be nearly invisible on another.
Internet
search engines are special sites on the Web that are designed to help people
find information stored on other sites. They accept queries supplied by web
users and return a list of resources that best fit the query of the user. There
are differences in the ways various search engines work, but they all perform
three basic tasks:
- They search the Internet -- or
select pieces of the Internet -- based on important words.
- They keep an index of the words
they find, and where they find them.
- They allow users to look for
words or combinations of words found in that index.
Early
search engines held an index of a few hundred thousand pages and documents, and
received maybe one or two thousand inquiries each day. Today, a top search
engine will index hundreds of millions of pages, and respond to tens of
millions of queries per day
A search engine is a coordinated set of programs
that includes:
- A
spider (also called a "crawler" or a "bot") that goes
to every page or representative pages on every Web site that wants to be
searchable and reads it, using hypertext links on each page to discover
and read a site's other pages
- A
program that creates a huge index (sometimes called a "catalog")
from the pages that have been read
- A
program that receives your search request, compares it to the entries in
the index, and returns results to you
Major
components of crawler-based search engines
Crawler-based search
engines have three major components.
1) The crawler: Also
called the spider. The spider visits a web page, reads it, and then follows
links to other pages within the site. The spider will return to the site on a
regular basis, such as every month or every fifteen days, to look for changes.
2) The index: Everything
the spider finds goes into the second part of the search engine, the index. The
index will contain a copy of every web page that the spider finds. If a web
page changes, then the index is updated with new information.
3) The search engine
software: This is the software program that accepts the user-entered query,
interprets it, and sifts through the millions of pages recorded in the index to
find matches and ranks them in order of what it believes is most relevant and
presents them in a customizable manner to the user.
All crawler-based search
engines have the basic parts described above, but there are differences in how
these parts are tuned. That is why the same search on different search engines
often produces different results. Our comparisons will then be based on these
differences in all three parts.
All search
engines contain the following main components:
|
Spider
|
A browser-like programme that
downloads web pages
|
|
Crawler
|
A program that automatically follows all of the links on each web page
|
|
Indexer
|
A program that analyzes web pages downloaded by the spider and the crawler
|
|
Database
|
Storage for downloaded and
processed pages
|
|
Results engine
|
Extracts search results from the
database
|
|
Web server
|
A server that is responsible for
interaction between the user and other search engine components
|
Different types of search engines
When people mention the
term "search engine", it is often used generically to describe both crawler-based
search engines and human-powered directories however these two types of search
engines gather their listings in radically different ways and therefore are
inherently different.
Crawler-based search
engines, create their listings
automatically by using a piece of software to “crawl” or “spider” the web and
then index what it finds to build the search base. Web page changes can be
dynamically caught by crawler-based search engines and will affect how these
web pages get listed in the search results. Crawler-based search engines
are those that use automated software agents (called crawlers) that visit a Web
site, read the information on the actual site, read the site's meta tags and
also follow the links that the site connects to performing indexing on all
linked Web sites as well. The crawler returns all that information back to a
central depository, where the data is indexed. The crawler will periodically
return to the sites to check for any information that has changed. The
frequency with which this happens is determined by the administrators of the
search engine.
Crawler-based search
engines are good when you have a specific search topic in mind and can be very
efficient in finding relevant information in this situation. However, when the
search topic is general, crawler-base search engines may return hundreds of
thousands of irrelevant responses to simple search requests, including lengthy
documents in which your keyword appears only once.
Human-powered
directories, such as the Yahoo directory, Open
Directory and LookSmart, depend on human editors to create their listings.
Typically, webmasters submit a short description to the directory for their
websites, or editors write one for the sites they review, and these manually
edited descriptions will form the search base. Therefore, changes made to
individual web pages will have no effect on how these pages get listed in the
search results. Human-powered search engines rely on humans to submit
information that is subsequently indexed and catalogued. Only information that
is submitted is put into the index.
Human-powered directories
are good when you are interested in a general topic of search. In this situation,
a directory can guide and help you narrow your search and get refined results.
Therefore, search results found in a human-powered directory are usually more
relevant to the search topic and more accurate. However, this is not an
efficient way to find information when a specific search topic is in mind.
Meta-search engines, such as Dogpile, Mamma, and Metacrawler, transmit user-supplied keywords simultaneously to several
individual search engines to actually carry out the search. Search results
returned from all the search engines can be integrated, duplicates can be
eliminated and additional features such as clustering by subjects within the
search results can be implemented by meta-search engines.
Meta-search engines are
good for saving time by searching only in one place and sparing the need to use
and learn several separate search engines. "But since meta-search engines
do not allow for input of many search variables, their best use is to find hits
on obscure items or to see if something can be found using the Internet.
|
No.
|
Name
|
Language
|
Website
|
|
1
|
|||
|
2
|
|||
|
3
|
|||
|
4
|
|||
|
5
|
|||
|
6
|
|||
|
7
|
|||
|
8
|
|||
|
9
|
Multilingual
|
||
|
10
|
Multilingual
|
||
|
11
|
|||
|
12
|
|||
|
13
|
Multilingual
|
||
|
14
|
|||
|
15
|
|||
|
17
|
Web Search Engines
Typically, Web search engines
work by sending out a spider to fetch as many documents as possible. Another program, called
anindexer, then reads these documents and creates
an index based on the words contained in each document. Each search
engine uses a proprietary algorithm to create its indices such that, ideally, only meaningful
results are returned for each query.
Definition of:browser rendering engine
Software that renders HTML pages (Web pages). It turns the HTML layout tags in the page into the appropriate commands for the operating system, which causes the formation of the text characters and images for screen and printer. Also called a "layout engine," a rendering engine is used by a Web browser to render HTML pages, by mail programs that render HTML e-mail messages, as well as any other application that needs to render Web page content. For example, Trident is the rendering engine for Microsoft's Internet Explorer, and Gecko is the engine in Firefox. Trident and Gecko are also incorporated into other browsers and applications. Following is a sampling of browsers and rendering engines. See HTML and render.
Rendering
Browser Engine Source
Internet Explorer Trident Microsoft
AOL Explorer Trident Microsoft
Firefox Gecko Mozilla
Netscape Gecko Mozilla
Safari WebKit WebKit
Chrome WebKit WebKit
Opera Presto Opera
Konqueror KHTML KHTML
Building a Search
Searching through an index involves a user building
a query and submitting it through the search engine. The query
can be quite simple, a single word at minimum. Building a more complex query
requires the use of Boolean
operators that allow you to refine and extend the terms of the
search.
The Boolean operators most often seen are:
- AND
- All the terms joined by "AND" must appear in the pages or
documents. Some search engines substitute the operator "+" for
the word AND.
- OR
- At least one of the terms joined by "OR" must appear in the
pages or documents.
- NOT
- The term or terms following "NOT" must not appear in the pages
or documents. Some search engines substitute the operator "-"
for the word NOT.
- FOLLOWED BY
- One of the terms must be directly followed by the other.
- NEAR
- One of the terms must be within a specified number of words of the
other.
- Quotation Marks
- The words between the quotation marks are treated as a phrase, and that
phrase must be found within the document or file.
Human-powered search engines rely on humans to
submit information that is subsequently indexed and catalogued. Only
information that is submitted is put into the index
No comments:
Post a Comment