• The Spider (aka crawler) -  visits web pages that have been submitted, reads the web page text and follows the page links to other pages within your website.
• The Index - the results the spider obtains is put into the index or catalog as available context for the search results. Different Search Functions also have different sizes of Indexes - some can retain lots of the information, some can only hold a little bit of the information.
• The Search Engine software - sifts through the index to find matches to a search and rank them in order of what it believes (based on individual algorithms) to be relevant. 

Examples of 'True Search Engines':  Alta Vista, Direct Hit, Excite, Fast Search, InfoSeek/Go, Google, Inktomi, Lycos, Netscape Search, Northern Light, Web Crawler.

• Directories are human compiled search results.  You submit a short description to the directory for your entire site.   The search results look only in the descriptions submitted.

Examples of 'Directories': Yahoo, Look Smart, Magellen, Open Directory, Snap.

Based on this - you can see getting your site listed and ranked has become almost a science. 

MetaTags

Some Search Functions will provide you a better placement with use of Meta Tags. See the Search Engine General Guidelines listed below for more details.

To help devise a list of keywords-

• Do a search on words that come to mind relevant to your website in a Search Engine you know supports Meta Tags directly or indirectly.  Bring up the web site that ranks highly.  From the web page, right click on the page and select view source.  This will allow you to see what keywords (if any) they have used. Remember they may be receiving their ranking for another source.   Use this strategy for the first 10-20 websites that rank the highest
• Ask your friends / family / neighbor for suggestions for keywords for your site.  You'd be surpassed how many ways people can come up with for looking for a particular item.
• List your keywords individually separated by commas and if you have enough room combine the words for 'natural language' searches.

Frames, Image Maps & More

• Some Search Functions will support 'indexing' a web site that uses Frames, Image Maps, Dynamic HTML pages ....and some won't.

• Some words you use within the pages (usually characters #, etc) will cause the spider (aka crawler) to stop searching your web site...so only a little bit of your web site may be indexed.

• Some spammers use 'Splash Pages' to redirect the spider (aka crawler) to other pages - and have caused some Search Functions not to not index a web site that starts with a 'Splash Page'

• Domain Names - should be www.companyname.com.  Yahoo Search Function will not accept second level domain names.  An example of a second level domain name would be www.hostsite.com/your company.

• Spamming a Search Function may get your website 'blacklisted'. Examples of spamming include submitting the pages too many time (more than once a day), repeating the same word too many times on the page, Invisible Text, Tiny Text, Redirect Pages.

• Lead Times - Different Search Functions have different amounts of time that it takes to get your site indexed.  Alta Vista can have your website indexed within a day or two - others can take up to 4-6 weeks to have it listed. 

Not having all of your website / web pages indexed or 'blacklisted'  are things you want to avoid.

Search Function General Guidelines

Basic Design Considerations

Some Basic Design Considerations we compiled based on our research include:

Submitting Your Site

Automated Submittal

There are several services that will submit your website for 'free' or for cost.  We typically submit manually to the top 10-15 and use an automate link to submit to some of the others.

Be careful using some of the service based ones. I submitted our site for 'promotion' and they began sending out 'spam email' in our name.   Took quite a bit to get 'unlisted' - but in the meantime our company appeared to support spam email.

They will make promises, but if you have reviewed this document you can see that getting ranked is almost a science and just as soon as you think you have it figured out they will change the rules.

Manual Submittal

Prior to submitting, we typically create a document in Word or other text based editor (that way you can cut & paste the information into the typical questions they will ask, and you will have it again when you resubmit the site in 3 months) - In that file we keep:

• Company Name, Address, City, State, Zip, Phone, Fax, Contact email address
• The 25 word description for Yahoo and Directories
• The Keyword list
• The Site Description

Search Engines will typically let you submit to a URL - Find the URLs for submitting your website on our Search Function Details page.

Directories - Directories require you to find the appropriate category for your site and then submit your information. Find the starting URLs for the popular directories on our Search Function Details page.

Using Web Statistics