Search Engines and Web Directories

Search Engines and Web Directories
Many online shoppers use a search engine or a Web directory when they are trying to find something on the Internet. A search engine is a Web site that indexes the contents of millions of Web pages. A Web directory, on the other hand, organizes Web sites by category so that they can be easily browsed by Internet users. Unlike search engines, directories are usually compiled by human beings.
In the following table, we’ve listed the names and addresses of the most popular search engines and Web directories. Making sure that your Web site is registered with all of these sites is one of the most important things you can do to draw traffic to your store. Why all of them? Your customers (and potential customers) won’t all be using the same search engine or Web directory. Some people use Excite, some use AltaVista, some use Lycos, etc. By registering with all the major search engines and Web directories, you have the best chance of being found by online shoppers regardless of what search engine or Web directory they are using.


Popular Search Engines, Popular Web Directories
Having said that, as we will see, it is not necessarily easy to get listed in these search engines and directories. In fact, you may be required to pay just to be considered for inclusion in their listings.
When you submit a Web site to a search engine, your Web site is added to the search engine’s full text database, but it does not necessarily get your Web site included in the directory portion of the search engine. Many search engines use a different source for the directory listings, so if you want to get your Web site included in the directory portion of the search engine, you often have to go through a separate submission process. The same is true with Web directories (i.e., you may have to go through a separate submission process to get your Web site listed in the search engine portion of the Web directory).


Search Engine and Directory Databases
Before you submit your site to any search engine or Web directory, you need to understand how their databases are developed. A search engine database is significantly different from a directory database. Automated computer programs called spiders develop search engine databases. These programs scour the Internet indexing the full contents (i.e., all of the words on a page) of the millions of Web pages they find. These databases are intended to help you find instances of words or phrases on Web sites, similar to a dictionary or book index.
Web directory databases, on the other hand, list Web sites that have been selected by human beings and organized into distinct categories. These databases are similar to the yellow pages, which organize businesses by topic, and are more useful when you are interested in a specific topic.
The confusion comes from the fact that all the major search engine Web sites now have not only a search engine database, but also a directory database. Often when you do a search on such a Web site, the results that are returned include information from both databases. This is also true of Web directory sites, most of which have a search engine database in addition to the directory database.
As an Internet merchant, the ideal situation is to have your Web site included in both databases of any search engine or directory Web site.

Submitting Your Web Site to a Search Engine Database
When you launch a Web site, you want to make sure that it is included in the search engine databases of both search engine Web sites and directory Web sites. There is a good possibility that the spider programs that these Web sites use will find your Web site, index it, and add it to the search engine’s database. However, it could take months for these spiders to discover your site, if they ever do. For these reasons, it’s generally a good idea to visit each search engine and directory Web site and manually submit your Web site for inclusion in their search engine databases.


Getting Added to a Search Engine’s Database
To add your Web site to a search engine, go to the particular search engine site and look for a link or button somewhere on the main page that says something like “Add Your Web Site”, “Add a Page” or “Add URL” (URL stands for “Uniform Resource Locator” — it means the same thing as “Web address”). Click on the button or link, and you’ll usually be directed to a Web page where you can fill out a form and submit your Web site.



Getting Added to a Web Directory’s Search Engine Database
Web directory sites may have their own search engine spiders that develop their search engine database but quite often the Web directory partners with an existing search engine. Alliances are very common, so it is worth the time to find out who is partnered with whom and submit your Web site to all the relevant search engines. This will maximize the number of search engine databases you appear in.


Submitting Your Web Site to a Directory Database
First and foremost, make sure you register your Web site with the major directories that we listed earlier in the guide, as well as with any other Web directories that pertain to your industry. Look for information on their home pages explaining how your can submit your Web site to their database.
You generally have to decide what category and sub-category would best represent your site. In some cases you do not have a choice. Look through each Web directory’s help section to make sure you understand what is required and how to go about properly registering your site.
Second, visit the Web site of each of the major search engines, and determine who provides their Web directly listings. Then visit each of those Web directories and submit your Web site, if you haven’t already done so.
Is There a Charge to Submit Your Web Site to a Search Engine or Web Directory?
The answer is that it depends. If the submission is free, it can take anywhere from a few weeks to a few months for your Web site to be added to the database, assuming that it is accepted for inclusion.
There is the very real possibility that your Web site will never appear in the search engine’s index.
But don’t let this discourage you! Submit your Web site anyway — you have nothing to lose. And don’t hesitate to keep submitting it again over time.
The problem is that the Internet is growing so rapidly that many of the search engine and Web directory sites can barely keep up with the massive increase in the number of pages and sites. Not only that, but they complain that many Web sites, particularly adult-oriented ones, continually submit requests for them to check new sites. Hence, they have been falling behind in indexing new sites and pages.
The result is that many search engines and Web directories now charge you a fee to submit your Web site for inclusion in their respective databases. However, payment of these fees does not guarantee that your Web site will be included in the search engine or Web directory database. In some cases the fee may only guarantee that your Web site will be considered for inclusion. Check each search engine’s and Web directory’s help files for specific information.
Should You Pay to Have Your Web Site Included?
You may be wondering whether it’s a good idea to pay to have your Web site included in a search engine and/or Web directory. If a paid submission service is available, we recommend that you use it. In some cases, it’s the only way to get your Web site added to a search engine or Web directory.
On those Web sites where both free and paid submission services are offered, we still recommend that you opt for the paid submission. Why? As noted earlier, with free submission, there is no guarantee
that your Web site will ever make it into the search engine or Web directory. By paying, you guarantee that your Web site will at least be considered for the search engine or Web directory in a timely manner. If you use the free submission option, you could be waiting around forever!

Increasing Your Likelihood of Success
Throughout this section, we have talked about the process of submitting your site to search engines and Web directories, and we indicated that you might or might not manage to get listed. Particularly if you are paying a search engine or Web directory to be considered for inclusion in their site, you might want to know if there are ways to design your site so that it is more acceptable for inclusion.
Look through the help files of search engines and Web directories to see if they provide any hints, tips, and other guidance for your site. Some have submission criteria you may need to follow.
There are no hard and fast rules — it can be a very subjective process. Suffice it to say you can be judged on content, appearance, relevance of your site, and other factors. All we can suggest is that you make your site and /or store as comprehensive and professional as possible, which will undoubtedly increase the likelihood of being listed.
Are Search Engines and Web Directories Effective Marketing Tools?
Working with search engines and Web directories can easily consume a lot of time, and many Web site owners wonder whether the effort is really justified. Internet users rely heavily on search engines and Web directories when they are looking for information on the Web, so it’s definitely worth your time to ensure that your Web site is included in all the major search engine and Web directory sites.
At the same time, we caution you against going overboard and relying too much on search engines and directories to drive traffic to your Web site. Search engines and directories are just one part of the marketing mix. Your marketing activities should include using offline media such as radio, television, and print, as well as other techniques that we discuss in this guide.
Later in the guide, we discuss the use of Web site analysis tools such as WebTrends that can help you analyze which search engines and directories your customers are using to find you. These types of tools can help you understand how much traffic on your Web site originates from search engines and directories. Furthermore, using these tools, you’ll be able to quantify the importance of being listed on various search engine and directory sites. One of the authors of this guide, for
example, has discovered that Yahoo! is responsible for a lot of the visits his Web site receives.