Googlebot the well-known crawler. A while ago, a new beta program of Google started. Google Sitemaps. There, Webmasters can upload sitemaps of their sites. Thus, pages not even linked properly can be made accessible to Googlebot easier.

Uploading happens with a well-defined XML format, which is specified on the Google Sitemaps homepage. I made a little php program to generate such sitemaps automatically out of the installation of a phpWebSite. You can even specify multiple phpWebSites on the same server and let the program generate a site map for each.