User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / User-Agent: msnbot Crawl-Delay: 30 User-agent: Slurp Crawl-delay: 30