# Disallow all crawlers access to certain pages and directories. User-agent: * Sitemap: http://www.club-bdway.com/sitemap.xml Disallow: http://club.bdway.com/peacefully.located/Perepix.htm Disallow: http://club.bdway.com/peacefully.located/malepere.htm Disallow: http://club.bdway.com/peacefully.located/B&B in the Razes.htm Disallow: http://www.club-bdway.com/RAZES%20HILLSIDE%20walks/ Disallow: /Perepix.htm Disallow: /malepere.htm Disallow: /B&B in the Razes.htm Disallow: /peacefully.located/Perepix.htm Disallow: /peacefully.located/malepere.htm Disallow: /peacefully.located/B&B in the Razes.htm Disallow: /RAZES%20HILLSIDE%20walks/ User-agent: msnbot Crawl-delay: 10 User-agent: Teoma Crawl-delay: 10 User-agent: Slurp Crawl-delay: 10 User-agent: aipbot Disallow: / User-agent: BecomeBot Disallow: / User-agent: psbot Disallow: / --------> End of robots.txt file