User-agent:* Disallow:/ #Disallow: /perl/ #Disallow: /cgi-bin/ #Disallow: /html/ #Disallow: /htdocs/