User-agent: * Disallow: /cgi-bin/ Disallow: /entropybanner/ Disallow: /forum/memberlist.php Disallow: /forum/profile.php Disallow: /forum/ Disallow: /blog/ # Disallow: /cercar_google.php # Disallow: /especies-peixos.php # Disallow: /documents-competicions-oficials.php # Disallow: /arxiu-noticies.php # 2010-01-05 # Yahoo! robot insists on visiting 410 Gone URIs months after first request. # Error and access logs become full of noise from this bot, increasing the cost # of site maintenance. # Moreover, stats show this search engine as barely used to access our site. # All considered, it is better to ban it. User-agent: Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp) Disallow: /