Remove special casing of Googlebot and Slurp in robots.txt
authorTom Hughes <tom@compton.nu>
Thu, 27 Jun 2013 17:43:56 +0000 (18:43 +0100)
committerTom Hughes <tom@compton.nu>
Thu, 27 Jun 2013 17:43:56 +0000 (18:43 +0100)
commit39c0df691e03c2ff9c86c86852abb07245f69c34
tree7e07143c1c4b6b7c932fee2946fedab7323d13d7
parent51528b274f450210eb1d07fb5f54f6fd2e795b94
Remove special casing of Googlebot and Slurp in robots.txt
public/robots.txt