Add Host: to robots.txt
[rails.git] / public / robots.txt
2015-12-26 Grant SlaterAdd Host: to robots.txt
2014-11-03 Richard FairhurstMerge pull request #24 from zerebubuth/routing-merge
2014-11-03 Matt AmosMerge remote-tracking branch 'upstream/master' into...
2014-11-01 Tom HughesMerge branch 'master' into overpass
2014-06-17 Tom HughesRemove redundant robot rules
2013-11-16 John FirebaughMerge branch 'master' into redesign
2013-10-17 Tom HughesRemove robots.txt block for /user/*/diary
2013-06-27 Tom HughesRemove special casing of Googlebot and Slurp in robots.txt
2012-11-20 Tom HughesMerge branch 'master' into notes
2012-10-26 GrantAdd /user_blocks/ to robots.txt disallow
2010-05-02 Kai KruegerMerge branch 'master' of git://git.openstreetmap.org...
2010-04-27 Tom HughesMerge branch 'master' into terms
2010-04-22 Tom HughesMerge branch 'copyright'
2010-04-20 Ævar Arnfjörð Bjar... Merge branch 'master' into openID
2010-04-20 Tom HughesDon't allow robots to fetch user edit lists
2010-04-19 Tom HughesMerge branch 'master' into openid
2010-03-18 Kai KruegerMerge -r 20198:20540 from trunc into openstreetbugs...
2010-03-03 Grant SlaterUpdate robots.txt to Disallow browser [node|way|relatio...
2010-02-27 Kai Kruegermerge 19889:20181 of rails_port into the openID branch
2010-02-26 Tom HughesNew robots file from Firefishy.
2009-03-08 Tom HughesMerge 12304:14009 from trunk.
2009-02-06 Tom HughesStop robots crawling the object browser.
2008-11-08 Shaun McDonaldResyncing from head 10895:11795
2008-10-26 Grant SlaterBots Disallow /edit* & /login*, Nothing to see here...
2008-10-09 Grant SlaterBlock bots from /trace/, so they don't attempt to eat...
2006-07-28 Steve Coastmore basic framework
2006-07-28 Steve Coastbeginning of osm on rails