ENH: Add robots.txt

wrdrd · Sep 9, 2014 · a3197a3 · a3197a3
1 parent e7872a2
commit a3197a3
Showing 1 changed file with 5 additions and 0 deletions.
diff --git a/robots.txt b/robots.txt
@@ -0,0 +1,5 @@
+# www.robotstxt.org/
+
+# Allow crawling of all content
+User-agent: *
+Disallow: