Explicitely allow all robots everywhere

 - used format description http://www.robotstxt.org/wc/exclusion-admin.html
 - TODO: Forbid indexing of content-concatenated files by <meta> tags

diff --git a/robots.txt b/robots.txt
new file mode 100644 (file)
index 0000000..eb05362
--- /dev/null
+++ b/robots.txt
@@ -0,0 +1,2 @@
+User-agent: *
+Disallow: