From: short <> Date: Tue, 8 Oct 2002 09:21:12 +0000 (+0000) Subject: Explicitely allow all robots everywhere X-Git-Url: https://git.jankratochvil.net/?p=www.energie.vellum.cz.git;a=commitdiff_plain;h=9d92d5cb57115acbb6189fad1e99847123900fc6 Explicitely allow all robots everywhere - used format description http://www.robotstxt.org/wc/exclusion-admin.html - TODO: Forbid indexing of content-concatenated files by tags --- diff --git a/robots.txt b/robots.txt new file mode 100644 index 0000000..eb05362 --- /dev/null +++ b/robots.txt @@ -0,0 +1,2 @@ +User-agent: * +Disallow: