git://git.jankratochvil.net
/
www.energie.vellum.cz.git
/ commitdiff
commit
grep
author
committer
pickaxe
?
search:
re
summary
|
shortlog
|
log
|
commit
| commitdiff |
tree
raw
|
patch
| inline |
side by side
(parent:
1cc0875
)
Explicitely allow all robots everywhere
author
short
<>
Tue, 8 Oct 2002 09:21:12 +0000
(09:21 +0000)
committer
short
<>
Tue, 8 Oct 2002 09:21:12 +0000
(09:21 +0000)
- used format description http://www.robotstxt.org/wc/exclusion-admin.html
- TODO: Forbid indexing of content-concatenated files by <meta> tags
robots.txt
[new file with mode: 0644]
patch
|
blob
diff --git a/robots.txt
b/robots.txt
new file mode 100644
(file)
index 0000000..
eb05362
--- /dev/null
+++ b/
robots.txt
@@ -0,0
+1,2
@@
+User-agent: *
+Disallow: