openstack/openstack-manuals
Code Issues Proposed changes

robots.txt: keep web crawlers out of /trunk

Browse Source 
We don't want users to accidentally refer to documentation for the
upcoming release. Search engines should not lead them to work in
progress.

With this patch, robots.txt instructs web search engines to ignore
docs.openstack.org/trunk/.

Change-Id: I9484d7d527d025c5ff3631bccb9321df583920b3
This commit is contained in:
Roger Luethi 2014-06-22 07:50:53 +02:00 committed by Andreas Jaeger
parent b38b6e6ae5
commit 3558d2961d
1 changed files with 7 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
www/robots.txt
View File

@ -1,2 +1,7 @@
User-agent: Googlebot
Disallow: /*.pdf$
# openstack-manuals/www/robots.txt
User-agent: Googlebot
Disallow: /*.pdf$
# Keep OpenStack users from accidentally using docs for the upcoming release
User-agent: *
Disallow: /trunk/