Bug 1199046 - Add an in-repo robots.txt to be served by WhiteNoise

Stage/prod's robots.txt is currently served by Apache, and is in the
private puppet repo. Since Heroku can't use it, we need to check one
into our repo, that will then be served by WhiteNoise along with the
other static files.

The robots.txt file here is a direct copy of that currently at:
https://treeherder.mozilla.org/robots.txt

Once this change is deployed, we can remove the alias in the stage/prod
Apache config, leaving WhiteNoise to serve this file. As an added bonus,
it will also then be much easier for us to alter it in the future.
This commit is contained in:
Ed Morley 2015-08-27 15:02:47 +01:00
Родитель ad3dc73715
Коммит f7b14e03e7
1 изменённых файлов: 2 добавлений и 0 удалений

2
ui/robots.txt Normal file
Просмотреть файл

@ -0,0 +1,2 @@
User-agent: *
Disallow: /