If the application is set up at the server root (e.g. bugzilla.mozilla.org) then this robots.txt disallows robots from scanning the site except for the top level file.

webtools/bonsai/robots.txt

@@ -0,0 +1,4 @@
+User-agent: *
+Allow: /index.html
+Allow: /cvsqueryform.cgi
+Disallow: /

webtools/bugzilla/robots.txt

@@ -0,0 +1,3 @@
+User-agent: *
+Allow: /index.html
+Disallow: /

webtools/lxr/robots.txt

@@ -1,2 +1,3 @@
 User-agent: *
+Allow: /index.html
 Disallow: /

webtools/tinderbox/robots.txt

@@ -0,0 +1,3 @@
+User-agent: *
+Allow: /index.html
+Disallow: /