This adds a `Content-Security-Policy-Report-Only` header for static assets
served by WhiteNoise (such as our frontend), which includes a first pass
at a possible policy that should work for Treeherder.
The header also includes a `report-uri` directive, which points at a newly
added API for collecting CSP violation reports. Reports are logged as
warnings (so will appear in Papertrail) and sent to New Relic as a custom
event. This will allow us to see whether the policy would block valid
requests, so we can refine it prior to converting to the real (ie blocks
things) `Content-Security-Policy` header.
The addition of `ng-csp` to `perf.html` is to enable AngularJS's ngCSP
feature, which turns off use of `eval()` and automatic stylesheet
injection, so that the policy directives `unsafe-eval` and
`unsafe-inline` don't have to be used. This requires us to then manually
import the AngularJS stylesheet to include the styles that would have
previously been injected:
https://docs.angularjs.org/api/ng/directive/ngCsp
See:
https://developer.mozilla.org/en-US/docs/Web/HTTP/CSPhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policyhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy-Report-Only
* Switches from the `ignore` setting to the new `extend_ignore`, which
doesn't overwrite the default ignore list, meaning we no longer have
to duplicate it ourselves.
* Remove the rarely used `[pycodestyle]` config section, since it's
only used when using tools like autopep8, which should really learn
to use the `[flake8]` section themselves.
* Enables the previously ignored F403 and F405 rules, adding `# noqa`
entries to instances that we do not wish to fix.
* Adjust max line length down to 100, since we already disable the
`E501: line too long` rule, making the length mostly redundant
other than in IDEs, where it's probably good to show a warning if
exceeding 100 characters.
* Fixes:
```
treeherder/intermittents_commenter/commenter.py:202:10:
W605 invalid escape sequence '\['
treeherder/intermittents_commenter/commenter.py:202:24:
W605 invalid escape sequence '\]'
treeherder/webapp/graphql/schema.py:7:1:
F403 'from treeherder.model.models import *' used; unable to detect undefined names
```
Closes#4177.
Refs #3425.
Refs #3565.
Since as of #3980 (bug 1470622) the frontend no longer calls the
`/retrigger/` `/cancel/` or `/cancel_all/` Treeherder APIs.
Whilst looking at the pulse related fixtures, I spotted that the
`mock_message_broker` fixture was already unused.
## Rough summary of the changes
### Front end
The auth callback is written in React and lives under the /login.html endpoint. It communicates with Treeherder using the localStorage.
### Credential expiration
The Django user session expiration is set to expire when the client access token or the id token expires (whichever one expires first). These values are controlled by the IAM team. Presently, the access token expires after 1 day and the id token expires after a week. That being said, the session will therefore expire after 1 day. If you want this value change, we simply need to send a request to the IAM team.
### Credential renewal
Renewals are set to happen every 15 minutes or so. The renewal is skewed slightly so that different open tabs don't renew at the same time. Once renewal happens, both tokens are renewed and the Django session is updated.
### Migration
If the userSession localStorage key is not set, then the user will be logged out including logging out from the Django session. In other words, all users will be automatically logged out when the merge to production happens.
BZ_API_URL currently is used for both the bug filer and fetching bug suggestions through the rest of Treeherder. At the moment, both of them point to production bmo's API. Bug suggestions coming from the bugzilla instance that actually has most of the intermittent bugs filed makes sense. Filing bugs from staging Treeherder's bug filer into production bmo makes less sense, so this patch lets you specify a different API URL.
If the new environmental variable isn't present, the bug filer will still default to production bmo, so this should be deployable without any configuration changes, and everything should still just work.
This naming was a relic of the old datasource code we were using. For
the most part, we don't need it. Where we do need it, we should call it
what it is: a repository name.
In this commit, Sheriff access is still maintained in the
Treeherder DB, rather than using the scopes derived from
LDAP.
For local usage with Vagrant, this requires accessing
Treeherder with localhost instead of
local.treeherder.mozilla.org
Loggin in to the Django Admin is not enabled in this
branch. Do use the admin, you must first login through
the normal Treeherder front-end. Then the admin will
be accessible if the user has the privileges to do so.
Persona login will still be technically possible through the
login.taskcluster.net site. But that choice will go away
shortly.
* Bug 1292270 - Pass a User object down to JobManager.update_after_verification.
This is required to create the BugJobMap instance in the post-datasource world.
Add support for matching test failures where the test, subtest, status,
and expected status are all exact matches, but the message is not an
exact match. The matching uses ElasticSearch and is initially optimised
for cases where the messages differ only in numeric values since this is
a relatively common case.
This commit also adds ElasticSearch to the travis environment.
This endpoint receives a http POST request from the UI containing information about the bug to be filed (product, component, summary, version, description), then formats it properly as a submission to Bugzilla's REST API, using a server-side Bugzilla API key, and adding a "treeherder" comment tag.
The API then passes back either the bug ID (if the submission was successful) or Bugzilla's failure response if something went wrong.
We were previously using the same database (test_treeherder) for both the
jobs and reference data model. I centralized the new db name in the test
settings file. All the test requiring the jobs db or its repository counterpart
can now access it using the `test_project` fixture, while utility functions use
directly the metioned setting. Where the project name is hardcoded in a static
file, I just replaced it with the new name `test_treeherder_jobs`
Ed Morley
George Hickman
Hassan Ali
KWierso
Ed Morley
William Lachance
camd
jgraham
Wes Kocher
Mauro Doglio