Also remove the duplication between the two pages, by having the
submitting data section not mention requesting credentials at all, and
leave that to the common tasks page instead.
If the worker is not running, any `apply_async()` calls are silently
thrown away, due to `ingest_push`'s use of `CELERY_ALWAYS_EAGER` and:
https://github.com/celery/celery/issues/2910
As such, running the worker after ingest_push doesn't help (since the
rabbitmq queues are empty) and so if people are interested in perf/log
data, then they must start the worker first instead.
In update.py, the line outputting revision.txt has to be moved later,
since the `dist/` directory won't exist until grunt build has run. In
addition, since `grunt build` removes the entire `dist/` directory, we
no longer need to manually remove *.gz.
We use the `--production` options for both `npm install` and
`grunt build`, so that the `devDependencies` in package.json are
ignored, and we only install/load the ones listed under `dependencies`
in package.json - since that's all that is required for the build.
We have to use `./node_modules/.bin/grunt` rather than `grunt`, since
grunt-cli is not installed globally on the treeherder admin machine for
greater isolation between stage and production.
Whilst the packages listed in package.json are pinned to exact versions,
they will have their own dependencies, which may be specified via
version ranges. In order to make production/local behaviour more
deterministic, these can be pinned too, using `npm shrinkwrap`.
However the stock shrinkwrap command has a few deficiencies, so we're
using a wrapper around it:
https://github.com/uber/npm-shrinkwrap
Note: Only packages listed under `dependencies` will be shrinkwrapped,
not those under `devDependencies`. This is because using the `--dev`
option (which would include the dev packages in npm-shrinkwrap.json)
means there would then be no way to way to exclude the dev packages when
installing in production.
For more information about shrinkwrap in general, see:
https://docs.npmjs.com/cli/shrinkwraphttp://tilomitra.com/why-you-should-use-npm-shrinkwrap/https://nodejs.org/en/blog/npm/managing-node-js-dependencies-with-shrinkwrap/
And also:
* Explain the process for request/approval on stage/prod
* Remove the unnecessary export_project_credentials step in the
"add a new repository" section, since Treeherder's ETL no longer uses
credentials.json.
Since with the new per-user Hawk credentials, the same auth object can
be used for the whole session, so should just be passed when
instantiating TreeherderClient.
Since they're not specific to the Django app 'webapp'.
Whilst we're there, the local & example settings files have been
renamed. In the future I'd like to combine settings_local.example.py
with puppet/files/treeherder/local.vagrant.py, but I'll do that in
another bug.
Heroku now generates it on deploy, and for stage/prod we generate it
fresh on the stage/prod branch and force push each time. As such, we
have no need for the directory on master, and by removing it we avoid
confusion when new contributors grep the repo.
As an added bonus, the stage/prod deploy script should fail if the dist
directory is missing, so the grunt build cannot be forgotten prior to
deploying. (Currently if it's forgotten, we end up deploying the ancient
dist directory from master that was last updated prior to us switching
to the new deployment strategy.)
This documentation instructs a user on how to setup their local
machine to ingest data from existing exchanges as well as posting
to their own to test their jobs.
The MPL 2.0 terms state that as long as a LICENSE file is present, the
per-file header text is not required. See "Exhibit A" at the end of:
https://www.mozilla.org/MPL/2.0/
Since it can be performed whilst `vagrant up` is running, and whilst
modifying the hosts file is not necessary for running the tests, most
people will still want to do it.
Created using |isort -p tests -rc .| and a couple of manual tweaks.
The order is:
* futures
* std library
* third party packages
* local imports
* relative local imports
...with each group ordered with "import x" before "from x import y", and
then alphabetically.
* Reinforce push must be < 4 hours old
* s/mozilla-central/mozilla-inbound/ (better example, since mozilla-inbound is more likely to have pushes less than 4 hours old)
Since bug 1140349, the objectstore endpoint has been deprecated, and
performs the same function as the jobs endpoint. Now that there are no
remaining submitters to it, let's remove it.
After the previous commit, the Objectstore is effectively "dead code".
So this commit removes all the dead code after anything left over in
the Objectstore has been drained and added to the DB.
This adds the ability to specify a custom log name and have the log
viewer use the ``logname`` param of the ``text_log_summary`` to get the
right log.
This also improves the error message returned by the /logslice/ API if a
log name is used that is not found.
Since it only speeds up parsing by a few percent of total runtime, and
is therefore not worth the added complexity for deployment and local
hack-test-debug cycles when working on the log parser.
The .gitignore and update.py entries will be removed in a later commit,
once the stage/prod src directories have been cleaned up.
In order that we can serve the UI on Heroku, we wrap the Django wsgi app
with WhiteNoise, so both the UI and API requests are served by gunicorn.
In the Vagrant environment, Apache has been removed and Varnish instead
now proxies all requests to gunicorn/Django runserver directly, without
Apache as a go-between.
The UI on production will not be affected by this commit, since the
Apache config there will still intercept requests for the UI assets
rather than proxying them to gunicorn.
It's worth noting too, that we're not able to make use of WhiteNoise's
automatic Django GZip/caching support since that assumes we are using
Django templates and referring to resources using {% static "foo.css" %}
However, we can sub-class WhiteNoise (or more specifically the
DjangoWhiteNoise class) and override the is_immutable_file() method to
add caching support at a later date:
http://whitenoise.evans.io/en/latest/base.html#caching-headers
Documentation for WhiteNoise can be found at:
http://whitenoise.evans.io/
The Varnish config uses |return (pass)| unconditionally, so never
caches anything, so there's no need to suggest restarting it after
making changes to the UI.
William Lachance
William Lachance
Ed Morley
Mauro Doglio
Cameron Dawson
Jonathan French
MikeLing
Vaibhav Agrawal