Граф коммитов

15 Коммитов

Автор SHA1 Сообщение Дата
Armen Zambrano b4463cc9c1
Changes to Heroku Review Apps (#5805)
* For Heroku Review Apps define tasks defaulting to the Firefox CI Taskcluster instance

The Pulse sources are configured differently on Heroku Review Apps because `PULSE_PUSH_SOURCES` and `PULSE_TASK_SOURCES` are defined as arrays rather than a string.

This is because `app.json` is a Json file and we cannot specify an array by mixing single & double
quotes.

* Bump the Redis and JawsDB add-ons to handle load
* Remove ingesting android-components as it causes noise
* Default LOGGING_LEVEL to INFO as we can handle the extra logging
2020-01-14 09:06:22 -05:00
Armen Zambrano G 40b7d98921 Permit listening to a subest of projects
For Heroku Review apps, we're trying to use a small database plan (with limited
number of collections allowed) and a small number of workers processing tasks
in order to reduce the cost of the app.

Using the PROJECTS_TO_INGEST variable we can define a small number of repos to
pay attention to and discard all other tasks.

You can remove PROJECTS_TO_INGEST to go back to using a normal ingestion behaviour.
2019-11-12 09:09:10 -05:00
Armen Zambrano G 8fa5790f09 Heroku Review apps to use random Pulse queue names
This allows all Review apps to use the same Pulse user, however,
they each have their own separate queues.
2019-11-12 09:09:10 -05:00
Armen Zambrano G 37c96d328d Allow using PULSE_URL to define push_sources and task_sources
Defining PULSE_TASK_SOURCES and PULSE_PUSH_SOURCES is a bit of a nightmare
because `app.json` is a Json file and using double and single quotes for a
value in the file is impossible.
2019-11-12 09:09:10 -05:00
Armen Zambrano G f3c113995f Use Tiger plan for Cloudamqp 2019-11-12 09:09:10 -05:00
Armen Zambrano G 45b2f62487 Add JawsDB add-on 2019-11-12 09:09:10 -05:00
Armen Zambrano G 25bfda34f5 Add Papertrail add-on 2019-11-12 09:09:10 -05:00
Armen Zambrano G 5778ad71b1 Adjust workers for Heroku Review Apps
Some workers are set to 0 dynos so we only focus on ingesting pushes and tasks. Nevertheless,
the number of workers can be adjusted after the review app is created. The current settings
are to get us the minimum amount of functionality for the app and the minimum throughput
for the queues not to backlog.
2019-11-12 09:09:10 -05:00
Armen Zambrano G 1af3f69554 If SITE_URL is not available try using HEROKU_APP_NAME 2019-11-12 09:09:10 -05:00
Armen Zambrano G 7da739673e Set first set of env variables 2019-11-12 09:09:10 -05:00
Dustin J. Mitchell 5684321c50 Bug 1574651 - ingest from multiple rabbitmq servers (#5399)
* Bug 1574651 - remove unused JobConsumer and related code

The `update_pulse_test_fixtures` management command listens for job
messages, so is of no use anymore.

* Bug 1574651 - refactor pulse listening to support multiple AMQP servers

This looks forward to supporting ingesting jobs and tasks from multiple
Taskcluster depoyments, each of which is on its own AMQP server (or, at
least, a vhost).

* Bug 1574651 - pass rootUrl from pulse to celery, verify against repository

When jobs and pushes are loaded, the repo's root URL is known.  This
just serves to ensure that the rootUrl for the repo and the rootUrl for
the event match up.

* Bug 1574651 - use root_url from message to make in-job URLs
* Bug 1574651 - update ingest-and-push-tasks to take --root-url
2019-10-10 16:53:18 -07:00
Armen Zambrano G c0880fb908 Fix prettier issue 2019-08-23 09:34:17 -04:00
Armen Zambrano 868bee8551 Add Heroku generated app.json 2019-08-22 14:35:52 -04:00
Ed Morley bf2b5d1370 Bug 1245472 - Heroku: Update to the new release-phase implementation
The beta release-phase feature is making a backwards incompatible change
today - moving from an app.json (and corresponding buildpack) approach
to specifying a `release` Procfile entry that blocks the app deploy.

For more info, see:
https://devcenter.heroku.com/articles/release-phase?preview=1
2016-02-03 12:33:54 +00:00
Ed Morley a1ee98cdd2 Bug 1201455 - Heroku: Run migrate & other DB tasks after deploy
Uses Heroku's new Release Phase beta to run external-resource-touching
tasks (that shouldn't be run during the slug compile) in a one-off dyno
that runs after the deploy. See:
https://devcenter.heroku.com/articles/release-phase?preview=1

We're using the buildpack to help populate the .heroku/bin/release file
from the `post-release` key in app.json:
https://github.com/heroku/heroku-buildpack-postrelease/blob/master/README.md
https://github.com/heroku/heroku-buildpack-postrelease/blob/master/bin/compile
(The devcenter release phase docs page instructions haven't yet been
updated to cover this buildpack).

The tasks being run are the external-resource ones from update.py:
0ecb647f79/deployment/update/update.py (L79-L88)

...minus clear_cache (since we want to stop doing it, bug 1223384), and
export_project_credentials (since Heroku doesn't use oauth any more, and
that code is soon to be ripped out).
2015-11-11 00:23:21 +00:00