mozilla
/
treeherder
зеркало из https://github.com/mozilla/treeherder.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
treeherder/Procfile

14 строки
1.4 KiB
Plaintext
Исходник Обычный вид История

Bug 1164881 - Add MPL2.0 headers to recent treeherder repo files
2015-05-14 18:43:05 +03:00
# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this
# file, you can obtain one at http://mozilla.org/MPL/2.0/.
Bug 1145606 - Setup treeherder to deploy on heroku I added a Procfile listing all the different python services treeherder needs. Heroku provides deployment-specific settings via environment variables, so I had to modify the settings file to listen to them where that wasn't the case. I created an enviroment variable IS_HEROKU which allows to have a heroku-only configuration where needed. The db service is provided by Amazon RDS, which requires a ssl connection. To enable ssl in the MySQLdb python client I had to modify Datasource (and bump up the version used). The cache service is provided by the memcachier heroku addon. Heroku recommends to use pylibmc, so I set it up according to the docs here https://devcenter.heroku.com/articles/memcachier#python. The amqp service is provided by the CloudAMQP addon. I added a post_compile script that runs every time we deploy. We should run every build step we require in there, like static asset minification, collection, etc. To share the oauth credentials among the various services I used an environment variable. I also added an option to export_project_credentials so that the credentials can be printed to stdout. This should come handy when we will need to update the environment-stored credentials with the ones in the db.
2015-05-01 13:01:11 +03:00
web: newrelic-admin run-program gunicorn treeherder.webapp.wsgi:application --log-file - --timeout 29 --max-requests 2000
worker_beat: newrelic-admin run-program celery -A treeherder beat
worker_pushlog: newrelic-admin run-program celery -A treeherder worker -Q pushlog,fetch_missing_push_logs --maxtasksperchild=500 --concurrency=5
Bug 1164868 - Split the buildapi tasks into {pending,running,4hr} queues Previously all three of the buildapi ETL ingestion tasks (pending, running, 4hr) were run under one queue. In order to be able to tell issues (such as backlogs or leaks) apart, these have now been split onto their own queues. On Heroku, these queues now also have a dyno each - which should mean we can easily tell which is leaking and possibly also downgrade from the expensive performance dyno even before the leak is fixed.
2015-05-21 17:44:58 +03:00
worker_buildapi_pending: newrelic-admin run-program celery -A treeherder worker -Q buildapi_pending --maxtasksperchild=20 --concurrency=5
worker_buildapi_running: newrelic-admin run-program celery -A treeherder worker -Q buildapi_running --maxtasksperchild=20 --concurrency=5
Bug 1164888 - Tune Heroku build4hr handling By taking the concurrency down to 1, we can control how much memory is used and increase throughput by assigning more dynos. This prevents us overflowing the memory allocations on each dyno due to the growth of the build4hr process.
2015-06-17 21:19:51 +03:00
worker_buildapi_4hr: newrelic-admin run-program celery -A treeherder worker -Q buildapi_4hr --maxtasksperchild=20 --concurrency=1
Bug 1140349 - Remove the objectstore code After the previous commit, the Objectstore is effectively "dead code". So this commit removes all the dead code after anything left over in the Objectstore has been drained and added to the DB.
2015-06-18 22:50:50 +03:00
worker_default: newrelic-admin run-program celery -A treeherder worker -Q default,cycle_data,calculate_eta,populate_performance_series,fetch_bugs --maxtasksperchild=50 --concurrency=3
Bug 1145606 - Setup treeherder to deploy on heroku I added a Procfile listing all the different python services treeherder needs. Heroku provides deployment-specific settings via environment variables, so I had to modify the settings file to listen to them where that wasn't the case. I created an enviroment variable IS_HEROKU which allows to have a heroku-only configuration where needed. The db service is provided by Amazon RDS, which requires a ssl connection. To enable ssl in the MySQLdb python client I had to modify Datasource (and bump up the version used). The cache service is provided by the memcachier heroku addon. Heroku recommends to use pylibmc, so I set it up according to the docs here https://devcenter.heroku.com/articles/memcachier#python. The amqp service is provided by the CloudAMQP addon. I added a post_compile script that runs every time we deploy. We should run every build step we require in there, like static asset minification, collection, etc. To share the oauth credentials among the various services I used an environment variable. I also added an option to export_project_credentials so that the credentials can be printed to stdout. This should come handy when we will need to update the environment-stored credentials with the ones in the db.
2015-05-01 13:01:11 +03:00
worker_hp: newrelic-admin run-program celery -A treeherder worker -Q classification_mirroring,publish_to_pulse --maxtasksperchild=50 --concurrency=1
worker_log_parser: newrelic-admin run-program celery -A treeherder worker -Q log_parser_fail,log_parser,log_parser_hp,log_parser_json --maxtasksperchild=50 --concurrency=5