treeherder/bin/run_celery_worker

#!/bin/bash

SRC_DIR=$(dirname "$(dirname "${BASH_SOURCE[0]}")")
cd $SRC_DIR

PROJECT_ROOT=$(readlink -f ../)
PATH=$PROJECT_ROOT/venv/bin:$PATH

source /etc/profile.d/treeherder.sh

if [[ ${NEW_RELIC_LICENSE_KEY+isset} = isset ]]; then
    NEWRELIC_ADMIN="newrelic-admin run-program"
fi

LOGFILE=/var/log/celery/celery_worker.log

if [ ! -f $LOGFILE ]; then
    touch $LOGFILE
fi

exec $NEWRELIC_ADMIN celery -A treeherder worker -c 3 \
     -Q default,cycle_data,calculate_durations,fetch_bugs,fetch_allthethings,generate_perf_alerts \
     -E --maxtasksperchild=500 \
     --logfile=$LOGFILE -l INFO -n default.%h
add daemon scripts 2014-01-06 19:49:32 +04:00			`#!/bin/bash`
Bug 1090689 - Add MPL2.0 headers to the repo 2014-11-03 20:20:00 +03:00
Bug 1146184 - Clean up the directory variables in runner scripts The existing name "curr_dir" isn't overly clear - does it mean the current working directory or the directory in which this script exists? It should also be uppercase. I think the behaviour is clearer when reworked like this :-) There's a lot of other cleanup that could be done (eg reducing duplication), but saving that for bug 1153971. 2015-04-15 17:59:39 +03:00			`SRC_DIR=$(dirname "$(dirname "${BASH_SOURCE[0]}")")`
			`cd $SRC_DIR`
fix difference between supervisord v2 and v3 2014-01-08 01:28:48 +04:00
Bug 1146184 - Add virtualenv's bin to PATH rather than using activate In order to rsync the virtualenv as part of deployment, we'll need to use \|virtualenv --relocatable\|. However this does not update the activate scripts (making them unusable), and the changes it makes in the virtualenv bin directory require that the virtualenv's python is first in the path: https://virtualenv.pypa.io/en/latest/userguide.html#making-environments-relocatable As such, we must make the Treeherder runner scripts add the virtualenv bin to PATH instead of using the activate pattern. 2015-04-15 19:13:55 +03:00			`PROJECT_ROOT=$(readlink -f ../)`
			`PATH=$PROJECT_ROOT/venv/bin:$PATH`

add log file/dir creation logic for all processes @mozjcrowe r? 2014-05-31 03:09:11 +04:00			`source /etc/profile.d/treeherder.sh`
add newrelic support to the worker scripts 2014-05-20 19:57:13 +04:00
add log file/dir creation logic for all processes @mozjcrowe r? 2014-05-31 03:09:11 +04:00			`if [[ ${NEW_RELIC_LICENSE_KEY+isset} = isset ]]; then`
add newrelic support to the worker scripts 2014-05-20 19:57:13 +04:00			`NEWRELIC_ADMIN="newrelic-admin run-program"`
add log file/dir creation logic for all processes @mozjcrowe r? 2014-05-31 03:09:11 +04:00			`fi`
add newrelic support to the worker scripts 2014-05-20 19:57:13 +04:00
add log file/dir creation logic for all processes @mozjcrowe r? 2014-05-31 03:09:11 +04:00			`LOGFILE=/var/log/celery/celery_worker.log`

bash needed more spaces 2014-05-31 03:24:20 +04:00			`if [ ! -f $LOGFILE ]; then`
add log file/dir creation logic for all processes @mozjcrowe r? 2014-05-31 03:09:11 +04:00			`touch $LOGFILE`
			`fi`

Bug 1146184 - Add virtualenv's bin to PATH rather than using activate In order to rsync the virtualenv as part of deployment, we'll need to use \|virtualenv --relocatable\|. However this does not update the activate scripts (making them unusable), and the changes it makes in the virtualenv bin directory require that the virtualenv's python is first in the path: https://virtualenv.pypa.io/en/latest/userguide.html#making-environments-relocatable As such, we must make the Treeherder runner scripts add the virtualenv bin to PATH instead of using the activate pattern. 2015-04-15 19:13:55 +03:00			`exec $NEWRELIC_ADMIN celery -A treeherder worker -c 3 \`
Bug 1241583 - celery scripts include autoclassify tasks that shouldn't be currently running These tasks should not be currently running on stage and prod. They will be activated in a later PR. r=emorley@mozilla.com 2016-01-21 22:02:51 +03:00			`-Q default,cycle_data,calculate_durations,fetch_bugs,fetch_allthethings,generate_perf_alerts \`
Bug 1122139 - Split common tasks to separate queues 2015-01-15 22:02:46 +03:00			`-E --maxtasksperchild=500 \`
Bug 1016117 - update celery project layout and worker init script 2014-10-10 20:46:05 +04:00			`--logfile=$LOGFILE -l INFO -n default.%h`