treeherder/bin/run_celery_worker

#!/bin/bash

SRC_DIR=$(dirname "$(dirname "${BASH_SOURCE[0]}")")
cd $SRC_DIR

PROJECT_ROOT=$(readlink -f ../)
PATH=$PROJECT_ROOT/venv/bin:$PATH

source /etc/profile.d/treeherder.sh

LOGFILE=/var/log/celery/celery_worker.log

if [ ! -f $LOGFILE ]; then
    touch $LOGFILE
fi

exec newrelic-admin run-program celery -A treeherder worker -c 3 \
     -Q default,cycle_data,calculate_durations,fetch_bugs,fetch_allthethings,generate_perf_alerts,detect_intermittents \
     -E --maxtasksperchild=500 \
     --logfile=$LOGFILE -l INFO -n default.%h
add daemon scripts 2014-01-06 19:49:32 +04:00			`#!/bin/bash`
Bug 1090689 - Add MPL2.0 headers to the repo 2014-11-03 20:20:00 +03:00
Bug 1146184 - Clean up the directory variables in runner scripts The existing name "curr_dir" isn't overly clear - does it mean the current working directory or the directory in which this script exists? It should also be uppercase. I think the behaviour is clearer when reworked like this :-) There's a lot of other cleanup that could be done (eg reducing duplication), but saving that for bug 1153971. 2015-04-15 17:59:39 +03:00			`SRC_DIR=$(dirname "$(dirname "${BASH_SOURCE[0]}")")`
			`cd $SRC_DIR`
fix difference between supervisord v2 and v3 2014-01-08 01:28:48 +04:00
Bug 1146184 - Add virtualenv's bin to PATH rather than using activate In order to rsync the virtualenv as part of deployment, we'll need to use \|virtualenv --relocatable\|. However this does not update the activate scripts (making them unusable), and the changes it makes in the virtualenv bin directory require that the virtualenv's python is first in the path: https://virtualenv.pypa.io/en/latest/userguide.html#making-environments-relocatable As such, we must make the Treeherder runner scripts add the virtualenv bin to PATH instead of using the activate pattern. 2015-04-15 19:13:55 +03:00			`PROJECT_ROOT=$(readlink -f ../)`
			`PATH=$PROJECT_ROOT/venv/bin:$PATH`

add log file/dir creation logic for all processes @mozjcrowe r? 2014-05-31 03:09:11 +04:00			`source /etc/profile.d/treeherder.sh`
add newrelic support to the worker scripts 2014-05-20 19:57:13 +04:00
add log file/dir creation logic for all processes @mozjcrowe r? 2014-05-31 03:09:11 +04:00			`LOGFILE=/var/log/celery/celery_worker.log`

bash needed more spaces 2014-05-31 03:24:20 +04:00			`if [ ! -f $LOGFILE ]; then`
add log file/dir creation logic for all processes @mozjcrowe r? 2014-05-31 03:09:11 +04:00			`touch $LOGFILE`
			`fi`

Bug 1169944 - Use the New Relic wrapper when running scripts locally too Previously if the `bin/` directory scripts were run in the Vagrant environment, the processes run would not have been started via the New Relic wrapper command, since the New Relic licence key is not set. For greater consistency between Vagrant and production, the wrapper command is now always used. This works since `NEW_RELIC_DEVELOPER_MODE` is defined in the Vagrant environment (thanks to a previous commit in the same bug), which prevents the agent from trying to submit real data: https://docs.newrelic.com/docs/agents/python-agent/installation-configuration/python-agent-configuration#developer_mode 2016-05-03 20:40:10 +03:00			`exec newrelic-admin run-program celery -A treeherder worker -c 3 \`
Bug 1255087 - Add detect_intermittents and autoclassify queues to celery worker files This will not cause any additional tasks to actually run since the code that adds messages to these queues is gated on the AUTOCLASSIFY_JOBS setting. 2016-03-09 16:23:44 +03:00			`-Q default,cycle_data,calculate_durations,fetch_bugs,fetch_allthethings,generate_perf_alerts,detect_intermittents \`
Bug 1122139 - Split common tasks to separate queues 2015-01-15 22:02:46 +03:00			`-E --maxtasksperchild=500 \`
Bug 1016117 - update celery project layout and worker init script 2014-10-10 20:46:05 +04:00			`--logfile=$LOGFILE -l INFO -n default.%h`