зеркало из https://github.com/mozilla/treeherder.git
116 строки
4.7 KiB
ReStructuredText
116 строки
4.7 KiB
ReStructuredText
SETA
|
||
====
|
||
|
||
SETA finds the minimum set of jobs to run in order to catch all failures that our automation has found in the recent past on Firefox development repositories.
|
||
There's one main API that SETA offers consumers (e.g. the Gecko decision task) in order to show which jobs are consider low value
|
||
(less likely to catch a regression). After a certain number of calls, the API will return all jobs that need to be run.
|
||
|
||
SETA creates job priorities for all jobs found in the runnable-jobs API for that repository.
|
||
Initially all jobs will be treated as low value, however, once we run the test to analyze past
|
||
failures we will mark certain jobs as high value (priority=1).
|
||
|
||
Also note that jobs from different platforms (linux64 vs win7) or different CI systems (Buildbot vs TaskCluster)
|
||
will be treated the same (use the same job priority). In other words, a job priority can represent a multiple
|
||
number of jobs.
|
||
|
||
Jobs that appear on Treeherder for the first time will be treated as a job with high priority for a couple of
|
||
weeks since we don't have historical data to determine how likely they're to catch a code regression.
|
||
|
||
In order to find open bugs for SETA visit list of `SETA bugs <https://bugzilla.mozilla.org/buglist.cgi?query_format=specific&order=relevance%20desc&bug_status=__open__&product=Tree%20Management&content=SETA&comments=0&list_id=13358642>`_.
|
||
|
||
APIs
|
||
----
|
||
* /api/project/{project}/seta/{version}/job-priorities/
|
||
|
||
* This is the API that consumers like the Gecko decision task will use
|
||
|
||
* /api/project/{project}/seta/{version}/job-types/
|
||
|
||
* This API shows which jobs are defined for each project
|
||
|
||
* /api/seta/{version}/failures-fixed-by-commit/
|
||
|
||
* This API shows job failures that have been annotated with "fixed by commit"
|
||
|
||
Local set up
|
||
============
|
||
After you set up Treeherder, ssh (3 different tabs) into the provisioned VM and follow these steps:
|
||
|
||
1st tab
|
||
-------
|
||
.. code-block:: bash
|
||
|
||
./manage.py runserver
|
||
|
||
2nd tab
|
||
-------
|
||
.. code-block:: bash
|
||
|
||
yarn start:local
|
||
|
||
3rd tab
|
||
-------
|
||
.. code-block:: bash
|
||
|
||
./manage.py initialize_seta
|
||
|
||
Try out the various APIs
|
||
------------------------
|
||
|
||
* http://localhost:8000/api/project/mozilla-inbound/seta/v1/job-priorities/?build_system_type=buildbot
|
||
* http://localhost:8000/api/project/mozilla-inbound/seta/v1/job-priorities/?build_system_type=taskcluster
|
||
* http://localhost:8000/api/project/mozilla-inbound/seta/v1/job-types/
|
||
* http://localhost:8000/api/seta/v1/failures-fixed-by-commit/
|
||
|
||
* This one won't work until https://bugzilla.mozilla.org/show_bug.cgi?id=1389123 is fixed
|
||
|
||
Maintenance
|
||
===========
|
||
|
||
Sometimes the default behaviour of SETA is not adequate (e.g. new jobs noticed get a 2 week expiration date & a high priority)
|
||
when adding new platforms (e.g. stylo).
|
||
Instead of investing more on accomodating for various scenarios we’ve decided to document how to make changes in the DB when we have to.
|
||
|
||
If you want to inspect the priorities for various jobs and platforms you can query the JobPriority table from reDash:
|
||
Use this a starting query: https://sql.telemetry.mozilla.org/queries/14771/source#table
|
||
|
||
Steps for adjusting jobs
|
||
------------------------
|
||
To connect to Treeherder you need Heroku permissions. Run this from a treeherder checkout:
|
||
|
||
.. code-block:: bash
|
||
|
||
heroku run --app treeherder-prod -- bash
|
||
|
||
Sometimes, before you can adjust priorities of the jobs, you need to make sure they make it into the JobPriority table.
|
||
In order to do so we need to:
|
||
|
||
* Make sure the scheduling changes have made it into mozilla-inbound
|
||
|
||
* SETA uses mozilla-inbound as a reference for jobs for all trunk trees
|
||
* Make sure the job shows up on the runnable jobs table
|
||
|
||
* You can check the `API <https://treeherder.mozilla.org/api/project/mozilla-inbound/runnable_jobs/>`_, however, it can time out
|
||
* You can update the table with ``export TREEHERDER_DEBUG=True && ./manage.py update_runnable_jobs`` (it will take several minutes)
|
||
* Update the job priority table from the shell:
|
||
|
||
.. code-block:: bash
|
||
|
||
from treeherder.seta.update_job_priority import update_job_priority_table
|
||
update_job_priority_table()
|
||
|
||
|
||
If you want to remove the 2 week grace period and make the job low priority (priority=5) do somthing similar to this:
|
||
|
||
.. code-block:: bash
|
||
|
||
./manage.py shell
|
||
from treeherder.seta.models import JobPriority;
|
||
# Inspect the jobs you want to change
|
||
# Change the values appropriately
|
||
JobPriority.objects.filter(platform="windows7-32-stylo", priority=1)
|
||
JobPriority.objects.filter(platform="windows7-32-stylo", expiration_date__isnull=False)
|
||
# Once satisfied
|
||
JobPriority.objects.filter(platform="windows7-32-stylo", priority=1).update(priority=5);
|
||
JobPriority.objects.filter(platform="windows7-32-stylo", expiration_date__isnull=False).update(expiration_date=None)
|