add filter for ``exclude_empty`` for result set list

docs/dataload.rst

@@ -1,30 +1,26 @@
 Loading buildbot data
 =====================
 
-There are 2 sources of data (for now) for treeherder: buildapi and pulse.
-Here are the steps needed to load data coming from both the sources.
-
 Buildapi
 --------
-There are 2 celery tasks available in treeherder to fetch data from buildapi:
+This is the list of celery tasks in Treeherder for fetching data from buildapi:
 
+* fetch-push-logs
 * fetch-buildapi-pending
 * fetch-buildapi-running
+* fetch-buildapi-build4h
 
-You can setup those 2 tasks in the django admin interface at /admin/djcelery/periodictask/add/.
+You can setup those tasks in the django admin interface at
+``/admin/djcelery/periodictask/add/``.
 
-The information needed are:
+For each task, set the following fields:
 
 * name: just a description of the task
-* task(registered): choose the task you want to setup up as periodic
+* task(registered): from the list above
 * enabled: true
 
-In the schedule section you can choose between a simple interval and a crontab-like definition.
-Choose the one that best suits your needs.
+In the schedule section you can choose between a simple interval and a
+crontab-like definition.  Choose the one that best suits your needs.
 
-In the arguments section, insert the url of the buildapi service that you want to fetch:
-
-* "https://secure.pub.build.mozilla.org/builddata/buildjson/builds-pending.js" for pending
-* "https://secure.pub.build.mozilla.org/builddata/buildjson/builds-running.js" for running
-
-You can now save and exit. Based on the interval/crontab that you set up, your new periodic task will be executed!
+You can now save and exit. Once you start the celery task process, your tasks
+will be run according to the interval you chose.

treeherder/model/derived/jobs.py

@@ -237,6 +237,9 @@ class JobsModel(TreeherderModelBase):
         if "revision" in kwargs and len(kwargs["revision"]) > 5:
             repl[0] += " AND `rev`.`revision` = '{0}'".format(kwargs["revision"])
 
+        if "exclude_empty" in kwargs and kwargs["exclude_empty"] is True:
+            repl[0] += " AND job.id is not null"
+
         proc = "jobs.selects.get_result_set_list"
         iter_obj = self.get_jobs_dhub().execute(
             proc=proc,

treeherder/model/sql/jobs.json

@@ -227,7 +227,7 @@
                        ON rs.id = rm.result_set_id
                    INNER JOIN revision as rev
                        ON rm.revision_id = rev.id
-                   WHERE job.id is not null
+                   WHERE 1
                    REP0
                    ORDER BY `rs`.`push_timestamp` DESC
                    LIMIT ?,?

treeherder/webapp/api/views.py

@@ -218,7 +218,7 @@ class ResultSetViewSet(viewsets.ViewSet):
         GET method for list of ``resultset`` records with revisions
         """
 
-        filters = ["author", "revision"]
+        filters = ["author", "revision", "exclude_empty"]
 
         offset = int(request.QUERY_PARAMS.get('offset', 0))