These were formerly used when submitting job data using the Python client
(support for which was removed in bug 1349182), and in `buildapi.py` as
part of buildbot ingestion (until #4087 / bug 1443251).
This removes the final usages in our tests so we can drop them entirely.
Since the `request` package's `iter_lines()` returns bytes by default.
We cannot pass `decode_unicode=True` to `iter_lines()` since it splits on
Unicode newlines (which can exist in test message output), so instead we
manually `.decode()` each line before parsing it.
Fixes the following exception under Python 3:
`TypeError: a bytes-like object is required, not 'str'`
The test utility `load_exp()` had to be modified to no longer use append
mode when opening the expected output JSON files, in order to fix:
`json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)`
Now that autophone and AWFY have migrated to Taskcluster, there are
no more submitters of jobs to the REST API (confirmed via New Relic
Insights). As such, this deprecated data ingestion method can now be
removed, along with support for API Hawk auth, API POST throttling
and `treeherder-client` job submission capability.
After this lands we'll need to manually drop the `credentials` table.
* Update pytest from 3.7.4 to 3.8.0
* Fix django.core.urlresolvers deprecation warnings
The new version of pytest now correctly catches warnings that occur
within tests/fixtures, which has unearthed new Deprecation warnings
that need fixing to prevent test failures.
Prevents:
```
RemovedInDjango20Warning: Importing from django.core.urlresolvers is
deprecated in favor of django.urls.
```
To avoid unnecessary dependencies, and use a more conventional
django-rest-framework testing approach:
http://www.django-rest-framework.org/api-guide/testing/#apiclient
APIClient has a few API differences:
* `.json` -> `.json()`
* `.status_int` -> `.status_code`
* Get parameters are passed as keyword argument `data` not `params`
* The default hostname is `http://testserver` not `http://localhost`
* Additional HTTP headers are passed directly as keyword arguments,
rather than nested under a `headers` property.
* It doesn't check the status code itself, so explicit checks are
required, along with removing `expect_errors`.
* The `.post_json()` and `.put_json()` methods don't exist.
See also the docs for the Django test client (which APIClient wraps):
https://docs.djangoproject.com/en/1.11/topics/testing/tools/#the-test-client
Whilst making these changes, I also cleaned up the session fetching
in `test_auth.py` and `test_backends.py`, and added a `status_code`
check in `conftest.py`'s `mock_post_json()` - which makes the root
cause of test failures clearer.
Now that no submissions are using revision_hash, it can be removed.
This removes everything but the model field, which will be handled
later.
I've removed revision_hash from the Pulse jobs schema without bumping
the version, which wouldn't normally be ok, but no one is still using
it, and I'd rather have explicit failures later than if we left the
schema unchanged.
pytest treats objects starting with the string "Test" as tests, so an
underscore prefix has been added to prevent warnings of form:
```
WC1 .../test_detect_intermittents.py cannot collect test class 'TestFailureDetector' because it has a __init__ constructor
```
Change of new environment variable `PULSE_PUSH_SOURCES`.
Keep old `publish-resultset-runnable-job-action` task name by creating a
method that points to `publish_push_runnable_job_action`.
Doing so also fixes incorrect line numbers in logs that have Windows
line endings (bug 1328880), hence having to update the expected logview
output.
The tests have to be adjusted to use responses, since requests doesn't
support `file://` URLs.
This import only affects internal treeherder usage, people using the
PyPI package import from the `thclient` subdirectory instead.
Fixes:
treeherder/client/__init__.py:1:1: F401 '.thclient.*' imported but unused
This adds a FailureLine foreign key to TextLogError and duplicates the
best_classification and best_is_verified columns. Classifications are
still set on FailureLine and copied to the corresponding column on
TextLogError.
This is a prelude to future work which will remove classifications from
FailureLine altogether, so that all autoclassification data can be
retrieved from TextLogError.
This adds a FailureLine foreign key to TextLogError and duplicates the
best_classification and best_is_verified columns. Classifications are
still set on FailureLine and copied to the corresponding column on
TextLogError.
This is a prelude to future work which will remove classifications from
FailureLine altogether, so that all autoclassification data can be
retrieved from TextLogError.
* Can no longer store raw artifacts (anything treeherder doesn't understand
is ignored)
* Attempting to retrieve an artifact now returns a 405 (not allowed)
Instead, generate the data when required. We will store the return value
of this in memcache for a day to ensure things are responsive for the sheriffs
when classifying recent failures.
This changes ingestion, the API endpoints, and the frontend to match
the new structure. For now we continue to store text_log_summary artifacts,
though they don't do anything anymore.
New resultsets will still store a value in their ``revision_hash`` field, but it will
just be the same value as their ``long_revision`` field.
This will log an exception in New Relic when a new resultset or job is posted
to the API with only a ``revision_hash``and not a ``revision`` value.
This also switches to using the longer 40 char revisions along side the
12 char revisions. But we leverage the longer ones for most actions. The
short revisions are stored and used so that people and the UI can support
locating a resultset (or setting ranges) with short revisions.
It's almost entirely unnecessary (the few times we need base data
we can generate it ourselves) and can overwrite other test data
if we're not careful. Better just to remove it.
Ed Morley
pyup.io bot
George Hickman
Alisha Aneja
Max Chehab
Ed Morley
jgraham
William Lachance
William Lachance
Shruti Jasoria
Cameron Dawson