Граф коммитов

26 Коммитов

Автор SHA1 Сообщение Дата
Armen Zambrano c9d0ec610e
Add missing pytest flag to enable Python code coverage (#5923)
Add removed coverage flag #5860
2020-02-04 08:09:23 -05:00
Armen Zambrano 82efbe2967
Add Python test coverage to codecov (#5921)
Fixes #5853 

Co-authored-by: Shubhank Saxena <saxena.shubhank.19@gmail.com>
2020-02-03 12:03:17 -05:00
Armen Zambrano deaf8c850c Bug 1169263 - Switch from Vagrant to docker-compose (#4901)
* Change permissions for docker/entrypoint.sh
* Allow using Django in localhost
* Documentation changes
Allow running shellcheck tests
Use docker-compose for the Selenium tests
Install docs updates
Switch to yarn cache instead of npm
Install UI dependencies
* Add ability to override DATABASE_URL and use a .env file
2019-05-16 14:56:02 -07:00
Ed Morley 0809a5bd57
Bug 1337717 - Update to newer Celery, Kombu, py-amqp and billiard (#4722)
Updating in one go, since the Celery 4 release only supports the newer
versions of its dependencies and vice versa.

Of note, this fixes the unhelpful connection error messages shown when
in fact there was an authentication problem, and brings Celery/Kombu
support for Python 3.7. It's also likely that this will fix the pulse
listener hang seen in bug 1529404.

The new Celery release has renamed a number of the settings.
Most changes were performed by running:

```
celery upgrade settings treeherder/config/settings.py --django
celery upgrade settings tests/settings.py --django
```

The Django integration in celery.py has been cleaned up by following:
https://celery.readthedocs.io/en/latest/django/first-steps-with-django.html

The bug being hit that caused this to be reverted back in #2119/bug 1333079
was due to Celery 4 no longer supporting calling `apply_async()`
with just the `routing_key` - it now has to be called with either just
the `queue`, or else both the `routing_key` and `queue`, otherwise the task
ends up in the `default` queue. Sadly this isn't mentioned in the Celery
breaking changes list - I'll file an upstream issue shortly.

Changes:

http://docs.celeryproject.org/en/master/history/whatsnew-4.0.html
http://docs.celeryproject.org/en/master/changelog.html#rc1
https://github.com/celery/celery/compare/v3.1.26...v4.3.0rc1

http://docs.celeryproject.org/projects/kombu/en/stable/changelog.html#version-4-3-0
https://github.com/celery/kombu/compare/v3.0.37...v4.3.0

https://amqp.readthedocs.io/en/stable/changelog.html
https://github.com/celery/py-amqp/compare/v1.4.9...v2.4.1

https://github.com/celery/billiard/blob/v3.6.0/CHANGES.txt
https://github.com/celery/billiard/compare/v3.3.0.23...v3.6.0
2019-02-28 19:52:22 +00:00
Ed Morley ff2993d2fb
Bug 1330474 - Switch from Python 2.7 to Python 3.6 (#4668)
This makes Heroku/Vagrant use Python 3.6.8 instead of Python 2.7, and
inverts the versions used in the Travis testing matrix - leaving all
unit tests running against Python 2.7 to make it easier to roll back
if needs be. The Vagrant provision script and Heroku Python buildpack will
automatically detect the changed Python version and purge the existing
installation/site-packages prior to installing the new version.

We're using Python 3.6 rather than 3.7, since latest Celery/Kombu do not
yet support Python 3.7 (and we're on an older version anyway; bug 1337717).
Bug 1529243 is filed for updating to Python 3.7 later.
2019-02-21 07:38:24 +00:00
Ed Morley 54fb828ae7
Bug 1528601 - Surface 'BytesWarning's and make them fail the tests (#4655)
Invalid comparisons between bytes and strings will now cause the tests
to fail, making it easier to spot bugs that would appear under Python 3.

If there are any false positives in the future, the errors can be
suppressed using `filterwarnings` in `setup.cfg`.

See:
https://docs.python.org/3.6/using/cmdline.html#cmdoption-b
https://docs.python.org/3.6/library/exceptions.html#BytesWarning
2019-02-20 06:47:39 +00:00
Dave Hunt 1319f57c55
Rename py.test to pytest #dropthedot 2017-11-09 12:55:25 +00:00
Dave Hunt 9fd3c72241
Bug 1340305 - Allow Selenium tests to be run locally 2017-11-09 12:55:25 +00:00
Ed Morley 9a6c79745f Bug 1414255 - Use shellcheck to validate bash scripts
https://github.com/koalaman/shellcheck

Travis has shellcheck v0.4.6 pre-installed, so we only need to install
it in the Vagrant environment.
2017-11-05 20:53:29 +00:00
Ed Morley 934c6a2ab9 Bug 1414255 - Enable bash 'strict mode' in all shell scripts
This makes some of the error checking in `runtest.sh` redundant.
The shebangs have also been made more consistent.

See:
http://redsymbol.net/articles/unofficial-bash-strict-mode/
2017-11-05 20:53:29 +00:00
Ed Morley 270b2b25c1 Bug 1317275 - Run pip check on Travis
Verifies installed packages have compatible dependencies, to help
prevent issues like bug 1324707. This will reduce the time taken to
review pyup bot PRs.

Example output if errors found:
```
Running pip check
celery 3.1.25 has requirement kombu<3.1,>=3.0.37, but you have kombu 4.1.0.
```
2017-08-03 11:40:11 +01:00
Ed Morley 8fae27955a Revert "Bug 1333079 - Switch to new style Celery setting names"
This reverts commit 1db3e2baf4.
2017-02-01 16:28:55 +00:00
Ed Morley 1db3e2baf4 Bug 1333079 - Switch to new style Celery setting names
Converted using:

celery upgrade settings treeherder/config/settings.py --django
celery upgrade settings tests/settings.py --django

As suggested by:
http://celery.readthedocs.io/en/latest/whatsnew-4.0.html#lowercase-setting-names

(I manually reverted the env variable name back to simplify deployment,
we can always clean these up later.)

The Django integration in celery.py has been cleaned up by following:
https://celery.readthedocs.io/en/latest/django/first-steps-with-django.html
2017-02-01 14:11:45 +00:00
Ed Morley f397b9819b Bug 1326207 - Replace Django system check hack with --fail-level
Django 1.10 added a new `--fail-level` option that allows overriding the
default of only exiting 1 for ERROR and above, avoiding the need to rely
on awk hacks:
https://docs.djangoproject.com/en/1.10/ref/django-admin/#check
2017-01-10 20:39:37 +00:00
Ed Morley 9e20b7934d Bug 1247344 - Enable Django system checks during testing and deployment
`manage.py check --deploy` is now run during Travis testing and as part
of stage/prod/Heroku deployment. It checks for a number of common
configuration mistakes & ensures security best practices are being
followed:
https://docs.djangoproject.com/en/1.8/ref/checks/
2016-05-06 16:09:11 +01:00
James Graham 5ff72285c4 Bug 1255087 - Create a lint for ensuring that the heroku Procfile contains the same queues as CELERY_QUEUES in the settings file 2016-03-16 17:46:16 +00:00
Mauro Doglio 5930b4f4c1 Bug 1193836 - Use pytest-django to run tests
pytest-django doesn't setup a test database for every single test, but
only for those tests that actually require a db. Tests that require a db
need to either be marked with `@pytest.mark.django_db` or use a fixture
that has a dependency on `db` or `transactional_db`.
Using a non transactional db would make tests execution much faster, but
unfortunately it doesn't play well with the treeherder datasource
creation so I used a transactional_db.

pytest-django also allows you to specify a settings file to use for
tests in a pytest.ini file, which is nicer than monkeypatch the original
settings file in the pytest session start function 😃.
2015-10-27 11:19:13 +00:00
Mauro Doglio 247cc3de64 Bug 1193836 - Use a dedicated settings file for testing 2015-10-27 11:18:19 +00:00
Ed Morley 5d3bce9802 Bug 1058531 - Don't enable pytest coverage checker in runtests.sh
It shouldn't be enabled by default IMO. For better overall visibility,
we should be using something like coveralls or codeclimate.
2015-10-14 18:30:02 +01:00
Ed Morley c554745a2c Bug 1192957 - Add isort to the Travis run
Any import style deviations will cause the Travis run to fail, with the
diff output to the Travis log.
2015-10-02 17:55:29 +01:00
Ed Morley 90ba77e596 Bug 1192801 - Remove per-file MPL boilerplate since it's unnecessary
The MPL 2.0 terms state that as long as a LICENSE file is present, the
per-file header text is not required. See "Exhibit A" at the end of:
https://www.mozilla.org/MPL/2.0/
2015-08-18 23:32:11 +01:00
Jonathan French 19b71bc4b4 Bug 1164881 - Add MPL2.0 headers to recent treeherder repo files 2015-05-14 11:45:26 -04:00
Ed Morley 30d6576b43 Bug 1139894 - Add flake8 to runtests.sh
flake8 is run on Travis, so let's make it harder to forget to run
locally for people who use runtests.sh. Eventually we should move all of
this to setup.py.
2015-03-16 12:16:17 +00:00
Jonathan French dbb4d11e09 Bug 1090689 - Add MPL2.0 headers to the repo 2014-11-03 13:06:03 -05:00
mdoglio eb11feeace add db setup/teardown to py.test suite 2013-03-19 19:05:04 +00:00
mdoglio a2b0a00fcb add py.test setup with support for coverage 2013-03-11 20:01:17 +00:00