Граф коммитов

69 Коммитов

Автор SHA1 Сообщение Дата
Andrew Williamson 13a164a3cd
use curlylint (and existing fix template problems) (#15558)
* use curlylint (and existing fix template problems)

* refactor submit/done template
2020-09-29 10:56:33 +01:00
William Durand c2e92e7482
Introduce Prettier (#15131) 2020-08-03 14:55:07 +02:00
William Durand 1d37fe4122
Fix dates in stats export links (csv/json) (#15086) 2020-07-30 14:00:07 +02:00
Andrew Williamson 5fdf333e3a
shorter makefile recipes for common tasks 2020-05-15 14:53:13 +01:00
dbxnr 0c845636a9 Remove code for ui tests 2020-01-13 23:15:17 +00:00
jrbenny35 13dba0d358 Add reindex after addon creating for ui-tests. 2019-12-17 11:28:38 -06:00
Christopher Grebs b9ac5f4413
Run dockerfile commands with correct user. (#12579)
This updates various things…

* make use o `--user` argument of pip and `$PYTHONUSERBASE` to install python libraries in the correct places where the `olympia` user has permissions
* Add a new `rootshell` command

Fixes #12366

* Correct if statement

* Use separate tag for easier testing

* Try running directly with user olympia

* Small cleanup

* Add docs

* Fix comment

* Add 'rootshell' command

* Build circleci for local dev

* Fix pip execution by using 'python -m' to fetch the correct pip

* Pick up upstream user/group for builds and implement PYTHONUSERBASE and PIP_USER

* Correctly pass USER_ID and GROUP_ID to Dockerfile when building

* Small cleanups

* Build our test branch

* Fall back to manual uploads for testing

* Cleanup

* Revert back to :latest tag
2019-10-22 11:24:05 +02:00
Christopher Grebs 3f454bd39b
Update supervisor to 4.0.4
Fixes #11165
2019-10-07 06:34:26 +02:00
Andrew Williamson d7ae3e5797
Merge pull request #12349 from eviljeff/12333-switch-to-django-migrations
switch to django migrations
2019-09-30 11:08:57 +01:00
William Durand ecab073da1
Fix `make` when running it inside a Docker container (#12345) 2019-09-12 18:48:45 +02:00
Andrew Williamson f1ee9f49ca run django migrations on intialize/update rather than schematic 2019-09-12 16:25:46 +01:00
William Durand 464abd0d2c
Autogenerate makefile help (#12220) 2019-08-29 15:34:26 +02:00
Andrew Williamson 6748a96134
move all the make commands that operate on a single container to Makefile-docker (#12121)
* move all the make commands that operate on a single container to Makefile-docker

* add default user to Dockerfile

* revert the PYTHON_COMMAND var change
2019-08-29 12:10:25 +01:00
Christopher Grebs ec7becb5a7
Use django-babel for for Django 2.x compatibility
Fixes #11233
2019-05-28 09:23:20 +02:00
Mathieu Pillard 7999633e4f Fix populate_data target in Makefile-docker (therefore fixing initialize)
Fixes the following:
% make populate_data
export UITEST_FXA_EMAIL := uitest-87b0c546-6b2e-11e9-8d08-0242ac120008@restmail.net
/bin/sh: 1: export: :: bad variable name
Makefile-docker:88: recipe for target 'populate_data' failed
make: *** [populate_data] Error 2
2019-04-30 11:59:02 +02:00
Andrew Williamson 6aebb99b1f
remove support for django1.11 and python2 (#11195) 2019-04-15 15:17:10 +01:00
Benjamin Forehand Jr 1cbf6a0da8
Move fxa account creation commands in makefile. (#11069)
* Move fxa account creation commands in makefile.
2019-03-29 11:39:57 -04:00
Benjamin Forehand Jr 7bcc1641ae
Fix makefile command for creating data. (#11053) 2019-03-29 09:09:22 -04:00
Benjamin Forehand Jr 9f8f44dfe6
Add devhub upload, approve, install (#11021)
* Add devhub upload, approve, install integration-test
2019-03-28 09:05:41 -04:00
Andrew Williamson ec4ac13ad7
deploy django22 on -dev & defaul on local dev (#11025) 2019-03-27 11:37:52 +00:00
Andrew Williamson 85e9ea6f03
use utf8mb4 everywhere; replace reset_db (#10935) 2019-03-18 10:57:49 +00:00
Benjamin Forehand Jr 2d99d6f3da
Fix ui tests for new nginx config (#10888)
* Fixes for Ui-tests using new nginx config.
2019-03-11 19:20:06 -04:00
Andrew Williamson 70276d2b43
Enable django2.2 testing (#10752) 2019-02-25 12:39:13 +00:00
Mathieu Pillard 1d559fcd32
Fix codestyle and docs jobs (#10683)
* Fix docs for python 3

* Make travis behave like it's inside our container, fix tox/makefile accordingly

* Fix docs build by choosing another label than _search (which is already used)

* Fix codestyle linter issues

* Remove extra force_bytes import

* Add ignore for int|null in docs as it's used in a couple places and it's valid

* Fix more docs build failures in reviewers.rst

Also build docs with -W by default to make future docs breakage
easier to spot locally.

* Leave -f Makefile-docker for UI Tests as they are not running from the standard container
2019-02-13 16:21:05 +01:00
pyup.io bot 1b7c03afb2 Update flake8 to 3.7.5 (and add py3/py2 compat) (#10592)
This PR updates [flake8](https://pypi.org/project/flake8) from **3.7.3** to **3.7.5**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.7.5
   ```
   -------------------

You can view the `3.7.5 milestone`_ on GitLab for more details.

Bugs Fixed
~~~~~~~~~~

- Fix reporting of pyflakes &quot;referenced before assignment&quot; error (See also
  `GitLab!301`_, `GitLab503`_)


.. all links
.. _3.7.5 milestone:
    https://gitlab.com/pycqa/flake8/milestones/28

.. issue links
.. _GitLab503:
    https://gitlab.com/pycqa/flake8/issues/503

.. merge request links
.. _GitLab!301:
    https://gitlab.com/pycqa/flake8/merge_requests/301
   ```
   
  
  
   ### 3.7.4
   ```
   -------------------

You can view the `3.7.4 milestone`_ on GitLab for more details.

Bugs Fixed
~~~~~~~~~~

- Fix performance regression with lots of ``per-file-ignores`` and errors
  (See also `GitLab!299`_, `GitLab501`_)


.. all links
.. _3.7.4 milestone:
    https://gitlab.com/pycqa/flake8/milestones/27

.. issue links
.. _GitLab501:
    https://gitlab.com/pycqa/flake8/issues/501

.. merge request links
.. _GitLab!299:
    https://gitlab.com/pycqa/flake8/merge_requests/299
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/flake8
  - Changelog: https://pyup.io/changelogs/flake8/
  - Repo: https://gitlab.com/pycqa/flake8
</details>
2019-02-07 07:56:39 +01:00
Andrew Williamson 57c606e215
move schematic from mozilla/schematic (#10537) 2019-01-30 20:30:31 +08:00
Andrew Williamson 3290551aa9
remove webextension permission descriptions (#10514) 2019-01-28 18:15:56 +08:00
Christopher Grebs ca15cf8108
Call generate_jsi18n_files on `make update_assets`. (#10467)
* Call generate_jsi18n_files on `make update_assets`.

Fixes #10438 and thus local docker instances and circleci tests.

* Actually call update_assets in circleci
2019-01-21 10:09:18 +01:00
Mathieu Pillard af842876b7 Install Python3 in development container
This also adds support for specifiying which version to use when installing
packages, though at the moment it's not possible to switch python versions
without reinstalling everything, because both versions share /usr/local/bin
2019-01-14 16:19:47 +01:00
Christopher Grebs 04fe02162d
Remove django-product-details dependency. (#10371)
* Add new olympia.core.languages module as a replacement for supported
  languages
* Implement new ApplicationsVersions.get_latest_application_version
helper

Fixes #9516 

Fixes #8850 along the way, removes the need for #8779 and unblocks #2739
2019-01-14 14:33:56 +01:00
Andrew Williamson 76bd95e227
Use supervisor from master rather than the release on pypi to get py3 suport (#10340) 2019-01-09 01:25:40 +08:00
Christopher Grebs 69aba0e9ef
Remove raven.js from legacy + devhub pages.
Unless I'm mistaken there isn't anyone to really look at this and most
errors and problems should be related to legacy pages anyway.

Let's remove things we're not using or looking at.

Fixes #4845
2018-12-03 14:11:49 +01:00
Christopher Grebs b75709c89b
Attempt at fixing 'assets' job.
Let's just remove the manual `update_assets` call and document that
whenever it get's changed / updated the appropriate pytest tests should
be adapted too.

That way we only run it once and potentially fix our unstable 'assets'
job.

Fixes #9656
2018-10-11 10:35:46 +02:00
Andrew Williamson 9ecfbcf964
actually have makefile install python3 deps (#9628) 2018-10-09 10:32:44 +01:00
Andrew Williamson 3d03792f23
have Travis test on py3.6 as well as 2.7 (#9345) 2018-09-05 10:23:49 +01:00
Christopher Grebs 2571832756
Generate more useful data that can be used with addons-frontend. (#9289)
This is needed for mozilla/addons-nginx#7 and #9080 to be more useful.
The data that `generate_ui_test_addons` generates can directly be used
in the new frontend homepage.

This PR also stops seamonkey and thunderbird add-ons to be generated
by default.
2018-08-29 16:13:22 +02:00
Benjamin Forehand Jr 80bbc69dfd
Added configuration to run multiple ui tests in parallel. (#9266)
* Changed fxa account generation to allow assigning an fxa ID to a created user.
* Moved pytest execution to makefile to access variables.
* Removed second login tests as the devhub tests also log in.
2018-08-28 14:48:35 -04:00
Benjamin Forehand Jr c8d2ddbd9d
Revert "Enable circleci integration tests (#9075)" (#9108)
Until we can figure out a solution or a method of testing the login that will work for multiple integration test suits running they should be disabled.
2018-08-08 10:24:23 -04:00
Benjamin Forehand Jr 29a2748b0b Enable circleci integration tests (#9075)
Temp fix to get the integration-tests running. Should fix #9062 as the branch I am using is pretty much what the official release will be.
2018-08-07 21:53:42 +02:00
Mathieu Pillard 94e2889fda Work around 'Multiple .dist-info directories' error when updating deps 2018-07-20 13:37:32 +02:00
pyup.io bot 213e5a5730 Update django-extensions to 2.1.0 (#8892)
* Update django-extensions from 2.0.7 to 2.1.0
* Remove our no-utf8 hack
2018-07-17 09:40:45 +02:00
Benjamin Forehand Jr cafc5d0519
Add UI devhub login and upload tests (#8559)
* Added devhub login and upload tests.
* Add ui-test webext.
* Remove management commands.
2018-07-13 08:55:40 -04:00
Christopher Grebs 7407b70aa2
Update to Django 1.11 (#8738)
Port code and tests to Django 1.11.

* Update requirements to Django 1.11.14, remove session_csrf
* Use --run-syncdb for migration calls
* Remove unused setup-docker script
* Port code and tests to Django 1.11.
* Remove django-session-csrf everywhere, it got replaced by django's own
  session csrf protection
* Make sure we are calling `client.get()` with `follow=True` where necessary
* Replace django.core.urlresolvers import with django.urls
* Port custom `IconWidgetRenderer` to Django 1.11 as `IconTypeSelect`
* Properly use `_queryset_class` for custom manager classes
* Make use of `base_manager_name` on our models `Meta` instead of `use_for_related_objects`
  on the manager where appropriate. This is required to make translations, soft-delete
  and more work. See comment on Addons.Meta for more details.
* Various fixes along the way to make tests work
* Port form errors for locale editing forms to Django 1.11 and make use of
  form.add_error instead of our own custom process
* Mark TestDetailPage.test_304 as xfail for now, somehow Etags seem to be broken
  for that view during tests but work fine in manual tests
* Fix amo.celery, we cannot have any imports at the module-level that may
  import the ORM
* Fix global_settings context processor for 500-server error cases
* Fix `changed_data` handling in `AMOModelForm`
* Remove `queryset_transform` and implement it directly in amo/models
* Fix pagination, Django changed how `num_pages` work and how `count` can be overwritten
* Add a new test that tests database transforms as `BaseQuerysetTestCase`
* Make sure `legacy_redirect` only appends `?` if there are actual GET parameters
* Re-implement `LicenseRadioChoiceInput` as `LicenseRadioSelect` which is now Django 1.11 compatible
* Remove our custom `form_kwargs` uplift from Django 1.9
* Correct how we're using `Category.objects.create` and only pass `db_name` instead of `name`, previously
  this didn't error but now it (luckily!) does
* Update `FileSelectWidget` to correctly work with Django 1.11, use custom rendering
* Fix decorator ordering for `files.views:browse`
* Update database settings to correctly use `TEST` object
* Fix `select_related` call in `RatingViewSet`
* Remove custom `DateTimeCast` implementation
* Fix `addon_factory` calls in `reviewers/tests/test_views.py` to correctly set
  `is_webextension`
* Port our `TranslationDescriptor` and `TranslatedField` to Django 1.11. Also, make sure form error handling works
* Add a new test that checks that `default=None` is a valid setting for translation fields
* Add a new test that checks that translations from related fields are correctly fetched
* Remove possible ambiguity from translations `TestWidget` tests
* Make sure that the correct `MEDIA_URL` is set in `TestDownloadsUnlistedVersions` to fix
  external cdn tests

* Remove custom happyforms code. Django cleans text fields now by default
* Move translations app to the top of INSTALLED_APPS, add a lengthy comment
* Pass additional arguments to contribute_to_class, fix smaller nits
* Set MEDIA_URL and STATIC_URL to an external value by default. That's how they're configured in dev, stage and prod anyway.

Please note that we didn't replace all our custom form rendering with proper templated widgets yet. This will
happening in some follow-up work.

Fixes #5271, Fixes #1569
2018-07-11 06:56:54 +02:00
Christopher Grebs 1bf79b82fa
Fold jingo_minify into our code.
This will simplify the upgrade to Django 1.11 and allow us to keep this
under control, upstream isn't properly maintained.

* Cleanup jingo-minify, keep only parts we are using. Cleanup tests

Also fix and simplify compress_assets to link to the correct /static/ folder.

Originally we hardcoded to look for settings.STATIC_ROOT but that
doesn't suffice since it's set to /site-static/ by default.

We are now using hardcoding the source of our static files instead of
guessing and mistakenly re-using STATIC_ROOT.

This commit also removes verbosity as an argument and simply logs all
errors immediately.

To simplify testing it adds a --force parameter.

To verify this works locally:

* Change settings.STATIC_ROOT to `'/site-static/`'
* Add `/site-static/` to the folders served by nginx (I'll open an issue
on our nginx container to serve this by default too)
* You may have to restart the nginx container after that for nginx to
pick things up
* Remove everything inside site-static folder
* Run `make update_assets`

Fixes #8532
2018-07-09 16:14:42 +02:00
Christopher Grebs 812d885b64
Revert "Fold jingo_minify into our code (#8694)"
This reverts commit 762142ee5a.
2018-07-05 15:19:07 +02:00
Christopher Grebs 762142ee5a
Fold jingo_minify into our code (#8694)
This will simplify the upgrade to Django 1.11 and allow us to keep this
under control, upstream isn't properly maintained.

* Copies jingo-minify into our code
* Removed all parts we aren't using (yui, git integration, `{{ inline_css }}` template tag etc)
* Try to clean up how subprocess calls are being made
* Cleaned up tests

This doesn't change anything regarding how jingo-minify works. It's still using good old `cat` and other ugly things but the code now looks nicer.

See 742297fef9 for more specific details about the cleanup from the original jingo-minify dump.

Fixes #8532
2018-07-02 11:14:53 +02:00
Christopher Grebs ceae9d4fd0
Remove cache machine (#8606)
Fixes #7743
2018-06-28 13:20:06 +02:00
Andrew Williamson fd916c252e
split local dependencies from pure test dependencies and only test deps in travis (#8639) 2018-06-25 11:53:45 +01:00
Christopher Grebs d014a9b1b1
Introduce temporary workaround for circleci failures.
The --no-utf8 option accidentally got reversed in django-extensions
1.8.1. The issue got reported upstream in
https://github.com/django-extensions/django-extensions/issues/1213 and
there's a fix in
https://github.com/django-extensions/django-extensions/pull/1214 which,
once this lands means we can safely remove our "--no-utf8" in the
makefile.

Fixes #8593
2018-06-21 09:32:11 +02:00
Christopher Grebs d6b44de95e
Use manage.py migrate instead of deprecated syncdb.
This is just a small code-cleanup update towards Django 1.11 where
`syncdb` doesn't exist anymore.

Fixes #8530
2018-06-16 22:05:19 -07:00