This updates the `page` URL helper to wrap the view using the `require_safe`
decorator, and also limits the HTTP methods on the `L10nTemplateView` to `GET`
or `HEAD`. All other views explicitly get decorated with `@require_safe`.
* Switch away from pip-compile-multi in attempt to resurrect Dependabot support
Calling just pip-compile from pip-tools on each input file generates files
with no inheritance/cross-file dependencies, which hopefully will help
Dependabot process them properly
* Update ADRs in light of tooling change
* Delete redundant pip-compile-multi header file
* Update documentation
* Switch to NOT inheriting from prod reqs at all
...and plug the gap with a make command 👍
* Use requirements/prod.txt to constrain the duplicate deps in dev.txt
* Update make command for installing local deps
* Remove unnecessary -U and --no-cache-dir options
* Remove docs dependencies installation from makefile shortcut, because technically it may clash with dev.txt and we don't want developer confusion. The documentation still shows how to install the deps when required
Django 3.0 fixes:
* Remove six
* Convert LANGUAGES to a list
* Update redirects
* Django's `re_path` no longer sends they key for matched group name if the value is `None`. This caused some errors as we were depending on the old behavior.
* Add `private` for cache-control headers
* Remove context argument to `from_db_value`
* Re-arrange check due to `re_path` change.
* Tweak contentful query
* The change to using bitwise operations on the `Q` object here are due to this adding an empty `(AND: )` to the query. While this doesn't change anything about the underlying query, I made this change to clean up the query and match Django documented usage.
* Fix failing contentful test
* `is_ajax` is deprecated
* The `HttpRequest.is_ajax()` method is deprecated as it relied on a jQuery-specific way of signifying AJAX calls, while current usage tends to use the JavaScript Fetch API. Depending on your use case, you can either write your own AJAX detection method, or use the new HttpRequest.accepts() method if your code depends on the client Accept HTTP header.
* Update docs to follow Django upgrades and syntax
Django 3.2 fixes:
* Set value for the new `DEFAULT_AUTO_FIELD` setting
* Updates for new app config discovery
* Update to new response headers interface.
* Remove regex from URL `path`s
* Change quoting to match black in docs
* Move Bedrock to pip-compile-multi for easier Python dependency management
This changeset adds tooling to ease dependency management and also rationalises
our requirements files.
Before, we were just using hashin to manually hash pinned deps straight into a requirements file
Now we're using pip-compile-multi, which sits on top of pip-tools to do this.
We now get:
* Simpler syntax for adding and pinning dependencies via *.in files
* Automatic hash generation when the *.txt requirements files are produced
The dependency compilation/update tooling runs in a Docker container, so will be compatible
with the deployed service's containers.
We're also rationalising the existing split of dependency files:
* base -> being retired and used as the basis for prod requirements
* migration -> being retired and the two deps still useful to us (for moz-l10n-lint)
added to dev deps
* dev -> now extends from the prod requirements. We're not too concerned about image
size for dev and test builds
* prod -> still exists, but includes the base deps
* docs -> still exists as a standalone file, but also follows the "*.in"-file pattern
* Regenerate dependency files using pip-compile-multi
Note that to avoid clashes, the following balances were made:
* Keep meinheld at the lower version used in prod.txt, not the dev.txt one -- for now at least
* Downgrade Markdown to 3.3 to avoid a clash over importlib-metadata version
* Remove importlib-metadata==4.10.1 altogether as a hard pin and let pip-compile-multi work out the best fit
* Update docs to reference pip-compile-multi, replacing now-redundant notes on hashin
* Update Dockerfile to copy over and use freshly recut dependency files
* Attempting to tune deps to allow local builds to work, not just Docker ones
* Update pip-compile-multi config to inject a custom header that explains how to rebuild reqs
* Update Bedrock to use Python 3.9
* Update base images
* Update CI
* Update dependencies to make install run -- this involved manually using hashin to upgrade two hashed deps (greenlet and meinheld) then re-running make compile-requirements to update the top-level hash in prod.txt. It's a bit of a chicken-and-egg situation when the deps are built/re-locked in a container but you can't build the container itself unless the deps are viable, but it worked
* Upgrade everett in order to remove configobj, which is redundant and causing local install issues on MacOS M1
* Drop backports.cached-property and typed-ast from dev reqs because we don't need them on 3.9
* Update docs with local-installation guidance for pyenv and pyenv-virtualenv
* Remove 'upgrade requirements' option
Given that the --upgrade flag is implicitly / by-default true with
pip-compile-multi anyway, plus the fact we're hard-pininng everything,
there's no point having an explicit 'upgrade' path - so let's remove it
* Docs tweak to suggest simpler virtualenv name
* Update help option in Makefile
* Pin version of pip in the compile-requirements script
When unpinned, the build suddenly broke, so we're keeping it under strict
limits for now
* Upgrade Django to 2.2.27
* Upgrade newrelic package to latest, incl py3.9 support
* Switch to Python 3.9 Debian bullseye image, from buster
* Rationalise dependency input files to remove over-pinned subdeps
When we moved from hand-managed requirements.txt files, we were taking on files
that had literally every dependency and thier sub-deps in them. We don't want
the input (*.in) files to reference those subdeps, so this changeset tries to
thin things out and remove them
Note that the diff shows this was successful - there are very few changes to
the dependencies being mentioned in the output *.txt files, and the ones that
are there are all deliberate changes (eg removing 'pbr')
* Drop unused tenacity dep, bump APScheduler and link to a Python 3.9-patched version of mdx-outline
* Add --require-hashes option to pip usage in Dockerfile
It's implicitly set because the reqs files feature --hash=XXX
but better to be explicit
* Drop what appear to be redundant top-level dev dependencies
regex, pep8 and wcwidth appear to be subdeps that don't need pinning.
The others appear to not be in used, based on a search of the codebase.
Tests till pass
* Thin out some unnecessary top-level deps in prod.in
- funcsigs - old backport, redundant
- lxml - over-pinned subdep of BeautifulSoup?
- typing_extensions - over-pinned subdep
- zipp - over-pinned subdep
* Cap pip version to 21 for pip-compile-multi for now
* Reinstate lxml as a first-class dep: BeautifulSoup needs it as a user-specified parser
* Fix typo in pip-compile-multi header
* Hard-pin latest working combo of pip + pip-tools in compile-requirements.sh
* Update docs explaining why we're using 3.9.10 locally
This changeset makes Bedrock tests use coverage.py once more, but this time via the pytest-cov plugin
The configuration set up ignores a (seemingly) sensible set of files and folders, while also outputting HTML and XML reports, for human and [future] machine enjoyment.
Key changes:
* Config expanded for coverage.py
* Main test-running script updated to invoke pytest with the appropriate coverage-logging options
* Docs updated with a small note.
* 10614: Add pre-commit hook for including the MPLv2
Adds to Python, JS, SCSS, Jinja HTML, Fluent templates and shell scripts
Note that the order of application of the hooks is important - we want to add a missing license before we check the formatting of files
* 10614: Update MPLv2 comments on all templates to match standard format produced by pre-commit hook
* Updates existing MPLv2 text to use a https URL
* Amend a handful of Fluent templates that used a token instead of the string "Mozilla" - this standardised things; translation was not used or needed
* Add missing MPLv2 where needed
* Update three tests that regressed with these changes, above
* 10614: Update pre-commit config to not add MPL to JS libraries; Remove MPL from the four files which should not have had it
Rob Hudson
Alex Gibson
Stephanie Hobson
Nathan Barrett
Steve Jalim
Reem H
maureenlholland
Paul McLanahan
maureenlholland
Ayushsunny