We're using MySQL 8.0 from MySQL, not MariaDB, so client libs should
follow for consistency - especially as MariaDB doesn't support the
default auth method MySQL 8 uses.
Also contains a slight refactor of the Dockerfile-s to have fewer
steps and be more consistent with each other.
* Remove obsolete code to build a webext dictionary from a legacy one
* Simplify SafeZip initialization code
We don't need to store is_valid, since an exception will be raised
if something is invalid.
* Cleanup temporary directory for invalid zips in extract_extension_to_dest()
* Switch to Python 3.8
- Rename BadZipfile to BadZipFile as the former is deprecated
- Support BadZipFile being raised earlier (when instantiating
the ZipFile) in validation process
* Also remove test for build_webext_dictionary_from_legacy
* Remove a stray `is_valid` on a `SafeZip` instance
* Fix test now that we consider badzipfile errors fatal
* Move to a rolling debian-testing release for libmagic and libgit2
Fixes#13487
Unblocks #13391
And at least refs #12924, #12357 (these need further testing)
* Roll back temporary change
This updates various things…
* make use o `--user` argument of pip and `$PYTHONUSERBASE` to install python libraries in the correct places where the `olympia` user has permissions
* Add a new `rootshell` command
Fixes#12366
* Correct if statement
* Use separate tag for easier testing
* Try running directly with user olympia
* Small cleanup
* Add docs
* Fix comment
* Add 'rootshell' command
* Build circleci for local dev
* Fix pip execution by using 'python -m' to fetch the correct pip
* Pick up upstream user/group for builds and implement PYTHONUSERBASE and PIP_USER
* Correctly pass USER_ID and GROUP_ID to Dockerfile when building
* Small cleanups
* Build our test branch
* Fall back to manual uploads for testing
* Cleanup
* Revert back to :latest tag
* move all the make commands that operate on a single container to Makefile-docker
* add default user to Dockerfile
* revert the PYTHON_COMMAND var change
* Update 'file' version to fix json<->html detection for reviewer api.
Fixes#11534
* Add fixture file
* Rollback to xenial, install libmagic-dev manually
* Add libmagic1 dependency too
* test amd64
* Add required libmagic-mgc dependency
* Add explaining comment
* Try out using ubuntu disco sources for libmagic installation
* Force installation from the disco repo.
This implements more advanced file-type detection based on `python-magic` thus we don't rely on the files extensions anymore but actually verify their content (header) before serving it to the client and reviewer.
Fixes#10685 (and actually rewrites it's meaning)
* Remove amo-validator, remove more support for uploading legacy-addons.
This is dropping most of our compatibility checks for legacy add-ons and
completely blocks them from being uploaded.
We workaround this block for mozilla-signed add-ons by issuing the same
error, just as a warning instead of an actual error.
This affected *loads* of our tests so the PR contains more test-changes
then I'd like.
In addition to that this PR adds file extension validation for uploads,
we only did that in our JavaScript and never in our code. So from now
on, we have a hard limit to only allow whatever is in
`VALID_EXTENSIONS`.
Further...
* drop amo-validator and related dependencies
* remove `dump_apps` management command
* removed unused fixture files from the repo
Fixes#10502
Future improvement idea for our test-suite: Unify add-on file fixtures
under one single folder to avoid any duplication.
There is most certainly some more improvements that could be made to
code and test structure but I went for the least changes, it's way too
huge already :-/
* Fix syntax error in JavaScript, thus ui tests
* Switch back to passing 'channel' around
* Move valid addon file extensions to constants/base
* Fix validate() call
* Fix more tests
* Update src/olympia/devhub/tests/test_views_validation.py
Co-Authored-By: EnTeQuAk <cg@webshox.org>
* Remove old comment
* Issue description specific only to Thunderbird and Seamonkey if targeting thunderbird or seamonkey
This also adds support for specifiying which version to use when installing
packages, though at the moment it's not possible to switch python versions
without reinstalling everything, because both versions share /usr/local/bin
Our docker builds were broken because of some inter-dependency weirdness
going on when you're installing npm and nodejs in one apt-get call. To
be honest I was too lazy to look up the actual root cause given the fix
is quite simple and npm is part of nodejs anyway.
This PR updates the Python version to 2.7.15 along the way.
This ports all amo-validator related code to run via subprocess. This
makes us less blocked by it not running on Python 3, there may still be
some installation / execution related bits to actually run it via python
2 instead of python 3 but that's something for later.
* Cleans up how and where we're setting _BIN settings for all the
binaries we're calling
* Backports `unicodehelper` from validator
* Removes `requeue_uploads` since it's been broken and unused for at
least 3-4 years
* Remove Standalone Add-on Compatibility Checker (/developers/addon/check-compatibility/)
* Remove compat-check related bits from standalone add-on upload
* Remove more bulk compat report stuff that isn't used anymore
* Removed zadmin application_versions.json view, unused
Fixes#9670
"Fixes" #8857
* Add debian stretch backports repo as discussed with @autrilla
* Add libgit2-dev dependency that's needed by pygit2 (via debian stretch
to get the proper version that's required by up-to-date pygit2)
* Add pygit 0.27.1
Fixes#9496
Icons and previews for both themes and non-themes should go through pngcrush.
For the existing content, the command `crush_images_for_top_addons` focuses
on the content displayed on landing pages only and should only be run once.
* Make also use of setting PYTHONDONTWRITEBYTECODE inside the docker
image to reduce it's size considerably
* Remove some outdated code-parts
* Rename `olympia` inside our worker image to `olympia-worker`
* remove all build and cache folders after every `make update_deps`
(which might help avoid any more package update/install ambiquities)
Fixes#7518
Revert "docker: Fix Debian build issue for python package"
This reverts commit 2239e8483e.
Revert "docker: Include uwsgi plugin"
This reverts commit 807420bf16.
Revert "docker: Include uwsgi"
This reverts commit ceb36b5890.
Revert "docker: Update deploy to use Docker Python image"
This reverts commit 916547d2b5.
Revert "docker: Remove ‘centos’ from docker-indicating file"
This reverts commit d34b92e0e8.
Revert "docker: Update build to use Docker Python image"
This reverts commit b08b51375d.
This provides an updated Python (2.7.14) as part of
https://bugzilla.mozilla.org/show_bug.cgi?id=1390789
The Docker images are not based on Centos so package
names needed to be updated. Comparable packages were
added, but may not contain the same versions or even
components.
* Pin all our dependencies, including pip, six, ipython and ipdb.
This removes all dynamically installed python packages from the
installation pipeline and should result in a cleaner system install.
Fixes#6001
* Cleanup docs, prod and merge a few dependencies with system.
* Move more dependencies from prod to system :-/
* Finally fix docs task
* Revert "Remove old node 4 repo"
This reverts commit 7d221f0814.
* Add back nodesource v6 repo
We need to use nodesource v6 repo for latest security fixes. EPEL updates
are often delayed.
* Compile locales before we run tests, add gettext dependency
* Make compile-mo fail on linting errors.
* Fix tox compilation
* More tox fixes
* Fix path
* remove usedevelop for now
* re-add plural forms
* Install six before we upgrade setuptools.
See pypa/setuptools#964 for a few more details. Currently quite a few
travis and circleci jobs are failing because of that.
* Explicitly upgrade pip and setuptools in 'make update_deps' so for travis too.
* Potentially fix circleci
* Fix setuptools and potentially docs environments
* More explicitly install six and setuptools
* Add more requirements for docs
* Add pyparsing dependency
Mathieu Pillard
Lucky Sivagurunathan
William Durand
championshuttler
Christopher Grebs
Christopher Grebs
Andrew Williamson
Benjamin Forehand Jr
Mathieu Pillard
pyup.io bot
bqbn
Philip Cristiano
Andreas Wagner