* More efficient utilization of github action cache
* Set cache from/to only in CI
* Update Dockerfile
Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
---------
Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
* Fix Docker build:
- get correct Image ID from the build.
- set docker tag correctly supporting tag and digest
- add concurrency limit to github action ci jobs
* TMP: remove easy way to get build digest as it is not really intended for local dev
* Update .circleci/config.yml
Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
* TMP: recomment build
---------
Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
* TMP: refactor buildx bake definition and test
* TMP: better initialization and organization of make file env vars (docker relevant +)
* TMP: cleaner code.
* TMP: remove build
* Infer superuser credentials from git config
* Load service data via portable and persistent data volumes
Load storage via persistent data volume (host bound)
Make up/down commands to easily recreate local environments
* TMP: add redis volume too
* TMP: better control over volume state
* TMP: remove dead code
* TMP: more dead code gone
* TMP: add wait for mysql before restoring snapshots
* Use consistent and robust mapping of UID between host and container:
- collapse UID/GID to one value on host and container
- rename host value to HOST_UID for clarity and debugability
- rename fix_olympia_user to entrypoint and set as docker compose entrypoint
- uwsgi config to use runtime value of olympia UID/GID
* Docker compose fail after 5 attempts to start
* TMP: update outdated comment
* TMP: comment the entrypoint
* Run CI/local envs with ES 7 and update client libs
Also drive-by update CI to have the right versions of MySQL/Redis
* Remove deprecated, no longer necessary send_get_body_as parameter
* Fix deprecation warnings
* Update tests
* Unused import
* More deprecation fixes
* Import addons-nginx into addons-server
* Serve xpis through X-Accel-Redirect instead of redirecting to dedicated domain
The so called "CDN URLs" for files are less useful now that even AMO main domain
is behind a CDN, and we want to move away from it because internally it exposes
paths on EFS that nginx is serving directly, making directory structure changes
difficult.
This commit focuses on the download_file/download_latest endpoints
addons-server exposes, and leaves the old dedicated CDN domain intact for the
moment - it is still used by the update service but will go away in a future
commit.
* Few fixes and test updates
* filename is an optional argument
* Test fixes
* Add assertions about caching
* Fix latest url
* Rename type_ to download_type
* reformat url definition
* Upgrade to Elasticsearch 6.x
Because ES 6.x removed mapping types, move update_count and download_count in
separate ES indexes. This requires a full reindex, including stats, but we need
to create an entirely new cluster anyway.
ES 5.x compatibility is kept for now.
This updates various things…
* make use o `--user` argument of pip and `$PYTHONUSERBASE` to install python libraries in the correct places where the `olympia` user has permissions
* Add a new `rootshell` command
Fixes#12366
* Correct if statement
* Use separate tag for easier testing
* Try running directly with user olympia
* Small cleanup
* Add docs
* Fix comment
* Add 'rootshell' command
* Build circleci for local dev
* Fix pip execution by using 'python -m' to fetch the correct pip
* Pick up upstream user/group for builds and implement PYTHONUSERBASE and PIP_USER
* Correctly pass USER_ID and GROUP_ID to Dockerfile when building
* Small cleanups
* Build our test branch
* Fall back to manual uploads for testing
* Cleanup
* Revert back to :latest tag
* Move CI envvar definition to selenium override config.
This should fix autoreload.
* Actually use to be more explicit
* Disable integration tests
* Try out different approach
* Add docs, move back the variable definition to docker-compose.yml
* Update comment
* Sign add-ons with recommendation signer based on DiscoveryItem.
This also prepares for the possibility that we may have to use different
credentials for the recommendations signer. In case we don't, ops only
has to set the creds twice which seems fair.
Fixes#11062
* Unify travis and docker-compose configs, test separate signer hawk creds
* Add end-to-end test for reviewer tools signing
* Enable autograph for reviewers-and-zadmin
* Remove amo-validator, remove more support for uploading legacy-addons.
This is dropping most of our compatibility checks for legacy add-ons and
completely blocks them from being uploaded.
We workaround this block for mozilla-signed add-ons by issuing the same
error, just as a warning instead of an actual error.
This affected *loads* of our tests so the PR contains more test-changes
then I'd like.
In addition to that this PR adds file extension validation for uploads,
we only did that in our JavaScript and never in our code. So from now
on, we have a hard limit to only allow whatever is in
`VALID_EXTENSIONS`.
Further...
* drop amo-validator and related dependencies
* remove `dump_apps` management command
* removed unused fixture files from the repo
Fixes#10502
Future improvement idea for our test-suite: Unify add-on file fixtures
under one single folder to avoid any duplication.
There is most certainly some more improvements that could be made to
code and test structure but I went for the least changes, it's way too
huge already :-/
* Fix syntax error in JavaScript, thus ui tests
* Switch back to passing 'channel' around
* Move valid addon file extensions to constants/base
* Fix validate() call
* Fix more tests
* Update src/olympia/devhub/tests/test_views_validation.py
Co-Authored-By: EnTeQuAk <cg@webshox.org>
* Remove old comment
* Issue description specific only to Thunderbird and Seamonkey if targeting thunderbird or seamonkey
This doesn't run services yet, and is just useful to run tests. Because
it shares the same image as the regular web container, an initial
make update_deps is necessary to get it working. After that, running
pytest as normal should function normally and be using Python 3.
Kevin Meinhardt
Andreas Wagner
Mathieu Pillard
William Durand
Andrew Williamson
mirefly
dbxnr
Christopher Grebs
Benjamin Forehand Jr
Mathieu Pillard