* More efficient utilization of github action cache
* Set cache from/to only in CI
* Update Dockerfile
Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
---------
Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
* Use consistent and robust mapping of UID between host and container:
- collapse UID/GID to one value on host and container
- rename host value to HOST_UID for clarity and debugability
- rename fix_olympia_user to entrypoint and set as docker compose entrypoint
- uwsgi config to use runtime value of olympia UID/GID
* Docker compose fail after 5 attempts to start
* TMP: update outdated comment
* TMP: comment the entrypoint
* Add checks to verify built docker image
TMP: rename dependencies.txt
* Remove unused and uninstalled package
* More efficient multi stage docker buildTMP: reorganize docker more efficiently
* Organize dockerfile
- split base and olympia stages
- use heredocs for multiline RUN steps
- move file copy to source layer
- merge run steps where possible
* TMP: fix invalid bash syntax in dockerfile
* TMP: add comments to dockerfile
* local docker build command
* Move copy . to after installing packages (prevents always re-installing)
* Enable buildkit cache mount to cache pip/npm dependencies
- cache npm/pip dependencies across builds
- add clear logging to npm install for cache hit/miss
- move copy npm files to update_assets
* Split build into stages
- splitting to stages offers better caching of layers and more efficient use of disk/time
- The initial gains will be with better caching of locale compilation, but will expand as we can move more logic from the final stage
TODO: split up apt depedencies to specific stages, move update_assets to pre-final stage to prevent re-running on every build with a * file change.
* Document the docker file and build process
* update docs
* Revert "Build local mysql apt mirror to work around upstream repo breakage (#21018)"
This reverts commit 550a4a4754.
* Update MySQL GPG key
* Stop forcing buster
We're using MySQL 8.0 from MySQL, not MariaDB, so client libs should
follow for consistency - especially as MariaDB doesn't support the
default auth method MySQL 8 uses.
Also contains a slight refactor of the Dockerfile-s to have fewer
steps and be more consistent with each other.
* Remove obsolete code to build a webext dictionary from a legacy one
* Simplify SafeZip initialization code
We don't need to store is_valid, since an exception will be raised
if something is invalid.
* Cleanup temporary directory for invalid zips in extract_extension_to_dest()
* Switch to Python 3.8
- Rename BadZipfile to BadZipFile as the former is deprecated
- Support BadZipFile being raised earlier (when instantiating
the ZipFile) in validation process
* Also remove test for build_webext_dictionary_from_legacy
* Remove a stray `is_valid` on a `SafeZip` instance
* Fix test now that we consider badzipfile errors fatal
* Move to a rolling debian-testing release for libmagic and libgit2
Fixes#13487
Unblocks #13391
And at least refs #12924, #12357 (these need further testing)
* Roll back temporary change
This updates various things…
* make use o `--user` argument of pip and `$PYTHONUSERBASE` to install python libraries in the correct places where the `olympia` user has permissions
* Add a new `rootshell` command
Fixes#12366
* Correct if statement
* Use separate tag for easier testing
* Try running directly with user olympia
* Small cleanup
* Add docs
* Fix comment
* Add 'rootshell' command
* Build circleci for local dev
* Fix pip execution by using 'python -m' to fetch the correct pip
* Pick up upstream user/group for builds and implement PYTHONUSERBASE and PIP_USER
* Correctly pass USER_ID and GROUP_ID to Dockerfile when building
* Small cleanups
* Build our test branch
* Fall back to manual uploads for testing
* Cleanup
* Revert back to :latest tag
* move all the make commands that operate on a single container to Makefile-docker
* add default user to Dockerfile
* revert the PYTHON_COMMAND var change
Kevin Meinhardt
Mathieu Pillard
Andrew Williamson
Lucky Sivagurunathan
William Durand
championshuttler
Christopher Grebs
Christopher Grebs