Граф коммитов

105 Коммитов

Автор SHA1 Сообщение Дата
Kevin Meinhardt d7b21e18e6
Remove supervisor and enable more efficient container startup (#22592)
* TMP: add http server to expose version endpoint

* Use data volume for mounting repository files

* Use watchmedo directly from docker to run celery

* TMP: fix test

* TMP: remove data_olympia volume on ci yml

* TMP: add settings djanmgo modeul to celery worker.

* TMP: remove references to ssupervisor

* Update docs/topics/development/building_and_running_services.md

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>

---------

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
2024-08-27 12:42:10 +02:00
Kevin Meinhardt e4ceb64b9e
Remove the chown command from entrypoint. (#22584)
* Remove the chown command from entrypoint.

* TMP: remove redundant timeout on the health check for worker/web containers
2024-08-23 12:00:32 +02:00
Kevin Meinhardt 84d514e744
Remove version.json file from host environment (#22571)
* Remove version.json file from host environment

* tmp: fix

* TMP: fix tests

* TMP: remove unecessary test

* TMP: fix test for get_version_json

* TMP: fixy
2024-08-21 10:33:31 +02:00
William Durand 56c7318a9e
Fix storage volume config for the nginx service (#22409) 2024-07-02 12:39:01 +02:00
Kevin Meinhardt aa31195786
Remove egg-info without breaking locale extraction (#22384)
* Remove egg-info without breaking locale extraction

* Merge setup to pyproject
2024-06-21 15:19:29 +02:00
Kevin Meinhardt 59156f8674
Stability fixes for docker compose lifecycle (#22378)
* Stability fixes for docker compose lifecycle

* Update docs/topics/development/setup_and_configuration.md

Co-authored-by: Andrew Williamson <awilliamson@mozilla.com>

* TMP: response to comments

* Update docs/topics/development/setup_and_configuration.md

Co-authored-by: William Durand <will+git@drnd.me>

---------

Co-authored-by: Andrew Williamson <awilliamson@mozilla.com>
Co-authored-by: William Durand <will+git@drnd.me>
2024-06-21 14:22:42 +02:00
Kevin Meinhardt be592a247a
Ditch the /deps volume (#22332)
* Ditch the /deps volume

* TMP: remove update deps at the beggingin

* TMP: add comment
2024-06-10 13:18:25 +02:00
Andreas Wagner 2bbb6ff52c
Hard-code super user name and email (#22296) 2024-06-05 18:20:59 +02:00
Kevin Meinhardt 299dc4922b
More efficient utilization of github action cache (#22314)
* More efficient utilization of github action cache

* Set cache from/to only in CI

* Update Dockerfile

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>

---------

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
2024-06-05 14:48:42 +02:00
Kevin Meinhardt ddade2a62b
Fix Docker build and add concurrency limit (#22273)
* Fix Docker build:
- get correct Image ID from the build.
- set docker tag correctly supporting tag and digest
- add concurrency limit to github action ci jobs

* TMP: remove easy way to get build digest as it is not really intended for local dev

* Update .circleci/config.yml

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>

* TMP: recomment build

---------

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
2024-05-24 10:09:09 +02:00
Kevin Meinhardt d3b3a50ac5
Docker digest (#22260)
* Refactor unified setup script to manage .env and version.json files in readable python

* Support using digest to run docker image

* Update scripts/setup.py

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>

* TMP: build image

* TMP: better logs

---------

Co-authored-by: Mathieu Pillard <diox@users.noreply.github.com>
2024-05-21 13:02:19 +02:00
Kevin Meinhardt c4f4ba6487
"Fix makefile env variables definition" (#22246)
* TMP: refactor buildx bake definition and test

* TMP: better initialization and organization of make file env vars (docker relevant +)

* TMP: cleaner code.

* TMP: remove build
2024-05-16 14:49:37 +02:00
Kevin Meinhardt bc186195bc
Load service data via portable and persistent data volumes (#22235)
* Infer superuser credentials from git config

* Load service data via portable and persistent data volumes

Load storage via persistent data volume (host bound)

Make up/down commands to easily recreate local environments

* TMP: add redis volume too

* TMP: better control over volume state

* TMP: remove dead code

* TMP: more dead code gone

* TMP: add wait for mysql before restoring snapshots
2024-05-14 16:17:35 +02:00
Mathieu Pillard 4fe0fc5d5e
Bump rabbitmq to 3.12 on local envs. (#22244)
Note: you _need_ to run `docker compose down rabbitmq` at some point to start fresh:
https://www.rabbitmq.com/blog/2022/07/20/required-feature-flags-in-rabbitmq-3.11
2024-05-14 13:18:08 +02:00
Kevin Meinhardt 6212cc2c9e
Use environment variables to configure docker compose variable properties and build with bake (#22225)
* Remove redundant docker-cache directory

* Use environment variables to configure docker compose variable properties
2024-05-10 09:44:54 +02:00
Kevin Meinhardt 42513e191e
Use consistent and robust mapping of UID between host and container: (#22206)
* Use consistent and robust mapping of UID between host and container:
- collapse UID/GID to one value on host and container
- rename host value to HOST_UID for clarity and debugability
- rename fix_olympia_user to entrypoint and set as docker compose entrypoint
- uwsgi config to use runtime value of olympia UID/GID

* Docker compose fail after 5 attempts to start

* TMP: update outdated comment

* TMP: comment the entrypoint
2024-05-06 14:07:55 +02:00
Mathieu Pillard e9c1554d8b
Set ipdb as the breakpoint callback (#22135)
Allows inserting breakpoint() inside code to debug with ipdb
2024-04-19 12:15:22 +02:00
Kevin Meinhardt 6eb0858bad
chore(docker): Add port mapping for MySQL container (#21560) 2023-12-06 14:09:46 +01:00
Mathieu Pillard 04c983890c
Remove code and dependencies dealing with creating actual FxA users in tests (#20811)
* Remove code and dependencies dealing with creating actual FxA users in tests
2023-06-12 11:48:33 +02:00
Mathieu Pillard 3ef358c672
Switch to docker compose v2 (plugin) instead of v1 (docker-compose pip package) (#20755) 2023-05-26 14:05:21 +02:00
Mathieu Pillard 7730f6f3bb
Drop Django 3.2 tests (#20693) 2023-05-15 14:51:24 +02:00
Mathieu Pillard 5c0e7a3c88
Merge deployment and development Dockerfiles (#20480)
* Merge deployment and development Dockerfiles
* Stop using root to install dependencies
* Run make commands from os with --user olympia
2023-03-23 15:46:56 +01:00
Andreas Wagner df93e522e1
Specify platform docker images where necessary (#20366) 2023-02-20 19:44:55 +01:00
Mathieu Pillard 97ba20c9a2
Run CI/local envs with ES 7 and update client libs (#19441)
* Run CI/local envs with ES 7 and update client libs

Also drive-by update CI to have the right versions of MySQL/Redis

* Remove deprecated, no longer necessary send_get_body_as parameter

* Fix deprecation warnings

* Update tests

* Unused import

* More deprecation fixes
2022-07-11 11:35:18 +02:00
Mathieu Pillard 4b97ad368c
Elasticsearch 7.x compatibility & cleanups (#19126)
* Elasticsearch 7.x compatibility & cleanups

* Fix double import

* Bump versions

* Add position back
2022-04-21 15:28:32 +02:00
William Durand c2c1be15ae
Update redis docker image (#19001)
This is needed to run addons-server with Apple silicons.
2022-03-22 14:09:57 +01:00
Mathieu Pillard 0047612d29
Serve xpis through addons server (#18978)
* Import addons-nginx into addons-server

* Serve xpis through X-Accel-Redirect instead of redirecting to dedicated domain

The so called "CDN URLs" for files are less useful now that even AMO main domain
is behind a CDN, and we want to move away from it because internally it exposes
paths on EFS that nginx is serving directly, making directory structure changes
difficult.

This commit focuses on the download_file/download_latest endpoints
addons-server exposes, and leaves the old dedicated CDN domain intact for the
moment - it is still used by the update service but will go away in a future
commit.

* Few fixes and test updates

* filename is an optional argument

* Test fixes

* Add assertions about caching

* Fix latest url

* Rename type_ to download_type

* reformat url definition
2022-03-16 15:19:47 +01:00
Andrew Williamson 4b4b8b2407
default local dev to django3.2; deploy to addons-dev with 3.2 (#17452) 2021-07-08 10:31:48 +01:00
Andrew Williamson c535369fc6
serve sitemap files via HttpResponseXSendFile instead (#17114) 2021-05-18 09:23:29 +01:00
Mathieu Pillard 12f5e3d36a
Upgrade rabbitmq for local development (#16739) 2021-03-15 12:07:12 +01:00
William Durand 1b72db9fa6
Fix docker-compose config for addons-frontend (#16281) 2021-01-08 15:18:12 +01:00
William Durand 2a3ce33286
Migrate to Circle CI (#16189) 2020-12-18 12:13:48 +01:00
Mathieu Pillard 7054d9f9c8
Upgrade Travis and local envs to MySQL 8.0.x, fixing broken tests (#15911)
* Upgrade Travis and local envs to MySQL 8.0.x, fixing broken tests

* Avoid hardcoding the filename twice when downloading mysql/elasticsearch
2020-11-05 13:03:59 +01:00
Mathieu Pillard ec0d8f9293
Upgrade to Elasticsearch 6.x (#14206)
* Upgrade to Elasticsearch 6.x

Because ES 6.x removed mapping types, move update_count and download_count in
separate ES indexes. This requires a full reindex, including stats, but we need
to create an entirely new cluster anyway.

ES 5.x compatibility is kept for now.
2020-05-28 20:47:07 +02:00
mirefly ac89818085 Fix the startup order of nginx and frontend containers 2020-03-28 10:42:41 -06:00
mirefly 0ccf9c98fa
Hide disabled files when delete an add-on (#13531)
Hide disabled files when delete addon
2020-02-21 14:06:27 +01:00
dbxnr 0c845636a9 Remove code for ui tests 2020-01-13 23:15:17 +00:00
Christopher Grebs b9ac5f4413
Run dockerfile commands with correct user. (#12579)
This updates various things…

* make use o `--user` argument of pip and `$PYTHONUSERBASE` to install python libraries in the correct places where the `olympia` user has permissions
* Add a new `rootshell` command

Fixes #12366

* Correct if statement

* Use separate tag for easier testing

* Try running directly with user olympia

* Small cleanup

* Add docs

* Fix comment

* Add 'rootshell' command

* Build circleci for local dev

* Fix pip execution by using 'python -m' to fetch the correct pip

* Pick up upstream user/group for builds and implement PYTHONUSERBASE and PIP_USER

* Correctly pass USER_ID and GROUP_ID to Dockerfile when building

* Small cleanups

* Build our test branch

* Fall back to manual uploads for testing

* Cleanup

* Revert back to :latest tag
2019-10-22 11:24:05 +02:00
Christopher Grebs dcc103af6e
Fix uwsgi autoreload (#12316)
* Move CI envvar definition to selenium override config.

This should fix autoreload.

* Actually use  to be more explicit

* Disable integration tests

* Try out different approach

* Add docs, move back the variable definition to docker-compose.yml

* Update comment
2019-09-12 14:05:33 +02:00
Benjamin Forehand Jr 0e3725f031
Add additional docker compose 'ups' (#12058)
* Add additional docker compose 'ups'

* Trying to disable autoreload.

* Try disabling autoreload via env variable.

* Added docker compose changes.

* Change image name.
2019-08-08 11:57:45 -05:00
Christopher Grebs 5423c6476c
Sign add-ons with recommendation signer based on DiscoveryItem. (#11627)
* Sign add-ons with recommendation signer based on DiscoveryItem.

This also prepares for the possibility that we may have to use different
credentials for the recommendations signer. In case we don't, ops only
has to set the creds twice which seems fair.

Fixes #11062

* Unify travis and docker-compose configs, test separate signer hawk creds

* Add end-to-end test for reviewer tools signing

* Enable autograph for reviewers-and-zadmin
2019-06-12 15:01:36 +02:00
Christopher Grebs a4a9e699a2
Sign recommended add-ons with specific autograph signer (#11587)
Fixes #11062

* Upgrade autograph to 3.3.2
* Configure a new signer for recommended add-ons
* Sign recommended add-ons
* Adapt to recent specs
2019-06-06 12:37:30 +02:00
Christopher Grebs 59bb810842
Update to autograph 3.2.0
Fixes #11414
2019-05-14 09:28:49 +02:00
Andrew Williamson 6aebb99b1f
remove support for django1.11 and python2 (#11195) 2019-04-15 15:17:10 +01:00
Benjamin Forehand Jr 901ad6a5f8
Update docker-compose file to use latest addons-frontend image. Added new homepage test. (#11050) 2019-04-01 10:12:28 -04:00
Andrew Williamson ec4ac13ad7
deploy django22 on -dev & defaul on local dev (#11025) 2019-03-27 11:37:52 +00:00
Mathieu Pillard 19e782a96d Stop building Python 2 images 2019-03-25 11:55:09 +01:00
Benjamin Forehand Jr 2d99d6f3da
Fix ui tests for new nginx config (#10888)
* Fixes for Ui-tests using new nginx config.
2019-03-11 19:20:06 -04:00
Christopher Grebs 7b34c1b7d1
Fix addons-frontend running as default frontend. (#10692)
The frontend runs quite nicely with the changes from addons-nginx container but the initial page load runs into some wacky connection errors.

https://www.irccloud.com/pastebin/gTMteTUf/

This change fixes this.

Fixes https://github.com/mozilla/addons-server/issues/10855
2019-03-05 13:23:56 +01:00
Andrew Williamson 6fdf5bb9bf switch local dev to python3 2019-02-08 21:54:35 +08:00