addons-server/Makefile-docker

181 строка
6.5 KiB
Plaintext
Исходник Обычный вид История

# Determine the python version (2 or 3) so we can load the correct requirements
python_version_major := $(shell python -c 'import sys; print(sys.version_info.major)')
# FXA username and password for ui-tests
export FXA_EMAIL=uitest-$(shell uuid)@restmail.net
export FXA_PASSWORD=uitester
NUM_ADDONS=10
NUM_THEMES=$(NUM_ADDONS)
NPM_ARGS :=
ifneq ($(NPM_CONFIG_PREFIX),)
NPM_ARGS := --prefix $(NPM_CONFIG_PREFIX)
endif
NODE_MODULES := $(NPM_CONFIG_PREFIX)node_modules/
2017-06-21 18:06:31 +03:00
STATIC_CSS := static/css/node_lib/
STATIC_JS := static/js/node_lib/
STATIC_JQUERY_UI := static/js/node_lib/ui/
2017-06-21 18:06:31 +03:00
NODE_LIBS_CSS := \
@claviska/jquery-minicolors/jquery.minicolors.css \
@claviska/jquery-minicolors/jquery.minicolors.png \
NODE_LIBS_JS := \
less/dist/less.js \
jqmodal/jqModal.js \
jquery/dist/jquery.js \
jquery.browser/dist/jquery.browser.js \
jquery.cookie/jquery.cookie.js \
2017-06-21 18:06:31 +03:00
@claviska/jquery-minicolors/jquery.minicolors.js \
2017-06-19 11:10:05 +03:00
jquery-pjax/jquery.pjax.js \
2017-11-24 17:41:59 +03:00
jszip/dist/jszip.js \
raven-js/dist/raven.js \
timeago/jquery.timeago.js \
underscore/underscore.js \
NODE_LIBS_JQUERY_UI := \
jquery-ui/ui/version.js \
jquery-ui/ui/data.js \
jquery-ui/ui/disable-selection.js \
jquery-ui/ui/ie.js \
jquery-ui/ui/keycode.js \
jquery-ui/ui/escape-selector.js \
jquery-ui/ui/labels.js \
jquery-ui/ui/jquery-1-7.js \
jquery-ui/ui/plugin.js \
jquery-ui/ui/safe-active-element.js \
jquery-ui/ui/safe-blur.js \
jquery-ui/ui/scroll-parent.js \
jquery-ui/ui/focusable.js \
jquery-ui/ui/tabbable.js \
jquery-ui/ui/unique-id.js \
jquery-ui/ui/position.js \
jquery-ui/ui/widget.js \
jquery-ui/ui/widgets/menu.js \
jquery-ui/ui/widgets/mouse.js \
jquery-ui/ui/widgets/autocomplete.js \
jquery-ui/ui/widgets/datepicker.js \
jquery-ui/ui/widgets/sortable.js
help_redirect:
@$(MAKE) help --no-print-directory
help_submake:
@echo "Commands that are designed be run in the container:"
@echo " initialize to init the dependencies, the database, and assets"
@echo " update to update the dependencies, the database, and assets"
@echo " initialize_db to create a new database"
@echo " populate_data to populate a new database"
@echo " reindex to reindex everything in elasticsearch, for AMO"
@echo " update_deps to update the pythondependencies"
@echo " update_db to run the database migrations"
initialize_db:
rm -rf ./user-media/* ./tmp/*
python manage.py reset_db
Update to Django 1.11 (#8738) Port code and tests to Django 1.11. * Update requirements to Django 1.11.14, remove session_csrf * Use --run-syncdb for migration calls * Remove unused setup-docker script * Port code and tests to Django 1.11. * Remove django-session-csrf everywhere, it got replaced by django's own session csrf protection * Make sure we are calling `client.get()` with `follow=True` where necessary * Replace django.core.urlresolvers import with django.urls * Port custom `IconWidgetRenderer` to Django 1.11 as `IconTypeSelect` * Properly use `_queryset_class` for custom manager classes * Make use of `base_manager_name` on our models `Meta` instead of `use_for_related_objects` on the manager where appropriate. This is required to make translations, soft-delete and more work. See comment on Addons.Meta for more details. * Various fixes along the way to make tests work * Port form errors for locale editing forms to Django 1.11 and make use of form.add_error instead of our own custom process * Mark TestDetailPage.test_304 as xfail for now, somehow Etags seem to be broken for that view during tests but work fine in manual tests * Fix amo.celery, we cannot have any imports at the module-level that may import the ORM * Fix global_settings context processor for 500-server error cases * Fix `changed_data` handling in `AMOModelForm` * Remove `queryset_transform` and implement it directly in amo/models * Fix pagination, Django changed how `num_pages` work and how `count` can be overwritten * Add a new test that tests database transforms as `BaseQuerysetTestCase` * Make sure `legacy_redirect` only appends `?` if there are actual GET parameters * Re-implement `LicenseRadioChoiceInput` as `LicenseRadioSelect` which is now Django 1.11 compatible * Remove our custom `form_kwargs` uplift from Django 1.9 * Correct how we're using `Category.objects.create` and only pass `db_name` instead of `name`, previously this didn't error but now it (luckily!) does * Update `FileSelectWidget` to correctly work with Django 1.11, use custom rendering * Fix decorator ordering for `files.views:browse` * Update database settings to correctly use `TEST` object * Fix `select_related` call in `RatingViewSet` * Remove custom `DateTimeCast` implementation * Fix `addon_factory` calls in `reviewers/tests/test_views.py` to correctly set `is_webextension` * Port our `TranslationDescriptor` and `TranslatedField` to Django 1.11. Also, make sure form error handling works * Add a new test that checks that `default=None` is a valid setting for translation fields * Add a new test that checks that translations from related fields are correctly fetched * Remove possible ambiguity from translations `TestWidget` tests * Make sure that the correct `MEDIA_URL` is set in `TestDownloadsUnlistedVersions` to fix external cdn tests * Remove custom happyforms code. Django cleans text fields now by default * Move translations app to the top of INSTALLED_APPS, add a lengthy comment * Pass additional arguments to contribute_to_class, fix smaller nits * Set MEDIA_URL and STATIC_URL to an external value by default. That's how they're configured in dev, stage and prod anyway. Please note that we didn't replace all our custom form rendering with proper templated widgets yet. This will happening in some follow-up work. Fixes #5271, Fixes #1569
2018-07-11 07:56:54 +03:00
python manage.py migrate --noinput --run-syncdb
python manage.py loaddata initial.json
python manage.py import_prod_versions
schematic --fake src/olympia/migrations/
python manage.py createsuperuser
python manage.py loaddata zadmin/users
python manage.py update_permissions_from_mc
populate_data:
# reindex --wipe will force the ES mapping to be re-installed. Useful to
# make sure the mapping is correct before adding a bunch of add-ons.
python manage.py reindex --wipe --force --noinput
python manage.py generate_addons --app firefox $(NUM_ADDONS)
python manage.py generate_addons --app android $(NUM_ADDONS)
python manage.py generate_themes $(NUM_THEMES)
# These add-ons are specifically useful for the addons-frontend
# homepage. You may have to re-run this, in case the data there
# changes.
python manage.py generate_default_addons_for_frontend
# Now that addons have been generated, reindex.
python manage.py reindex --force --noinput
# Also update category counts (denormalized field)
python manage.py cron category_totals
python manage.py update_permissions_from_mc
cleanup_python_build_dir:
# Work arounds "Multiple .dist-info directories" issue.
rm -rf /deps/build/*
install_python_test_dependencies:
# Can't use --progress-bar=off for system packages as long as our docker image
# doesn't have pip 10 by default.
pip install --no-deps --exists-action=w -r requirements/system.txt
pip install --progress-bar=off --no-deps --exists-action=w -r requirements/prod_py${python_version_major}.txt
pip install --progress-bar=off --no-deps --exists-action=w -r requirements/tests.txt
install_python_dev_dependencies: install_python_test_dependencies
pip install --progress-bar=off --no-deps --exists-action=w -r requirements/flake8.txt
pip install --progress-bar=off --no-deps --exists-action=w -r requirements/dev.txt
pip install --progress-bar=off --no-deps --exists-action=w -r requirements/docs.txt
pip install -e .
install_node_dependencies: install_node_js copy_node_js
install_node_js:
npm install $(NPM_ARGS)
copy_node_js:
2017-06-21 18:06:31 +03:00
for dest in $(NODE_LIBS_CSS) ; do cp $(NODE_MODULES)$$dest $(STATIC_CSS) ; done
for dest in $(NODE_LIBS_JS) ; do cp $(NODE_MODULES)$$dest $(STATIC_JS) ; done
for dest in $(NODE_LIBS_JQUERY_UI) ; do cp $(NODE_MODULES)$$dest $(STATIC_JQUERY_UI) ; done
update_deps: cleanup_python_build_dir install_python_dev_dependencies install_node_dependencies
# Make sure we pull our latest mozilla-product-data.
# There were scenarios where this wouldn't happen automatically during our CircleCI ui-tests so we enforce it here as a workaround.
python manage.py update_product_details
update_db:
schematic src/olympia/migrations
update_assets:
python manage.py compress_assets
python manage.py collectstatic --noinput
update: update_deps update_db update_assets
python manage.py update_permissions_from_mc
reindex:
python manage.py reindex $(ARGS)
setup-ui-tests:
rm -rf ./user-media/* ./tmp/*
# Reset the database and fake database migrations
python manage.py reset_db --noinput
Update to Django 1.11 (#8738) Port code and tests to Django 1.11. * Update requirements to Django 1.11.14, remove session_csrf * Use --run-syncdb for migration calls * Remove unused setup-docker script * Port code and tests to Django 1.11. * Remove django-session-csrf everywhere, it got replaced by django's own session csrf protection * Make sure we are calling `client.get()` with `follow=True` where necessary * Replace django.core.urlresolvers import with django.urls * Port custom `IconWidgetRenderer` to Django 1.11 as `IconTypeSelect` * Properly use `_queryset_class` for custom manager classes * Make use of `base_manager_name` on our models `Meta` instead of `use_for_related_objects` on the manager where appropriate. This is required to make translations, soft-delete and more work. See comment on Addons.Meta for more details. * Various fixes along the way to make tests work * Port form errors for locale editing forms to Django 1.11 and make use of form.add_error instead of our own custom process * Mark TestDetailPage.test_304 as xfail for now, somehow Etags seem to be broken for that view during tests but work fine in manual tests * Fix amo.celery, we cannot have any imports at the module-level that may import the ORM * Fix global_settings context processor for 500-server error cases * Fix `changed_data` handling in `AMOModelForm` * Remove `queryset_transform` and implement it directly in amo/models * Fix pagination, Django changed how `num_pages` work and how `count` can be overwritten * Add a new test that tests database transforms as `BaseQuerysetTestCase` * Make sure `legacy_redirect` only appends `?` if there are actual GET parameters * Re-implement `LicenseRadioChoiceInput` as `LicenseRadioSelect` which is now Django 1.11 compatible * Remove our custom `form_kwargs` uplift from Django 1.9 * Correct how we're using `Category.objects.create` and only pass `db_name` instead of `name`, previously this didn't error but now it (luckily!) does * Update `FileSelectWidget` to correctly work with Django 1.11, use custom rendering * Fix decorator ordering for `files.views:browse` * Update database settings to correctly use `TEST` object * Fix `select_related` call in `RatingViewSet` * Remove custom `DateTimeCast` implementation * Fix `addon_factory` calls in `reviewers/tests/test_views.py` to correctly set `is_webextension` * Port our `TranslationDescriptor` and `TranslatedField` to Django 1.11. Also, make sure form error handling works * Add a new test that checks that `default=None` is a valid setting for translation fields * Add a new test that checks that translations from related fields are correctly fetched * Remove possible ambiguity from translations `TestWidget` tests * Make sure that the correct `MEDIA_URL` is set in `TestDownloadsUnlistedVersions` to fix external cdn tests * Remove custom happyforms code. Django cleans text fields now by default * Move translations app to the top of INSTALLED_APPS, add a lengthy comment * Pass additional arguments to contribute_to_class, fix smaller nits * Set MEDIA_URL and STATIC_URL to an external value by default. That's how they're configured in dev, stage and prod anyway. Please note that we didn't replace all our custom form rendering with proper templated widgets yet. This will happening in some follow-up work. Fixes #5271, Fixes #1569
2018-07-11 07:56:54 +03:00
python manage.py migrate --noinput --run-syncdb
schematic --fake src/olympia/migrations/
# Let's load some initial data and import mozilla-product versions
python manage.py loaddata initial.json
python manage.py loaddata zadmin/users
python manage.py loaddata src/olympia/access/fixtures/initial.json
python manage.py import_prod_versions
python manage.py update_permissions_from_mc
# Create a proper superuser that can be used to access the API
2018-01-26 10:19:33 +03:00
python manage.py waffle_switch super-create-accounts on --create
python manage.py waffle_switch activate-autograph-signing on --create
run-ui-tests: setup-ui-tests
# Generate test add-ons and force a reindex to make sure things are updated
pip install --progress-bar=off --no-deps -r requirements/uitests.txt
python manage.py generate_default_addons_for_frontend
python manage.py reindex --force --noinput --wipe
pytest --driver Firefox tests/ui/
perf-tests: setup-ui-tests
pip install --progress-bar=off --no-deps -r requirements/perftests.txt
locust --no-web -c 1 -f tests/performance/locustfile.py --host "http://olympia.test"
initialize: update_deps initialize_db update_assets populate_data