Граф коммитов

763 Коммитов

Автор SHA1 Сообщение Дата
Mathieu Pillard 32ba705b4b
Load addons-server static files and user-uploaded images from the main domain on stage and prod (#18110)
* Load addons-server static files and user-uploaded images from the main domain on stage and prod

* Adjust tests

* black
2021-10-15 11:53:13 +02:00
Andrew Williamson f58ecaebf0
switch memcache backend to pymemcache (#17691) 2021-08-11 11:46:08 +01:00
Andrew Williamson 6101446f3c
implement sitemap.xml with basic site and addon support (#16904)
* implement sitemap.xml with basic site and addon support

* rm commented out lines from development
2021-04-13 11:04:48 +01:00
William Durand 6765fbbe07
Update settings.py (#15977) 2020-11-12 13:39:24 +01:00
Mathieu Pillard 4e56d728ce
Add special URL to bypass authentication locally (#15702)
* Don't run generate_default_addons_for_frontend outside ui tests

* Add special URL to bypass authentication locally

* Always go through generate_default_addons_for_frontend but skip fxa if not needed

* Add fake FxA authentication flow for local development

* Revert "Add special URL to bypass authentication locally"

* Tests for the view

* Test fxa_login_url()

* More testing

* Remove leftover obsolete import
2020-10-12 19:19:50 +02:00
Lisa Chan cb3c4b457f
RE: Add validations to Shelf model (#14998)
* Draft pull request - validator for criteria

* Updated validation for criteria; revised models, test_admin, conftest

* Added additional criteria validation tests to test_admin

* Replaced validator with clean_criteria method

* Removed responses for shelves

* Revised clean_criteria method and added tests to test_admin.py

* Revised shelf types in models.py

* Added message to inform admin the total addons returned after successfully saving a shelf

* Updated success message and respective tests

* Created test_forms, revised forms, admin, test admin

* Updated settings to add INTERNAL_SITE_URL and nginx to ALLOWED_HOSTS

* Revised baseUrl, removed results/save method - to add later

* Revised settings (dev, stage, prod) to move INTERNAL_SITE_URL near EXTERNAL_SITE_URL, moved 'nginx' to list from settings_base.py

* Formatting
2020-08-05 11:35:25 -04:00
Andrew Williamson ac50305b57
drop Block.include_in_legacy and derive legacy state from legacy_id (#15043) 2020-07-27 10:47:51 +01:00
William Durand de18ebf5c8
Add waffle flag for the beta stats (#14542) 2020-06-05 17:45:39 +02:00
William Durand 75c3cbfce7
Fix logging config (#14486) 2020-06-05 13:30:11 +02:00
William Durand 18bf215bd6
Introduce new setting for internal routes (#14254) 2020-05-25 11:51:18 +02:00
William Durand 9a129eb7a4
Update logging config (#14310) 2020-05-19 16:54:15 +02:00
Andrew Williamson 068d33509f
submit blocks to Kinto blocklist bucket (v2) (#13602)
* submit blocks to Kinto blocklist bucket (v2)

* add is_imported_from_kinto_regex test

* change KINTO_API_IS_TEST_SERVER default to False
2020-03-03 18:27:34 +00:00
Andrew Williamson f6dafeb47a
move /authenticate api endpoint to new versionless api; update clients (#12570) 2019-10-11 17:20:43 +01:00
Andrew Williamson 855464d389
Merge pull request #12532 from eviljeff/12417-drop-skip-register-redirect
drop skip_register_redirect + some refactoring
2019-10-08 15:55:42 +01:00
Andrew Williamson 493f01d63a change default to be prod fxa server; stage to use prod fxa server 2019-10-08 11:47:24 +01:00
Andrew Williamson 8a5b67d43e update redirects_url comments to make it clearer what it is 2019-10-08 09:48:10 +01:00
MyeongJun Go 9cf0ac5b43 fixed debugtool middleware be after GZip middleware (#12443)
Fix django-debug-toolbar middleware be after GZip middleware
2019-10-07 15:33:49 +02:00
Andrew Williamson 75c4d139cb drop skip_register_redirect + some refactoring 2019-10-04 18:47:05 +01:00
William Durand 98348e2a43
Use XSENDFILE in local dev (#12312) 2019-09-10 12:57:24 +02:00
William Durand 87a2cf7d35
Add task to run `wat` (#12225) 2019-09-09 13:04:44 +02:00
William Durand 458cc64170
Add task to run `customs` (over HTTP) (#12191) 2019-08-29 12:21:20 +02:00
Christopher Grebs d6c44135ce Merge pull request from GHSA-jq22-wqf5-666p
* Serve files in old file-viewer through FileResponse.

Fixes bug 1566954

The old file-viewer will be going away "soon" anyway so we should be
using the same technique as for the new one.

The problem here specifically is that when going through Nginx via
X-Accel-Redirect nginx isn't setting proper CSP headers for the
download. Serving the files ourselves allows us to much easier restrict
CSP in the future even further.

* Explicitly restrict CSP config on download endpoints.

* Set report uri in settings.py too

* Explicitly add frame-ancestors

* Remove outdated test
2019-07-18 16:52:11 +01:00
Christopher Grebs 2de22598bd
Remove all usage of 'six' and as much compat code as I could f… (#11730)
* Remove all usage of 'six' and as much compat code as I could find.

Cleans up some imports along the way.

Fixes #11728

* Fix typo

* Fix rta related code paths, I actually misread the comment…

* Move ResourceWarning filtering to setup.cfg
2019-07-16 12:01:31 +02:00
Mathieu Pillard 766ac85e1a Re-add Django-Debug-Toolbar for easier debugging, particularly in the admin 2019-07-03 22:43:09 +02:00
Mathieu Pillard 0d5e70bc29 Make sure File.get_url_path() returns a relative path
Also make sure get_absolute_url() is used as much as possible when we
want the absolute url, instead of doing it ourselves using absolutify.
2019-06-06 12:25:53 +02:00
Kumar McMillan d7b9e6cc6a
Link to Code Manager from reviewer tools (#11038) 2019-03-28 23:35:03 -05:00
William Durand 2e44800eac
Add new code-manager FxA configs (#11016) 2019-03-27 12:24:05 +01:00
Andrew Williamson 85e9ea6f03
use utf8mb4 everywhere; replace reset_db (#10935) 2019-03-18 10:57:49 +00:00
Andreas Wagner bd3500a392
Fix flake8 errors outside src/ 2019-03-04 16:01:00 +01:00
Andrew Williamson 8420fb2ee2
replace django-cors-headers-multi with django-cors-headers (#10704) 2019-02-18 21:04:11 +08:00
Adrian Utrilla c8e1a28208
Use utf8mb4 for connections to MySQL in dev 2019-01-18 17:29:32 +01:00
Mathieu Pillard 70760159c3 Replace urlparse imports by six.moves.urllib_parse for Python 3 compatibility
This keeps backwards-compatibility to Python 2 but moves us closer to running
with Python 3.
2019-01-07 17:43:17 +01:00
Christopher Grebs 2c82ea7728
Enable memcached by default for tests. Upgrade python-memcached
* Adds a test to test unicode support for latest python-memcached
  versions
* Configures caches key prefix so that tests don't conflict with each other
* Remove a few cache.clear calls with `cache.delete` calls that avoids
deleting all memcached entries while other tests are running
* Remove zadmin.memcached as it's crazy dangerous, such power should
only be wielded by ops-folks who wake up at night when pressed
* Fixes waffle helpers to call .flush() to flush appropriate caches

This also fixes how parameters are forwarded to memcached and fix KEY_PREFIX initialization

We are now making sure parameters from `settings.CACHES` are properly forwarded
to the actual cache backend (now memcached). These errors were not seen
since LocMemCache didn't require any parameters.

The `KEY_PREFIX` setting is set to a uuid instead of the xdist-slaveid to
ensure test functions are properly isolated from each other.

`cache.clear` is not called anywhere unless absolutely necessary to reduce
the side-effects of our test-setup.

Fixes #8602
2018-11-21 18:26:30 +01:00
William Durand e6ed0f3371
Use the right domain in each env (#9727) 2018-10-17 17:52:21 +02:00
Christopher Grebs 5655295e05
Run validator via subprocess, huge cleanup along the way (#9682)
This ports all amo-validator related code to run via subprocess. This
makes us less blocked by it not running on Python 3, there may still be
some installation / execution related bits to actually run it via python
2 instead of python 3 but that's something for later.

* Cleans up how and where we're setting _BIN settings for all the
binaries we're calling
* Backports `unicodehelper` from validator
* Removes `requeue_uploads` since it's been broken and unused for at
least 3-4 years
* Remove Standalone Add-on Compatibility Checker (/developers/addon/check-compatibility/)
* Remove compat-check related bits from standalone add-on upload
* Remove more bulk compat report stuff that isn't used anymore
* Removed zadmin application_versions.json view, unused

Fixes #9670
"Fixes" #8857
2018-10-16 08:30:16 +02:00
Mathieu Pillard d7876929b3 Remove filesystem cache, use default (memcached) backend for everything
Only the language tools API used the filesystem cache, and that was
because it had a relatively low usage compared to cache-machine, which
meant everything it cached was quickly evicted. Now that cache-machine
is gone, let's use memcached again for this.
2018-07-27 11:35:42 +02:00
Christopher Grebs a2ddf0d5f4
Avoid logging through mozlog in tests experiment (#8528)
Closes #8451 (Doesn't necessary fix it, we don't want a unified logging-level but this PR makes logging much clearer.)

* Don't use verbose mode for tox
* Refactor logging settings
* Fix services logging, correct dev and stage settings, update docs
* Update logging
* Be vigilent about disabling all logging
* Remove MOZLOG_NAME from stage settings
2018-06-18 06:10:26 -07:00
Christopher Grebs 46087ea63e
Completely remove traces of caching.backends settings.
Most of this work landed already in
6a8fc341da and
91605a4c1f so this is just a follow-up to
remove all other traces.

The removed comment is no longer true, Django received all the necessary
fixes for LocMemCache. On the other hand, LoMemCache is no longer used
for a regular local setup because we are using memcached and try to be
as close to production as possible so this won't change anytime
soon.
2018-06-16 22:56:10 -07:00
Christopher Grebs 7949c933be
Report if there are any errors in local_settings.
This will issue a `UserWarning` in case some kind of exception is being
raised inside local_settings.py
2018-06-03 13:37:40 +02:00
Christopher Grebs c1112d89a1
WIP: Cleanup locustio based performance / smoke tests, add more entities (#8358)
* Port mozilla/amo-loadtest to FXA authentication and WebExtensions.

This is the first part of many to improve our load-tests and allow them
to be run regularly as part of a regression test suite.

This commit primarily ports the existing tests over to our repository,
updates them to use FxA authentication and uses a WebExtension for the
upload test.

This only implements the baseline for more work to come in #7744 but
it's important to have authentication and the basic infrastructure
working correctly.

It's also still using the legacy add-on detail and listing pages,
that'll obviously change - or be only true for SeaMonkey and Thunderbird
related tests since we still have to support them (very low priority though)

It also adds a first step of a summary report that links to new-relic.
That'll need a bit more tooling and testing but it worked quite well in
first tests.

Refs #7744

Next steps:
 * Implement most of the read-only tests from #7744
 * Check what needs auth in #7744, implement it properly
 * Implement database and cache query logging
 * Implement a unified merged test-report that uses the database and
 cache query logging and merges it with locust logs (using our new
 unique request id)

* Fix makefile, show dummy-usage for now.

* Delete generate-summary script for now

* Pin and cleanup dependencies

* Reverse all urls

* Use passed in account for login

* split things up

* Use gevent for waiting, some cleanups, add first version of Dockerfile, running script, disable developer stuff for now

* Add browsing collections and browsing categories

* Add loads more variation, add support for thunderbird, seamonkey, multiple languages

* Decrease size of docker images for local testing

* Get installation and docker image mostly to work, docs, cleanups

* Allow fxa environment variables be overwritten

* Hit legacy site of every variation

* More explicit theme testing

* Add browsing for app-versions and various rss feeds

* More variation in rss feeds, add featured and search tools

* Add a few more response.success() calls

* Test user profile pages, warn for empty collection pages

* More docs

* Add review browsing
2018-05-30 16:46:16 +02:00
Andrew Williamson 561f57138c
match both v3 and v4 when we're testing for api in middleware, etc (#8291) 2018-05-29 09:41:07 +01:00
William Durand cd989ed6f6
Redirect new users to edit profile in -dev (#8297) 2018-05-18 16:35:15 +02:00
Andrew Williamson cae6c3ad2f
drop mozilla-logger and all logging1.0 support (#8242)
* drop mozilla-logger and all logging1.0 support

* drop (hopefully) unused SYSLOG_CSP setting

* drop 'error' formatter; refactor 'debug' formatter format a little
2018-05-11 18:37:42 +01:00
Mathieu Pillard e01868ca5e Remove references to obsolete addon collector (retired in 2012) 2018-05-09 15:56:10 +02:00
Mathieu Pillard c3e6d3cc1f Add a setting controlling whether cache-machine is enabled or not
- Also, use that setting to disable cache-machine by default in tests,
  locally and on dev.
- Tweak translations tests to not depend on cache-machine at all
  (remove dependency on ModelBase, depend on django models only
   + the translation transformer)
2018-04-25 15:03:22 +02:00
Andrew Williamson 55bbcb1556
Simplify settings files (#8001) 2018-04-09 22:48:02 +08:00
Andrew Williamson 3faf451dca
remove Stylus (#7820) 2018-03-19 22:19:31 +08:00
Mathieu Pillard c125bf9354 Implement appversion compatibility filtering in language tools API
- Allow filtering by type to only show language packs
- Expose a new current_compatible_version if appversion is passed.
- This property will fetch the latest publicly available version
  compatible with the appversion passed.
- Replace caching with a filesystem-based implementation to work
  around cache eviction issues.
2018-03-05 15:44:46 +01:00
Andrew Williamson a00451308d
Update fxa config with new client for new redirect url 2018-02-08 16:40:45 +08:00
Christopher Grebs 17bb6968e5
First part moving to olympia.test 2018-01-26 07:11:48 +01:00