* Remove amo-validator, remove more support for uploading legacy-addons.
This is dropping most of our compatibility checks for legacy add-ons and
completely blocks them from being uploaded.
We workaround this block for mozilla-signed add-ons by issuing the same
error, just as a warning instead of an actual error.
This affected *loads* of our tests so the PR contains more test-changes
then I'd like.
In addition to that this PR adds file extension validation for uploads,
we only did that in our JavaScript and never in our code. So from now
on, we have a hard limit to only allow whatever is in
`VALID_EXTENSIONS`.
Further...
* drop amo-validator and related dependencies
* remove `dump_apps` management command
* removed unused fixture files from the repo
Fixes#10502
Future improvement idea for our test-suite: Unify add-on file fixtures
under one single folder to avoid any duplication.
There is most certainly some more improvements that could be made to
code and test structure but I went for the least changes, it's way too
huge already :-/
* Fix syntax error in JavaScript, thus ui tests
* Switch back to passing 'channel' around
* Move valid addon file extensions to constants/base
* Fix validate() call
* Fix more tests
* Update src/olympia/devhub/tests/test_views_validation.py
Co-Authored-By: EnTeQuAk <cg@webshox.org>
* Remove old comment
* Issue description specific only to Thunderbird and Seamonkey if targeting thunderbird or seamonkey
This doesn't run services yet, and is just useful to run tests. Because
it shares the same image as the regular web container, an initial
make update_deps is necessary to get it working. After that, running
pytest as normal should function normally and be using Python 3.
Unfortunately, we'll have to duplicate the environment config but I added
a comment so that should be fine. It's very rarely being used anyway.
Fixes#9288
* Changed fxa account generation to allow assigning an fxa ID to a created user.
* Moved pytest execution to makefile to access variables.
* Removed second login tests as the devhub tests also log in.
* Default to 'and' operator for match queries. Remove slug search, prioritize exact matches more.
References many "component: search" issues. What I tested with a
database of all public add-ons:
Example searches:
tab center redux - should find "Tab Center Redux" while "Tab Mix Plus" is probably second and "Redux DevTools" 4th or so
Open Image in New Tab -> should find "Open Image in New Tab" while "Open Bookmarks in New Tab" should be 2nd or 3rd
CoinHive -> Finds "Coinhive Blocker", "CoinBlock" (prefix search) and "NoMiners" (description)
Privacy -> Finds "Privacy Badger", "Privacy Pass", "Privacy Settings", "Google Privacy" (probably 4th or so) and "Blur" (summary + description + many users). Scores "Ghostery" on the first page but ranks it in the middle
firebu -> Finds "Firebug", "Firebug Autocompleter", "Firefinder for Firebug"
fireb -> Scores "Fire Drag" first, puts "Firebug" approximately 3rd or so
Menu Wizzard -> Finds "Menu Wizard" (fuzzy, typo) first, then "Add-ons Manager Context Menu" apparently because it matches good in the title and has many users
Frame Demolition -> Finds "Frame Demolition"
Demolition -> Finds only "Frame Demolition", same for "Demolation" (typo)
reStyle -> Finds "reStyle" and scores a few add-ons that match on "restore" next since the term is similar
MegaUpload DownloadHelper -> finds "MegaUpload DownloadHelper" first, scores "Video DownloadHelper" and "RadpidShare DownloadHelper" next. Doesn't find "Popup Blocker" anymore as currently happening on -prod
MegaUpload -> only finds "MegaUpload DownloadHelper" and nothing else
No Flash -> Scores "No Flash" first, then depending on users "Download Flash and Video", "YouTube Flash Video Player" and "YouTube Flash Player" (not necessarily in that order)
Disable Hello, Pocket & Reader+ -> finds "Disable Hello, Pocket & Reader+" first (yeay!), then scores "Reader", "Disable WebRTC" and "In My Pocket" next similarly to what's happening on -prod currently
Not working yet:
privacybadger -> "Privacy Badger" -> will probably need some kind of ngram filtering and analyzing (#591)
eyes -> 'decentraleyes' -> Not sure this should actually work, will probably need some more analyzing too (#591)
Not sure if it's specifically only because of these changes but #3248 is fixed.
This potentially fixes#7244, #6891, #6837, #6417, mozilla/addons#359.
Not sure if this fixes #mozilla/addons#567 but the results look much more promising and the amount of results doesn't explode here. I only have 2.8k add-ons for testing though so I'm not too sure.
And might be relevant to #6137.
This is a big step towards #2661, I doubt we can call this fixed though.
* Fix name tests
* Speed up ES tests, make scoring results more predictable by using only one shard and one replica
* Add tests
* Only test on new apiv3 based search.
* Fixup 'get_results'
* Remove debug print
* Isort imports
* Fix test settings, I'm blind.
* Adapt number of shards to what we define in settings_test
* Test search for grapple
* Remove property filtering in legacy api search, fix tests, fix flake8
* Add comment explaining shard config
* Fix tests again, add default platform, fix total counts again.
* Try to do some fixture cleanup, let's see...
* More test refactoring, make use of dfs-query-then-fetch during tests, allow us to test this later too via a waffle flag.
* Don't use waffle flag for legacy search, only for apiv3 searches.
* Fix unused import, fix usage of 'params'
* Use query-then-fetch in a regular addons-view tool, fix a few more tests to be able to handle the waffle-flag
* Create the dfs query then fetch flag only in ESTestCase, delete it properly. More serializer fixes
* precache the waffle flag for autocomplete tests too
* Minor cleanups
* Add docs, fix code style
* Add a todo
* Fix codestyle
* Use docker-compose>=1.18 and minimal docker-compose config version to 2.3
* Starts ui-tests in Firefox docker container
* Let's the Firefox docker-container hit directly nginx
* makes use of https://github.com/jrbenny35/selenium-firefox/ firefox+selenium image
* makes use of /user-media/ serving directly via nginx for XPI installs
* Adds a helper that creates an installable add-on
* Update `firefox_options` fixture with all options needed to install add-ons with a test signing signature
Fixes#7270Fixes#2488
Fixes#6640Fixes#6866
This pull request adds support for our new Autograph signing server, adds it to our dockerized services and adds tests that now actually call our signing server.
* Add a waffle-switch for calls to our new autograph signing server
* Add the autograph signing server to docker-compose
* Add the autograph signing server to our travis setup
* avoid us specifying the signing server settings everywhere in the code, one single source of truth
* more use of `responses` instead of mocking, leaning towards
the future where we can actually hit a signing server instead
of a mock
* disallow http requests generically, unless configured otherwise
* Add migration for waffle flag
* Update celery, config and dependencies to 4.x
* Remove usage of TaskSet, replace with group(), update `ampq`
* Enable redis on travis for better celery 4.x testing
* Fix lib/celery_tests
* Correct test testing email sending retrying on errors
* Fix celery subtask mocking errors.
* Fix celery broker url env variable
* Integrate watchdog for celery autoreload
* Ensure dependencies are updated for worker and web container.
* Restart all services after the dependency update
* Remove explicit celery loader environment variable.
* Restart all services in web and worker after running 'update_docker'
* Increase sleep time a bit
This reverts commit 541688d5f2.
This caused some breakage on dev. We suspect it's because newrelic
hooks into the celery task launching code but does not support
celery 4.x yet.
* Update celery config to seamlessly upgrade to 4.x
* Update celery initialization for new configs and 4.x
* Update dependencies to celery 4.x
* Fix flake8
* Fix typo
* Remove usage of TaskSet, replace with group(), update `ampq`
* Enable redis on travis for better celery 4.x testing
* Fix more test setting names
* Fix lib/celery_tests
* Correct test testing email sending retrying on errors
* Fix celery subtask mocking errors.
* Fix new line
* Fix celery broker url env variable
* Integrate watchdog for celery autoreload
* Try something weird... ui-tests
* do we need extra hosts?
* more sleep?
* Try logging, try docker run to avoid port collisions
* More debugging
* Cleanup again
* Fix logs?
* Ensure dependencies are updated for worker and web container.
* Remove not functioning log saving for onw
* Restart all services after the dependency update
* Updated config to use circleci as well as configure ui-tests to run within docker
* Removed hacky certifi installs
* Update requirements to master versions
Fixes#4697 (though not only that)
It took me days to find this and loads of useless refactoring but
apparently these tasks get immediately upgraded to a chord after we
chain them later with the other tasks (after the for-loop).
And, the only requirement for a chord is: do not ignore the results,
never!
Refs #3960
* Make sure index_stats isn't using call_command
* modernize create_subtasks, add support for countdowns, fix queries loading the world...
* half chunk size for stats reindexing
* Refactor stats reindex logging
* Upgrade elasticsearch to 5.4.1 for our docker config
* Make es test setup easier to inherit for our uitests.
* Restore order, make sure to start es before add-on get's created.
* Remove db requirement from es_test fixture
* Bump elasticsearch in travis to 2.3.3
* Fix index store config option for es 2.3.3
* Install es 5.1.1
* Fix es reference in travis.yml
* Fix es startup
* Let's try and switch to use jdk8
* Fix bulk_index import
* Set es 5.1 for docker compose too
* Fix docker image to use official elastic images.
* small travis cleanup
* Make sure xpack authentication is disabled for testing (for now)
* Let's try disabling xpack on travis too...
* Stupid hack to support es5 default auth on travis...
* Remove custom travis_es script
* Use ES 5.3 for now...
* Upgrade to Django 1.7.10
* Upgrade to Jingo 0.9
* Make sure the app minimalistically runs with Django 1.7
* Upgrades django-waffle and django-cache-machine
* Upgrades to a forked hack of django-piston
* Fixes systems checks
* update redisutils config
* Get rid of Jingo monkey patch.
* Update to proper SafeData support (which now supports __html__)
Andrew Williamson
Christopher Grebs
Mathieu Pillard
Benjamin Forehand Jr
Greg Guthe
bqbn
Stuart Colville
Jason Thomas
Mark Striemer
Kumar McMillan
Andy McKay