This removes all the Sphinx-related stuff.
You may need to do:
git rm --cached vendors/src/oedipus vendor/src/sphinxapi
to remove the vendor git submodule stuff.
* add --delete to reindex cmd: By default, it **won't** wipe the index.
It now only wipes and rebuilds the index from scratch if you tell it to.
* add note about fixing Redis scoreboard to the docs: If you're doing dev
work and you purge the celery tasks, you also need to reset the Redis
scoreboard that keeps track of outstanding tasks. This adds docs for that.
* add delete-then-index to admin: This reworks the reindex section so that
it has a "delete-then-index" button and makes it clearer which button
does what and why you'd use it.
* adds the DANGER style to admin page.
* reworks indexing to be parallelized when run from the admin
* changes the admin/search page to show outstanding tasks
* removes all the old progress-y stuff
* fixes the --checkindex flag for esstatus
* adds note to the ES documentation about purging celery--that's super
helpful
Since the weights weren't actually being applied since they were specified
wrong, I removed them a couple of commits ago.
Given that, I'm updating the docs to reflect that.
* fixed some typos
* add hacking howto, contact us, and contributor docs
* added an AUTHORS file
The hacking howto probably doesn't work, but it's close. It needs someone
to go through it and fix issues in the docs and in the code to make it
work better.
* Ricky suggested using schematic to tell us the version number
rather than hit the database for it.
* Ricky suggested adding a note for making sure to insert the
content type and default permissions for new models.
* Then I went through and fixed the wrapping for a bunch of
lines
* Splits searching and indexing so they use different index
settings.
* Rename eswhazzup to esstatus and make it more useful.
* Make search admin more useful so it's easier to spot problems.
* Updates eswhazzup to be more useful and nixes the cluster status
* Add x per 1000 documents calculations to reindexing output.
* Fixes wiki document indexing so that when doing a reindexing, we're
not calling unindex for things.
* Add esdelete command.
* Cleans up some code to make it more obvious and less errory.
* Update documentation.
* Update default settings.
This adds documentation for search scoring as it's currently implemented.
Additionally, it adds some minor notes about where the ES-related code is
and links to the search view where the filters are.
Also adds a link to elasticsearch-head.
* update the docs regarding hudson -> jenkins naming of the irc bot
* add note about model makers to coding docs
* fixed a header (cosmetic)
* I added the email/name setting to the quick section since if you miss
this step (which is not unlikely for new contributors), you have
to do some fancy footwork to fix the commits you've made.
* the ES connection already has code for forcing bulk, so we don't need to
repeat that. this changes the code to push the setting to ES.
* this also tweaks the estimation code so that it shows minutes and seconds and
shows the total delta later. Now I can stop running
"time ./manage.py esreindex".
* fix esreindex so that you can specify doctypes. This will appropriately
create/delete indexes so that what you don't want to delete won't get
deleted.
* adds basic handling for bad data.
This does a log.exception, but we really should log more than that and/or
make it more obvious to developers that there's bad data out there.
In the meantime, this allows us to continue indexing.
* reduced memory usage of indexing by iterating over ids---now it runs on
my laptop.
* ghanges _get_index() to get_es_index(). We use it so often it might as
well be part of the "public API".
* fixed create/delete indexes so that to switch doctypes to their own index
is now just a change in settings---no code changes needed.
* fix DEBUG = True case by reseting queries
* this also adds a bunch of helpful comments, moves reindex_model to
SearchMixin.index_all, and has some other cosmetic code cleanup.
End result of this is that indexing doesn't die if it hits bad data, indexing
takes much less memory to run, you can specify specific doctypes to index at
the command line, and the code is better.
This reduces the triplification of everything in es_search modules.
This reduces the triplification of everything in es_search modules.
This reduces the triplification of everything in es_search modules.
Also, I introduced ES_INDEXING_TIMEOUT, fixed documentation, and
cleaned up some other things while I was poking around.
Also also, I nixed all the ES mapping constants. Calling put_mapping
with typos will kick up an error, so there's no need to additionally
have constants around to prevent typos.
This tweaks the language to make it clearer that the vendor stuff
is in the kitsune repository and not a separate one.
It also turns a caveat thing into a warning so it shows up better.
KarmaManager() gets created before we stomp on the REDIS_BACKENDS settings
with REDIS_TEST_BACKENDS. This switches all that testy stuff around to use
settings_test.py which stomps on everything before modules get imported.
The advanced search works, but the front-page search and the searching from within the questions app are still buggy (returning 0 results). Thus, we'll leave the waffle flags off for now.
* unhardcodes indexes
* ditches ElasticMeta--elasticutils uses Model._meta.db_table for
queries, so we should use that for building indexes, too
* implements incremental updates for Question and adds test
* adds some test harness code for future elastic tests
* adds basic docs for setting up Elastic
* Videos are identified by title and locale
* Depends on an update in py-wikimarkup which adds support for <video> and <source> tags (py-wikimarkup commit cc06e6d264622891b6b018e8670c9ef4bb12d618)
* Attaches all the _hook_*s to the WikiParser class, because they need a contextual locale.
* Adds locale support for any of the hooks that do document lookup.
* Uses SWFobject JS lib to support flash fallback for video.
* Adds a migration for unique ('locale', 'title') on gallery_video and gallery_image
* Adds a WIKI_VIDEO_WIDTH|HEIGHT constant that may be used as MAX_WIDTH|HEIGHT in the future, once we get video thumbnails.
* Raise FixtureMissingError in test helpers, e.g. on creating media gallery image and wiki revision, instead of creating testuser
* Add ImageUploadForm and VideoUploadForm
* Simplify gallery app's urls.py
* Adds a bunch of gallery utils to create and upload media
* Adds two views up_media_async, del_media_async
* Rename upload_images to upload_imageattachments
* Define a more generic upload_media function to be used by the gallery app
* Document some of our coding conventions
* Add a migration for the video model
* Define a MAX_FILESIZE for video uploads (16 megabytes), overwritable in settings_local.py
Includes:
* migrations for the models
* a shared abstract model Media for the concrete Image and Video models
* restricting locale choices in the gallery and wiki models
* some check.py complaints and cleanup
* a documentation section for production regarding upload paths
* Tags are added for full and tenths-place variants (e.g., 4.0.1 and 4.0) of known Firefox versions.
* An OS will be auto-tagged if there exists a tag with its name (case insensitive). (I posit ease of administration outweighs risk of abuse.)
* Finished off questions_config based on Cww's etherpad.
* Show user agent string in System Details pop-up.
Will Kahn-Greene
Berker Peksag
Ricky Rosario
Rehan Dalal
Erik Rose
James Socol
Tanay Gavankar
Paul Craciunoiu
Joshua Gross