This switches document results to use DocumentDetailSerializer. It also
fixes DocumentDetailSerializer to include the document summary which is
on the current revision and the document url.
I have no idea if this is the format we want to go with, but it is some
documentation.
Maintaining it will be painful. It's probably better for us to figure
out a way to autogenerate the documentation or somehow move the
documentation to the code. That work is covered in bug #1164247.
nose has a skip plugin which enhances SkipTest raising beyond our
wildest imaginations. However, there are certain occasions where we want
skipped things to not skip with reckless abandon but instead don the
malevolent cloak of errorhood.
nose has a --no-skip argument for exactly this sort of sordid thing, but
it's totally busted and doesn't work.
This changes the code around so that it works correctly.
* the hacking howto has instructions for creating the db that don't
include setting the CHARSET so it would die on creating revisions
using the revision model maker which includes unicode data
* the revision model maker code value strings were ... "interesting"
* cosmetic: i tweaked the formatting in the MozillaNews.wiki
This updates polib and adds dennis to vendor/. Now we can do things
like:
./manage.py lint
./manage.py translate
Also, this nixes poxx.py and localelinter.py which aren't needed
anymore.
I have Firefox Nightly earlier in my path than Firefox stable. Nightly
has problems with Selenium that haven't been fixed, yet, (or something
like that). This allows me to specify exactly which Firefox binary to
use without having to fiddle with my PATH environment variable which
caused other problems because of virtual environments and such.
It defaults to exactly what happened before, so if it worked for you
before, you should see no change with this change.
Before running the LiveServerTestCase-based tests, we need to run:
./manage.py collectstatic
./manage.py compress_assets
This change attempts to clarify that in the test docs.
* add document_id to index
* update to elasticutils master tip -- this picks up pyelasticsearch
and ditches pyes
* nix get_indexing_es (it was dumb)
* remove bunch of now-unused settings (also dumb)
* tweak test scaffolding -- this should make tests run faster
for kitsune in a VM and on slower machines
* clean up docs
Note: This doesn't remove pyes. For the purposes of making this easier
to review, that'll get done in a future commit.
Note: This adds a new field to the mapping, but it's only used in indexing
and it's obviously a string. Thus it's ok for ES to infer what it is
and we don't have to do the "change the mapping" two-step dance.
* Make LESS required
* Move LESS and memcached instructions to the Requirements section.
* Tweaked and moved the NOTE about empty CSS to the part where
the issue may be encountered.
* URL for the add-on is now a setting, to change it easily.
* AMO will provide automatic updates to the add-on. We could have done
this in SUMO, but it is not trivial.
* The url used here will automatically point to the latest version.
* Remove the old add-on from the media directory.t
* Add docs about the add-on.
* The version of the add-on in AMO will work on -dev and -stage.
This adds:
1. xx locale which is Pirate
2. scripts/poxx.py which translates a .po file into Pirate
3. scripts/test_locale.sh which extracts strings and runs poxx.py
The xx locale is set up so that it doesn't show up in the language
chooser lists, but you can explicitly specify it in the url:
http://localhost:8000/xx/home
The locale is only available translated locally--it will show up
as English on -dev, -stage, and -prod. It'll show up as English
if you don't run scripts/test_locale.sh.
Nixed settings_local.prod.py from the codebase. It never gets updated,
it was old and stale and stinky like the floor of your local dive bar,
and we have better docs already elsewhere.
* clean up the settings_local.py example
* move the redis install and configuration bits to a separate chapter
* add links to the advanced installation section
* moved some bits that were in installation chapter to the hacking_howto
* fixed the note about Elastic Search 0.19.9 so it's correct.
We don't have it set up like this on our machines and if you don't have
redis running, then this causes a bunch of odd problems.
Better to nix it from the docs.
I nixed the Redis section. It shouldn't be in the hacking howto.
I also tweaked the text at the top to be clearer that if you're
running a production instance, you should read all the way through.
This allows us to reindex a specific doctype from the cli and the
admin. This helps when we only need to reindex the kb--now we don't
have to reindex everything with it.
* Splits the docs into three sections---trying to reduce the "omg!
there are a million sections!" overwhelming feeling
* Moves the test-related things to the tests chapter so they're all
in one place
* Turned "coding" into "conventions"---that's what it really is.
* Fleshed out the git commit message conventions---it keeps coming up
and we pointed to a guide which pointed to a guide and the links
were buried in both places.
After talking with Matt, we decided to use the terms "regular" and
"advanced" for searches. This updates the documentation accordingly
and also cleans up some of the other text.
This removes all the Sphinx-related stuff.
You may need to do:
git rm --cached vendors/src/oedipus vendor/src/sphinxapi
to remove the vendor git submodule stuff.
* add --delete to reindex cmd: By default, it **won't** wipe the index.
It now only wipes and rebuilds the index from scratch if you tell it to.
* add note about fixing Redis scoreboard to the docs: If you're doing dev
work and you purge the celery tasks, you also need to reset the Redis
scoreboard that keeps track of outstanding tasks. This adds docs for that.
* add delete-then-index to admin: This reworks the reindex section so that
it has a "delete-then-index" button and makes it clearer which button
does what and why you'd use it.
* adds the DANGER style to admin page.
* reworks indexing to be parallelized when run from the admin
* changes the admin/search page to show outstanding tasks
* removes all the old progress-y stuff
* fixes the --checkindex flag for esstatus
* adds note to the ES documentation about purging celery--that's super
helpful
Since the weights weren't actually being applied since they were specified
wrong, I removed them a couple of commits ago.
Given that, I'm updating the docs to reflect that.
* fixed some typos
* add hacking howto, contact us, and contributor docs
* added an AUTHORS file
The hacking howto probably doesn't work, but it's close. It needs someone
to go through it and fix issues in the docs and in the code to make it
work better.
* Ricky suggested using schematic to tell us the version number
rather than hit the database for it.
* Ricky suggested adding a note for making sure to insert the
content type and default permissions for new models.
* Then I went through and fixed the wrapping for a bunch of
lines
* Splits searching and indexing so they use different index
settings.
* Rename eswhazzup to esstatus and make it more useful.
* Make search admin more useful so it's easier to spot problems.
* Updates eswhazzup to be more useful and nixes the cluster status
* Add x per 1000 documents calculations to reindexing output.
* Fixes wiki document indexing so that when doing a reindexing, we're
not calling unindex for things.
* Add esdelete command.
* Cleans up some code to make it more obvious and less errory.
* Update documentation.
* Update default settings.
This adds documentation for search scoring as it's currently implemented.
Additionally, it adds some minor notes about where the ES-related code is
and links to the search view where the filters are.
Also adds a link to elasticsearch-head.
* update the docs regarding hudson -> jenkins naming of the irc bot
* add note about model makers to coding docs
* fixed a header (cosmetic)
* I added the email/name setting to the quick section since if you miss
this step (which is not unlikely for new contributors), you have
to do some fancy footwork to fix the commits you've made.
* the ES connection already has code for forcing bulk, so we don't need to
repeat that. this changes the code to push the setting to ES.
* this also tweaks the estimation code so that it shows minutes and seconds and
shows the total delta later. Now I can stop running
"time ./manage.py esreindex".
* fix esreindex so that you can specify doctypes. This will appropriately
create/delete indexes so that what you don't want to delete won't get
deleted.
* adds basic handling for bad data.
This does a log.exception, but we really should log more than that and/or
make it more obvious to developers that there's bad data out there.
In the meantime, this allows us to continue indexing.
* reduced memory usage of indexing by iterating over ids---now it runs on
my laptop.
* ghanges _get_index() to get_es_index(). We use it so often it might as
well be part of the "public API".
* fixed create/delete indexes so that to switch doctypes to their own index
is now just a change in settings---no code changes needed.
* fix DEBUG = True case by reseting queries
* this also adds a bunch of helpful comments, moves reindex_model to
SearchMixin.index_all, and has some other cosmetic code cleanup.
End result of this is that indexing doesn't die if it hits bad data, indexing
takes much less memory to run, you can specify specific doctypes to index at
the command line, and the code is better.
This reduces the triplification of everything in es_search modules.
This reduces the triplification of everything in es_search modules.
This reduces the triplification of everything in es_search modules.
Also, I introduced ES_INDEXING_TIMEOUT, fixed documentation, and
cleaned up some other things while I was poking around.
Also also, I nixed all the ES mapping constants. Calling put_mapping
with typos will kick up an error, so there's no need to additionally
have constants around to prevent typos.
This tweaks the language to make it clearer that the vendor stuff
is in the kitsune repository and not a separate one.
It also turns a caveat thing into a warning so it shows up better.
KarmaManager() gets created before we stomp on the REDIS_BACKENDS settings
with REDIS_TEST_BACKENDS. This switches all that testy stuff around to use
settings_test.py which stomps on everything before modules get imported.
The advanced search works, but the front-page search and the searching from within the questions app are still buggy (returning 0 results). Thus, we'll leave the waffle flags off for now.
* unhardcodes indexes
* ditches ElasticMeta--elasticutils uses Model._meta.db_table for
queries, so we should use that for building indexes, too
* implements incremental updates for Question and adds test
* adds some test harness code for future elastic tests
* adds basic docs for setting up Elastic