diff --git a/docs/migrating_0.90_to_1.0.rst b/docs/migrating_0.90_to_1.0.rst index 8500689..275138e 100644 --- a/docs/migrating_0.90_to_1.0.rst +++ b/docs/migrating_0.90_to_1.0.rst @@ -12,7 +12,7 @@ Summary ======= There are a bunch of API-breaking changes between Elasticsearch 0.90 -and 1.0. Because of this, it's really tricky to get over this hump +and 1.0. Because of this, it's really tricky to get over this hump without having downtime. This document covers a high-level walk through for upgrading from @@ -20,18 +20,6 @@ Elasticsearch 0.90 to 1.0 and the steps you should take to reduce your downtime. -Resources -========= - -.. seealso:: - - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/breaking-changes.html - Breaking changes when migrating to Elasticsearch 1.0 - - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_deprecations.html - Deprecated features when migrating to Elasticsearch 1.0 - - Steps ===== @@ -49,6 +37,9 @@ Steps 4. Make any changes to your code so that it works with both Elasticsearch 0.90 and 1.0 + There are some tricky things here, see the :ref:`tricky-things` section. + + 5. Upgrade to Elasticsearch 1.0.3 6. (Not implemented, yet) Upgrade to ElasticUtils 0.11 @@ -57,3 +48,145 @@ Steps At that point, you should be fine. + + +Resources +========= + +.. seealso:: + + http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/breaking-changes.html + Breaking changes when migrating to Elasticsearch 1.0 + + http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_deprecations.html + Deprecated features when migrating to Elasticsearch 1.0 + + +.. _tricky-things: + +Tricky things +============= + +There are a few tricky differences between Elasticsearch 0.90 and 1.0 that +will affect your code. + + +Changes with ``.values_dict()`` and ``.values_list()`` +------------------------------------------------------ + +Explanation +~~~~~~~~~~~ + +In Elasticsearch 1.0+, you get back different shapes of things depending +on whether you specify "fields". To smooth this out and normalize the +differences between Elasticsearch 0.90 and 1.0, ElasticUtils now **always** +passes in ``fields`` property when you use +:py:meth:`elasticutils.S.values_list` and +:py:meth:`elasticutils.S.values_dict`. + +Let's show some code to illustrate the new behavior. + +First, a bunch of setup: + + >>> from elasticutils import get_es, S + >>> from elasticsearch.helpers import bulk_index + >>> URL = 'localhost' + >>> INDEX = 'fooindex' + >>> BOOK_DOCTYPE = 'book' + >>> PERSON_DOCTYPE = 'person' + >>> es = get_es(urls=[URL]) + >>> es.indices.delete(index=INDEX, ignore=404) + +Now define the two document mappings we're going to use: book and person. +Book has no stored fields. Person has two. + + >>> mapping = { + ... BOOK_DOCTYPE: { + ... 'properties': { + ... 'id': {'type': 'integer'}, + ... 'title': {'type': 'string'}, + ... 'tags': {'type': 'string'}, + ... } + ... }, + ... PERSON_DOCTYPE: { + ... 'properties': { + ... 'id': {'type': 'integer', 'store': True}, + ... 'name': {'type': 'string', 'store': True}, + ... 'weight': {'type': 'integer'} + ... } + ... } + ... } + +Create the index with the mappings, add some books and add some people. + + >>> es.indices.create(INDEX, body={'mappings': mapping}) + >>> books = [ + ... {'_id': 1, 'id': 1, 'title': '10 Balloons', 'tags': ['kids', 'hardcover']}, + ... {'_id': 2, 'id': 2, 'title': 'Puppies', 'tags': ['animals']}, + ... {'_id': 3, 'id': 3, 'title': 'Dictionary', 'tags': ['reference']}, + ... ] + >>> bulk_index(es, books, index=INDEX, doc_type=BOOK_DOCTYPE) + (3, []) + >>> people = [ + ... {'_id': 1, 'id': 1, 'name': 'Bob', 'weight': 40}, + ... {'_id': 2, 'id': 2, 'name': 'Jim', 'weight': 44}, + ... {'_id': 3, 'id': 3, 'name': 'Jim Bob', 'weight': 42}, + ... ] + >>> bulk_index(es, people, index=INDEX, doc_type=PERSON_DOCTYPE) + [...] + >>> es.indices.refresh(index=INDEX) + [...] + +Now let's do some queries so we can see how things work now. + +Let's build a ``basic_s`` that looks at our Elasticsearch cluster and +the index. Also a ``book_s`` and a ``person_s``. + + >>> basic_s = S().es(urls=[URL]).indexes(INDEX) + >>> book_s = basic_s.doctypes(BOOK_DOCTYPE) + >>> person_s = basic_s.doctypes(PERSON_DOCTYPE) + +How many documents are in our index? + + >>> basic_s.count() + 6 + +Call ``.values_list()`` on books which has no stored fields so we get back +the ``_id`` and ``_type`` for each document returned and all values are lists: + + >>> list(book_s.values_list()) + [([u'1'], [u'book']), ([u'2'], [u'book']), ([u'3'], [u'book'])] + +``.values_list('id')`` on books, so we get id returned and all values are +lists: + + >>> list(book_s.values_list('id')) + [([1],), ([2],), ([3],)] + +``.values_list()`` on persons which does have stored fields (id and +name, but not weight), so we get the stored fields returned and all +values are lists: + + >>> list(person_s.values_list()) + [([1], [u'Bob']), ([2], [u'Jim']), ([3], [u'Jim Bob'])] + +``.values_list('id')`` on persons which works just like books because +we've specified which fields we want back: + + >>> list(person_s.values_list('id')) + [([1],), ([2],), ([3],)] + + +The same goes for ``.values_dict()``. + + +What you need to do +~~~~~~~~~~~~~~~~~~~ + +1. If you have calls to ``.values_list()`` and ``.values_dict()`` that + don't specify any fields, then you either need to change the + mapping and store the fields you want back, or change the calls so + they specify the fields you want back. + +2. Every time you use results from a ``.values_list()`` or ``.values_dict()`` call, + you need to change it to always treat the values as lists.