github
/
vitess-gh
зеркало из https://github.com/github/vitess-gh.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
9 422 коммитов 104 Ветки 22 Теги 482 MiB
Граф коммитов

1766 Коммитов

Автор SHA1 Сообщение Дата
Dean Yasuda 3e10eec9e2 Merge vtgatev3 into vtgatev2. 
* vtgatev2 will later be named vtgate_gorpc_client.
  * Pass more info to converted exceptions.
  * Clean up lint errors.
 2015-11-11 15:13:23 -08:00
Alain Jobart ac5763e042 Removing deprecated service_map entry. 2015-11-11 10:24:40 -08:00
Anthony Yeh e4ee03146c test/schema: Fix shard_2 setup/teardown. 
The test requires that some steps occur before shard_2 comes up,
so shard_2 has a separate setup function. However, the test was trying
to combine teardown of shard_2 with the rest, which was confusing and
resulted in attempting to init mysql twice without tearing it down.

This splits out teardown of shard_2 so it behaves correctly and is
easier to understand.
 2015-11-10 11:41:30 -08:00
Anthony Yeh b89d988e30 Make MySQL users and privileges configurable. 
This gets rid of the opaque mysql-db-dir.tbz archive, replacing it with
a .sql file. The .sql file approach makes it clear what state the DB is
initialized with, and also makes it easy to customize.
 2015-11-10 11:41:24 -08:00
Alain Jobart 6751c65693 Using enumerate() in a few places where it's appropriate. 
Thanks @aaijazi for the suggestion.
 2015-11-09 14:23:14 -08:00
Alain Jobart bc991f703d RIP gorpc tabletmanager. We have a fully functional gRPC version. 2015-11-09 13:47:41 -08:00
Alain Jobart 35ee67fd9e Adding a test to make sure an interrupted vtctl command works. 2015-11-09 13:14:59 -08:00
Alain Jobart b940f5278f Removing native StreamEvent, replacing it with proto3 version. 2015-11-09 09:23:29 -08:00
Dean Yasuda b3423f99a0 Fix test/resharding.py typo. 2015-11-05 20:31:38 -08:00
Dean Yasuda f8632302bd Merge branch 'master' into pylint 2015-11-05 16:57:23 -08:00
Dean Yasuda b6a7a01efe Use pylint test-mode for fiels in test/. 
Clean some obvious pylint errors in test dir.
 2015-11-05 16:55:30 -08:00
Alain Jobart 6ff9ebd280 Fixing test and comment. 2015-11-05 12:24:30 -08:00
Alain Jobart f41a173e2d Fixing a couple details: 
- grpc_vtctl_client still had deprecated lock_timeout.
- adding a test for vtctld dying on startup.
- fixing a few lint errors.
 2015-11-05 09:18:53 -08:00
Alain Jobart cb00443e17 Merge pull request #1276 from alainjobart/resharding 
Now vttablet will run a RebuildKeyspace in its cell if the SrvKeyspace object doesn't exist
 2015-11-03 16:58:41 -08:00
Anthony Yeh 06a59af397 test/resharding.py: Keep track of master changing. 
The test does a planned reparent, but then later steps assume the old
tablet is still the master. After the reparent, we should update the
test's expectation of who the master is.
 2015-11-03 14:50:03 -08:00
Alain Jobart beb8d4cfbf Now vttablet will run a RebuildKeyspace in its cell if the 
SrvKeyspace object doesn't exist. Fixing an issue with zktopo
that was not returning correct error for missing SrvKeyspace.
Also fixing some (but not all) pylint errors in the files
I touched.
 2015-11-03 11:20:12 -08:00
Alain Jobart 11c40b5442 Fixing binlog end to end test. 2015-11-02 14:58:14 -08:00
Alain Jobart 51bc93a851 Fixing two more end to end tests. 
Also fixing the logs in a test.
 2015-11-02 13:43:39 -08:00
Alain Jobart 024a12186b Fixing resharding.py: 
- vttablet now uses query service to talk to other tablet for health check.
- making all retries and timeouts configurable (using short values in tests).
- doing a single manual health check on source tablets so their health is good.
 2015-11-02 13:28:47 -08:00
Anthony Yeh 8100fbc2d1 Merge pull request #1263 from enisoc/local 
Update local example scripts and docs
 2015-10-29 22:59:36 -07:00
Anthony Yeh 785c99688a Add CI test for local example scripts. 2015-10-29 18:24:31 -07:00
Ammar Aijazi b92e75b4ce Make worker.py test wait longer between resolving the master tablet 2015-10-29 17:14:09 -07:00
Anthony Yeh a05f48ba42 test.go: Add tags for grouping tests. 
The Makefile previously listed tests explicitly for groups like
site_test and worker_test. These lists got out of date when tests were
removed from test/config.json, and the make rules broke. Now the groups
are defined in config.json itself, so there is one place to update
everything.
 2015-10-29 13:27:23 -07:00
dumbunny 5895f58345 Merge pull request #1259 from dumbunny/entity_ids 
Clean up vtgatev2.py after merging entity_ids in.
 2015-10-28 17:34:05 -07:00
Dean Yasuda 8b5fa5e77b Fix vtgatev2._execute logic. 
vtgatev2._execute should use entity_keyspace_id_map if provided,
overriding keyspace_ids / keyranges.

Fix pylint errors.
 2015-10-28 15:00:24 -07:00
Dean Yasuda a9f427d6fa Add sql_builder.NotLike where expr. 2015-10-28 11:59:37 -07:00
Anthony Yeh 0e602ee49a Merge branch 'master' into proto 2015-10-27 21:40:29 -07:00
Alain Jobart e397a2b266 Removing zkns / pdns code. Mostly unused. 2015-10-26 16:09:19 -07:00
Alain Jobart e2812a5e93 Removing ChangeSlaveType's -force option. Extra code for not much, 
can use 'vtctl InitTablet -update' to achieve the same result.
 2015-10-26 09:48:45 -07:00
Sugu Sougoumarane 52f291de7d proto: fix custom_sharding test 2015-10-25 16:49:01 -07:00
Dean Yasuda c079f66739 Merge branch 'master' into entity_ids 2015-10-23 17:24:04 -07:00
Michael Berlin cf87989c51 Change client protocols from BSON RPC to gRPC. 
This commit changes the following protocols:

- binlog_player_protocol
- vtctl_client_protocol

The only BSON protocol left is vtgate pending the implementation of the
gRPC vtgate client.

Note that we originally added this change in
https://github.com/youtube/vitess/pull/1230

However, we reverted it because the Kubernetes tutorial and images were
out of sync. Therefore, this commit technically is the revert of the
revert.

Revert "Revert "Change protocol defaults to grpc.""

This reverts commit 5e5f40a04e.
 2015-10-23 13:46:52 -07:00
Alain Jobart 477c948cc2 Using topo proto3 in tests to get tablet types. 2015-10-23 07:12:49 -07:00
Alain Jobart cff02b5c10 Removing SCHEMA_UPDATE tablet type, unusued. Re-ordering existing 
types in proto3.
 2015-10-23 07:12:49 -07:00
Alain Jobart 9ab82adb35 Fixing overzealous fixes. 2015-10-23 07:12:49 -07:00
Alain Jobart 4954056bdd Fixing a bunch of linter errors. 2015-10-23 07:12:49 -07:00
Alain Jobart e7f8647f00 Fixing initial_sharding.py to delete old master with flag. 2015-10-23 07:12:49 -07:00
Alain Jobart 3de786ba87 Fixing iexternal reparent test to delete the old unhealthy master. 2015-10-23 07:12:49 -07:00
Alain Jobart 73497d2a16 Checkpoint of removing IDLE and SCRAP states. All unit tests pass, 
most integration tests pass too.
 2015-10-23 07:12:49 -07:00
Ammar Aijazi 27c7182b3f Make some fixes for test/worker.py now that we create the db on shard init 2015-10-22 15:52:48 -07:00
Dean Yasuda 2fbbec7995 Merge execute_entity_ids into execute. 
This merges the functions in py/vtdb/{vtgate_cursor,vtgatev2}.py.
 2015-10-22 06:34:33 -07:00
Anthony Yeh 6286868264 Fix schema.py test for new InitShardMaster behavior. 
It creates the db now.
 2015-10-21 16:47:01 -07:00
Anthony Yeh 19cfae8351 backup: Date should be first in backup name. 
We assume the latest backup is the one that's at the end when the list
of directories is sorted. This assumption is violated if we put the
tablet alias first in the name.
 2015-10-20 21:03:30 -07:00
Anthony Yeh 4e74752def Rebalance tests. 2015-10-20 12:15:02 -07:00
Alain Jobart 07fb5b4a8e Fix flakiness in this test: ReloadSchema might be asynchronous. 2015-10-20 11:29:00 -07:00
Alain Jobart a9ec13cd6e Moving vtctld_test to shortest running shard. 2015-10-20 09:47:31 -07:00
Alain Jobart a605eb92d2 Fixing vtctld test, adding it to travis run. 2015-10-20 09:33:54 -07:00
Alain Jobart 50a1db7040 Deflaking this test: if table creation takes time to 
propagate through replication, we were raising an exception.
 2015-10-20 08:36:16 -07:00
Alain Jobart 46701b2590 Fixing flakiness in this test: triggering a ReloadSchema 
much earlier, so schema is there when we try to access
tables.
 2015-10-20 08:23:39 -07:00
Alain Jobart a68b69e32b Fixing reparent.py to work with required sharding key column name. 2015-10-20 08:03:43 -07:00
Alain Jobart 877991915f Moving KeyRange query rule setting to tablet startup. 
It is not expected to change after initial setup.
 2015-10-19 08:23:36 -07:00
Alain Jobart 9284be4938 Merge pull request #1217 from alainjobart/resharding 
Using protocol_flavors in vtgatev2_test.
 2015-10-16 08:51:18 -07:00
Ammar Aijazi 21e934a6a2 Revert "Merge pull request #1215 from youtube/revert-1209-aaijazi_cleanup_error_flag" 
This reverts commit 3ae5b3b510, reversing
changes made to 6860161fcf.
 2015-10-15 17:37:56 -07:00
Alain Jobart 621e6e2276 Using protocol_flavors in vtgatev2_test. 2015-10-15 14:17:43 -07:00
Ammar Aijazi 5ce1f5492e Revert "Remove --rpc-only-in-reply flag from VTGate" 2015-10-14 16:05:49 -07:00
Alain Jobart 99a1901d6e Merge pull request #1208 from alainjobart/resharding 
Removing a couple unused imports.
 2015-10-14 14:01:02 -07:00
sougou 896bc75519 Merge pull request #1211 from youtube/suguwork 
v3 demo: is now self-contained
 2015-10-14 12:30:19 -07:00
Sugu Sougoumarane 573f9f62b2 v3 demo: is now self-contained 
It's been changed to use vttest, and there is a script
that brings up everything, and tears down at the end.
The one problem with the new scheme is that the querylog
doesn't tell you which tablet the query came frome. Something
to think about in the future.
 2015-10-13 21:30:27 -07:00
Ammar Aijazi 79659856ef Java GoRPC SplitQuery correctly reads error response 2015-10-13 19:38:33 -07:00
Ammar Aijazi 3e1361c8b2 Remove --rpc-only-in-reply flag from VTGate 2015-10-13 17:05:33 -07:00
Alain Jobart 4b7c28583b Removing a couple unused imports. 2015-10-13 13:19:30 -07:00
Alain Jobart 37ab2fae49 Merge pull request #1207 from alainjobart/resharding 
Removing python object library, and python direct tablet access.
 2015-10-13 12:36:07 -07:00
Alain Jobart db64050a99 Fixing resharding.py to not try to query a tablet directly, 
but instead use vtctl's VtTabletExecute.
 2015-10-13 08:43:13 -07:00
Alain Jobart 5991aac9b7 Removed toporeader from service map in tests. 2015-10-13 08:14:08 -07:00
Alain Jobart d379eab458 Removing two unused modules, and fixing setup.py. 2015-10-13 07:52:34 -07:00
Alain Jobart 87693af4c7 Removing direct tablet access from python. 
One hack left in vtrouting.py, and then topology.py can go too.
Dean, I'll let you handle that one.
 2015-10-13 07:41:58 -07:00
Alain Jobart 9aaa24c5a0 Removing the object python library, as we decided it's too much 
for us to maintain.
 2015-10-13 07:06:33 -07:00
Sugu Sougoumarane 36276567fa Merge branch 'master' into suguwork 2015-10-12 13:23:15 -07:00
Alain Jobart 8fd2543994 Merge pull request #1201 from alainjobart/resharding 
Switching tabletmanager to the new tabletserver API.
 2015-10-12 13:17:59 -07:00
Sugu Sougoumarane 720ef1f1a8 tests: remove obsolete framework.py 
rowcache_invalidator was the last one using it for MultiDict.
Moved that class into the test itself.
 2015-10-12 13:11:07 -07:00
Alain Jobart f2abbff2a0 Fixing this test: when we go from replica to spare, 
we do not turn off invalidator any more. So removing that test case.
 2015-10-12 12:53:03 -07:00
Alain Jobart 5d0b747661 Adding the 'serving' flag to StreamHealthCheck. 2015-10-12 12:05:43 -07:00
Sugu Sougoumarane 7403a556b8 RIP queryservice_test 2015-10-11 14:26:30 -07:00
Jeff (Zhefu) Jiang dd91fe0473 fix callerid sequence 2015-10-10 19:30:23 -07:00
Jeff (Zhefu) Jiang ac6715a8b1 fix opensource test cases 2015-10-10 19:24:38 -07:00
Sugu Sougoumarane 85e2dd7e18 Merge branch 'master' into suguwork 2015-10-10 17:16:14 -07:00
Sugu Sougoumarane 11273730ff tabletserver endtoend: cache_cases2 2015-10-10 16:47:44 -07:00
Sugu Sougoumarane 519819fc62 tabletserver endtoend: cache_cases1 2015-10-10 16:21:43 -07:00
Sugu Sougoumarane 3553772c61 tabletserver endtoend: nocache cases 
I've written a code generator that converts the python cases
into go. Some massage is needed after the conversion.
The downside is that it's a brainless port. No re-interpretation,
regrouping or gap analysis.
 2015-10-10 14:02:20 -07:00
Alain Jobart 965140c42f R.I.P. vtocc. 2015-10-09 16:01:33 -07:00
Alain Jobart c144bba292 Adding a unit test of the vttest setup in vttest_sample_test.py. 
This requires using the protocol from flavors.
Also using ValueError exception instead of plain Exception
in vtgate_client.py.
 2015-10-09 13:47:41 -07:00
sougou 2f5d097d7a Merge pull request #1192 from youtube/suguwork 
tabletserver endtoend: most cache tests done
 2015-10-09 10:39:59 -07:00
Sugu Sougoumarane c3b2901e2c tabletserver endtoend: most cache tests done. 2015-10-09 00:02:02 -07:00
Sugu Sougoumarane 88da603501 tabletserver endtoend: a few cache tests 2015-10-08 18:59:02 -07:00
Alain Jobart ffa5c9bd26 Removing vtocc+vtgate support in vttest, now only does vtcombo. 2015-10-08 07:31:09 -07:00
Anthony Yeh bc84f82666 test/config.json: rebalance test shards 2015-10-06 17:49:41 -07:00
Sugu Sougoumarane a28b120196 tabletserver endtoend: nocache_tests all done 2015-10-05 20:56:54 -07:00
Sugu Sougoumarane 971f5a94c4 tabletserver endtoend: health check 
Also changed DebugVars fetch delimiter to be '/' because some tags
may contain '.'
 2015-10-05 16:43:18 -07:00
Sugu Sougoumarane 7c99b152ed tabletserver endtoend: more tests migrated 
TypeLimits, BindInSelect, QueryRules
 2015-10-04 18:53:24 -07:00
Sugu Sougoumarane 5c84087b0c tabletserver endtoend: ExecuteBatch tests 2015-10-04 11:36:29 -07:00
Sugu Sougoumarane d51a37779b tabletserver endtoend: consoildation test 
The race detector found a bug in the QueryExecutor. It probably
hasn't failed on production because the racy code paths were always
trying to set the variable to the same value.
 2015-10-03 19:26:23 -07:00
Sugu Sougoumarane f523f628c4 tabletserver endtoend: config tests done 2015-10-02 22:08:02 -07:00
Sugu Sougoumarane fac3a448ad tabletserver: deflake some tests 2015-10-02 17:23:49 -07:00
Sugu Sougoumarane 37d004837f tabletserver: test cleanup 
remove tests that have become obsolete so far.
I'm also removing the corresponding SET functionality support.
 2015-10-02 16:15:32 -07:00
Ammar Aijazi 71127afcf8 Disable some tests that don't work for all clients 2015-10-01 18:12:50 -07:00
sougou b43a7c4419 Merge pull request #1151 from youtube/suguwork 
tabletserver endtoend: WIP
 2015-10-01 10:17:26 -07:00
Ammar Aijazi 5c65d6073a Merge pull request #1150 from aaijazi/aaijazi_stop_parsing_errors_in_python_client 
Stop parsing errors in python vtgate client
 2015-09-30 16:28:17 -07:00
Sugu Sougoumarane 355bed457f tabletserver: reorg some timeouts 
queryTimeout moved from QueryEngine to TabletServer.
poolTimeout moved from TxPool to TabletServer, and renamed to
beginTimeout.
 2015-09-30 15:30:10 -07:00
Anthony Yeh d2a5aff177 healthcheck: Re-read tablet after going SPARE when shutting down. 
We were using ChangeType to go SPARE, but we were then using the
outdated tablet record to do a point update in the serving graph.
 2015-09-30 15:24:55 -07:00
Ammar Aijazi e158a99f06 Add tests for vtgate_utils.extract_rpc_error 2015-09-30 15:13:16 -07:00
Ammar Aijazi d96f036b7c Addressing code review comments to fix minor style issues 2015-09-30 15:03:19 -07:00
Ammar Aijazi b839417d08 Set rpc-error-only-in-reply=False for Java tests 2015-09-29 17:19:41 -07:00
Ammar Aijazi 3b6b0276a8 Make vtgatev2 client extract errors from RPC response, and stop string parsing those errors 2015-09-29 16:56:58 -07:00
Ammar Aijazi 591a3d065a Add python client tests for transient errors 2015-09-29 13:59:37 -07:00
Ammar Aijazi a7916b9d28 Refactor python_client_tests.py to split error test cases into their own class. 2015-09-28 18:25:59 -07:00
Ammar Aijazi 6075687096 vtgateclienttest: add support for forcing errors in Begin, Commit, Rollback using callerid 2015-09-25 11:55:27 -07:00
Ammar Aijazi cabdd2c55b Add support for partial errors in vtgateclienttest errors service 2015-09-25 11:55:27 -07:00
Ammar Aijazi 8142cf80fa Support SteamExecute methods in vtgateclienttest errors service 2015-09-25 11:55:27 -07:00
Ammar Aijazi 10f6a6d725 Add more errors, with error codes, to vtgateclienttest 2015-09-25 11:55:27 -07:00
Alain Jobart 67bcb09b87 Fixing a few vtcombo-related things: 
- now rebuild serving graph in vtcombo to allow range queries.
- adding vtctl VtGateExecuteKeyspaceIds command.
- added an 's' to vtctl VtGateExecuteShard to be consistent.
- pass in an empty Session to Begin in vtcombo, otherwise panics.
 2015-09-24 10:44:41 -07:00
Ammar Aijazi 80f7c7072a Merge branch 'master' into aaijazi_single_gitignore 2015-09-23 17:23:40 -07:00
Ammar Aijazi 4cba9d02c1 vtdb.tablet client uses error codes to return errors instead of string parsing 2015-09-23 15:58:59 -07:00
Alain Jobart da0f6e58ef Fixed parameter I removed. 2015-09-23 13:10:26 -07:00
Alain Jobart d3c42c0941 Adding support for vtcombo in vttest framework. 2015-09-23 13:10:26 -07:00
Alain Jobart 8efc861504 WIP on vtcombo: queries now work, if they don't need a db name. 2015-09-23 13:10:26 -07:00
Ammar Aijazi 1fbe779d56 Merge branch 'master' into aaijazi_single_gitignore 2015-09-23 12:34:52 -07:00
Erez Louidor 5f295323e5 ; 2015-09-23 10:39:17 -07:00
Erez Louidor fc4dc2f7be ; 2015-09-23 10:35:51 -07:00
Erez Louidor cdf8b42ae5 ; 2015-09-23 10:32:56 -07:00
Erez Louidor 0b6b29f9d2 Merge remote-tracking branch 'upstream/master' 2015-09-23 10:32:39 -07:00
Erez Louidor 0e36c01223 Merge remote-tracking branch 'upstream/master' 2015-09-22 18:35:51 -07:00
Erez Louidor fbfdb57799 Addressed comments 2015-09-22 17:48:39 -07:00
Anthony Yeh 31087ab447 Rebalance test shards. 2015-09-22 17:19:55 -07:00
Dean Yasuda fb8d99d1af Merge branch 'master' into batch 2015-09-22 11:26:10 -07:00
Alain Jobart 4a86e49b31 Fixing this test after the SplitQuery change. 2015-09-21 12:17:09 -07:00
Dean Yasuda 604cd01259 Support ExecuteBatchShard in vtgatev2.py. 
Change the vtgateclienttest/services/callerid.go test to always return
an error when callerid is handled. The error will contain word
"SUCCESS:" if callerid matches. This is necessary since falling through
breaks the expectations of the caller.

In the connection's _execute_batch call, some routing will be
keyspace_ids, and some will be by shards. This means that an
_execute_batch may create both a ExecuteBatchKeyspaceIds and
ExecuteBatchShard call. Merge the results back together.

Add tests for ExecuteBatchShard in python_client_test.py and
vtgatev2_test.py.
 2015-09-18 16:10:39 -07:00
Ammar Aijazi 53be99f0a6 Get rid of directory-specific .gitignores in favor of a single global .gitignore 2015-09-17 15:48:55 -07:00
Dean Yasuda 8140617001 Combine vtgate_cursor.BatchVTGateCursor. 
Since BatchVTGateCursor only adds executemany and nextset functionality,
just add that functionality to vtgate_cursor.VTGateCursor. Fix tests and
code that refers to BatchVTGateCursor.
 2015-09-17 08:55:31 -07:00
Dean Yasuda 4d8cb66908 Add BatchVTGateCursor.executemany. 
To comply with PEP0249, remove the BatchVTGateCursor execute and flush
sequence and use executemany instead. The first argument to executemany
is sql if all commands share a common sql (this is supposed to be an
"operation" parameter). The params_list will supply sql if the first
argument is None.

Each params in the executeany params_list generates a result set.
Implement the nextset() to walk through the result_set.

Update tests to use the new BatchVTGateCursor API.
 2015-09-17 03:35:50 -07:00
Dean Yasuda 9d7614ac4f Merge branch 'master' into batch 2015-09-15 14:31:29 -07:00
Dean Yasuda 0343bd6ba4 Fix potential vtgatev2 stream cursor bug. 
Always open a new connection for every stream_execute. Reusing an
existing connection may cause problems if you are in a transaction; this
logic is simpler.

Also, change the cursor._get_conn() method into a cursor.connection property.
 2015-09-15 14:27:48 -07:00
Sugu Sougoumarane 59a13c7800 Merge branch 'master' into suguwork 2015-09-15 13:50:43 -07:00
Alain Jobart 8a5a3d8b1d Merge branch 'master' into replication 2015-09-14 14:48:11 -07:00
Alain Jobart bf04192ed2 Addressing comments. 2015-09-14 14:47:26 -07:00
Alain Jobart bba69d34fc Adding sandbox. 2015-09-14 08:57:26 -07:00
Dean Yasuda 2753ed386c Merge branch 'master' into callerid4 2015-09-14 08:48:40 -07:00
Alain Jobart 982db28803 First version of this doc. 2015-09-14 08:44:33 -07:00
Sugu Sougoumarane 3b3584d54d Merge branch 'master' into suguwork 2015-09-14 08:00:55 -07:00
Dean Yasuda d11c5ec854 Remove cursor effective_caller_id parameters. 
Only cursor.set_effective_caller_id(effective_caller_id) is now
supported. Sending effective_caller_id as an argument to PEP0249
functions will fail or be ignored.
 2015-09-14 05:34:02 -07:00
Alain Jobart 13fc216b65 Fixing failing tests. 2015-09-13 19:16:40 -07:00
Alain Jobart 838d75f7fb Merge branch 'master' into replication 2015-09-13 17:57:14 -07:00
Alain Jobart 02fa09fb8a Fixing bson python client for srvkeyspace transition. 2015-09-13 17:56:25 -07:00
Dean Yasuda 715fc9c671 Fix some lint errors, missing init. 2015-09-13 04:07:20 -07:00
Dean Yasuda 0d7f6a8b70 Merge branch 'master' into stream 2015-09-12 17:44:03 -07:00
Dean Yasuda faa72fc805 Python cursor changes. 
In base cursor classes, Make set_effective_caller_id a separate call
rather than passing effective_caller_id through interfaces (cleanup to
come later).

Remove some try-except log-and-raise wrappers in vtgatev2 test methods.
The default logging, and put the logging in get_connection instead.
 2015-09-12 17:15:32 -07:00
Sugu Sougoumarane d82ce0059f Add vtgatev3 to integration tests 2015-09-11 20:35:53 -07:00
Dean Yasuda 5bb9a4bc6b Allow multiple vtgate stream_execute calls. 
Vtgate connection stream_execute now detaches the BsonRPCClient from the
connection and closes it after the generator exits. The connection will
quietly open a new BsonRPCClient if another execute is called.
 2015-09-11 15:18:46 -07:00
Alain Jobart fe2bcecfb2 Fixing a couple exceptions. 2015-09-11 13:13:39 -07:00
Alain Jobart 80ca29cedc Small clean-up. 2015-09-11 12:50:18 -07:00
Alain Jobart 318af78ad6 Switching to beta versions of protobuf and grpc. 
Fixing python clients to work.
 2015-09-11 11:55:54 -07:00
Dean Yasuda 2803385021 Reorganize python base cursor classes. 
Add more tests to vtgatev2_test.py. Try to make vtgatev2 able to
break out of a stream_execute and restart. This needs more work.
 2015-09-11 06:37:44 -07:00
Dean Yasuda bb3c38e16c Fix test traceback.print_exc messages. 
Change traceback.print_exc to traceback.format_exc.  Traceback.print_exc
writes to stdout, while traceback.format_exc returns a string.
 2015-09-08 21:43:45 -07:00
Dean Yasuda e55bb14d67 Add py/vtdb/base_cursor.py to consolidate cursors. 
Move some common code into base_cursor.py.

Make the third_party StreamCursor._stream_execute return generators and
migrate stage variables out of the vitess clients.
 2015-09-07 18:25:12 -07:00
Michael Berlin 312b7a6d3a Merge pull request #1074 from michael-berlin/export_worker_test_larger_inserts 
Exporting internal changes back to open-source.
 2015-09-02 18:09:10 -07:00
Michael Berlin 6ce9973341 Exporting internal changes back to open-source. 
NOTE: Changes were already LGTM'd internally.
 2015-09-02 17:55:17 -07:00
Ammar Aijazi 5e00421dab Revert the remove of the --rpc-error-only-in-reply flag for Gorpc VtGate, as there are clients that depend on it for now 2015-09-02 17:28:21 -07:00
Alain Jobart 56cd05b8ed Fixing last test to pass. 2015-09-01 17:18:05 -07:00
Alain Jobart 371151271d Fixing this test. Running into other issues, so it doesn't pass yet. 2015-09-01 15:01:34 -07:00
Alain Jobart 1017ce6cfc Removing SASL over BSON support. 2015-09-01 09:15:37 -07:00
Alain Jobart 62da287c77 Not authorizing any bson protocol by default. 2015-09-01 08:42:08 -07:00
Michael Berlin d512d27a5d Merge pull request #1046 from michael-berlin/unit_test_runner 
travis: Custom runner for Go unit tests.
 2015-08-31 14:05:38 -07:00
Michael Berlin 168588e279 Detect flaky unit tests based on their suffix _flaky_test.go. 
Renamed all flaky unit tests.

Removed previous blacklist file because it's no longer necessary.

Moved custom unit test runner from travis/ to tools/.
 2015-08-30 22:28:14 -07:00
Dean Yasuda 91318ff64d Fix a few more python lint errors. 2015-08-30 22:03:22 -07:00
Michael Berlin e865716269 Merge branch 'master' into unit_test_runner 2015-08-30 21:23:05 -07:00
Dean Yasuda 86215c7b2d Fix docstrings and var names in test/protocols_flavor.py. 2015-08-30 20:22:16 -07:00
Dean Yasuda 452db673d9 Fix error from previous commit in test/queryservice_test.py. 2015-08-30 10:54:24 -07:00
Dean Yasuda f636e15166 Second pass of python lint fixes. 
This is the first pass for some subdirectories.

Recurring issues include:

1. Inconsistent quote chars. Use the same quote character as the first
use in the file (usually single-quote). Double quotes are OK if the
quoted string contains single quotes.

2. Bad indentation. Indent 2 for logic, 4 for continuations if the
previous line ends with '(', '[', or '{'. If drop does not immediately
follow the group-symbol, align with the opening group-symbol. I suggest
immediate drops, since these are stable if the content of the first
line changes, while aligning with the opening group-symbol requires
shifting the continuation lines.

3. Bad docstrings. A full docstring looks like:

   """Initial line ending with punctuation.

   Optional text.

   Args:
     Arg1: Description.
     Arg2: Description with
         indented continuation.

   Returns:
     Return text.

   Raises:
     ErrorClass: Description.
   """

4. Vertical spacing. Exactly two blank lines before top-level classes
or functions. Exactly one blank line before non-top-level classes or
functions. No more than one blank line within top-level constructs.
 2015-08-30 09:32:00 -07:00
Anthony Yeh 5434ab0a2c Deflake vtgatev2_test. 
Some of the tests connect to vtgate, then kill the tablet,
and execute a vtgate call that's expected to fail. Out of those,
some of them then go on to try to continue using the vtgate conn.

The problem is that depending on how long it takes to fail, the
first vtgate call might end up hitting a client-side timeout,
which closes the vtgate conn. In the case of gRPC, there is a
set of non-configurable reconnection attempts, which means the
calls can take a while to fail (based on the server-side timeout).

This change increases the client-side timeout for those tests,
to avoid racing with the server-side timeout.
 2015-08-28 17:43:25 -07:00
Michael Berlin 3bd6db500d Merge branch 'master' into auto_add_MigrateServedTypes 2015-08-28 13:08:50 -07:00
Dean Yasuda 4c61dd2f3f In sql_builder, rename Or to OrExprs, add AfterPrevValues. 
The rename makes it clear that this is to OR non-trivial statements
for a single column together, not a general purpose OR, and not to
be used when a simple IN statement could be used.

AfterPrevValues is used for multi-dimensional, ordered traversal after
a continuation point.
 2015-08-27 12:10:04 -07:00
Michael Berlin bc13841f76 Revert "Proof of concept for separating flaky Go unit tests using "go test -run <regex>"." 
This reverts commit b382512fc3.
 2015-08-26 21:38:40 -07:00
Michael Berlin b382512fc3 Proof of concept for separating flaky Go unit tests using "go test -run <regex>". 2015-08-26 20:58:01 -07:00
dumbunny f99b2c9af2 Merge pull request #1048 from dumbunny/callerid3 
Make vtgate_client.CallerID object to provide a common object to pass
 2015-08-26 09:45:56 -07:00
Alain Jobart eb99e38778 Couple changes to make run_local_database more portable. 2015-08-26 07:56:47 -07:00
Dean Yasuda 4958abb860 Make vtgate_client.CallerID object to provide a common object to pass 
to client methods.
 2015-08-26 07:36:14 -07:00
Alain Jobart bfe056ba37 Fixing comments. 2015-08-26 07:15:02 -07:00
Alain Jobart 4c12c4c7da Merge branch 'master' into resharding 2015-08-26 07:06:57 -07:00
Michael Berlin a8c3103d31 test.go: Allow tests to specify a RetryMax value. 
Use travis/unit_test_runner.sh to run the Go unit tests.
 2015-08-26 00:09:18 -07:00
Dean Yasuda fe15c10907 Fix remaining syntax errors caused by previous test/*.py lint fix. 2015-08-25 23:21:32 -07:00
Dean Yasuda 8be90a9cce Fix some more obvious lint errors in test/*.py. 2015-08-25 22:45:44 -07:00
Dean Yasuda 09a23591a9 Post-merge: fix some simple lint issues on more test/*.py files. 2015-08-25 21:00:52 -07:00
Dean Yasuda 3c9949f75e Merge branch 'master' into lint1 2015-08-25 19:01:33 -07:00
Dean Yasuda 6f1e821634 Make some lint fixes to test/*.py. 2015-08-25 18:54:04 -07:00
Michael Berlin a87b019def test: Capitalize class constant. 2015-08-25 17:42:03 -07:00
Michael Berlin 93b4b3b7b9 test: Log keyspace, shard and tablet type when utils.check_tablet_query_service() fails. 2015-08-25 16:32:19 -07:00
Michael Berlin f9e68df8a5 automation: Add task for vtctl MigrateServedTypes. 
Extended end-to-end test to verify the new task.
 2015-08-25 16:32:09 -07:00
Alain Jobart a3f18d5dc9 Adding unit test code. 2015-08-25 16:27:31 -07:00
Alain Jobart 543d5f2c72 Adding integration test for vttest. 2015-08-25 16:20:23 -07:00
Shengzhe Yao c4ff493b5b stop using hard coded caller id in tablet.py 
1. Stop using youtube specific name as caller id.
2. Set caller id as TabletConnection field and use it in the remaining methods.
3. Delete _stream_execute2 in tablet.py since it is now identical to _stream_execute.
 2015-08-25 11:40:48 -07:00
Alain Jobart c8330abe55 Also passing correct prototol to vtworker. 2015-08-24 18:29:58 -07:00
Alain Jobart 9619fdafed Merge pull request #1035 from alainjobart/resharding 
Passing the right protocol flavor to vtctld for tablet connections.
 2015-08-24 18:23:42 -07:00
Alain Jobart 382696b67a Passing the rigth protocol flavor to vtctld for tablet connections. 2015-08-24 18:04:00 -07:00
Anthony Yeh 715c660104 test.go: Don't use bash for launching non-Docker tests. 
I can't get signals to work on the child processes launched by bash,
even if I use a helper script that explicitly signals its children.
That meant that test timeouts didn't actually terminate the test in
non-Docker mode.

We were only using bash to split the command line into args anyway.
Instead, let's just have the config give us the args pre-split.
Sending signals without bash in the way seems to work better.
 2015-08-24 13:44:52 -07:00
Anthony Yeh 7ef6c48c0a Merge branch 'master' into php-bson 2015-08-23 20:48:08 -07:00
Anthony Yeh 1ac3d8166a Use test.go for Travis and "make test". 2015-08-23 19:51:42 -07:00
Anthony Yeh e02629932a test.go: Record stats for tests. 2015-08-23 17:43:41 -07:00
Anthony Yeh 082f9afd6d test.go: Add goveralls script as a manual target. 2015-08-23 17:43:41 -07:00
Dean Yasuda 59278ea991 Test that effective_caller_id is passed to VTGate.StreamExecute interfaces. 
Send both good and bad effective_caller_ids through the interfaces to make sure
that verification is actually done.
 2015-08-23 13:33:47 -07:00
Dean Yasuda 818e34527a Fix test/vertical_split tearDownModule bug. 2015-08-22 20:40:58 -07:00
Dean Yasuda a5715c19e0 Propagate effective_caller_id parameter from vtgate_cursor to 
vtgate_connection to vtgate.
 2015-08-22 19:40:51 -07:00
Anthony Yeh 83bc3849ef Add PHP tests to Docker-based test runner (test.go). 2015-08-22 17:49:09 -07:00
Anthony Yeh e830e28dea Make grpc the default for Go binaries talking to queryservice. 
Since go/bson no longer encodes the non-standard uint64 type,
it's not safe to send uint64 over bsonrpc for a field that is
unmarshaled into interface{}, since it will be unmarshaled as int64.

The only case where this happens is bind vars. We could fix bsonrpc
by switching it to use bson-encoded proto3 structs, since those
use concrete field types for bind vars rather than interface{}.

However, bsonrpc is deprecated anyway so instead of fixing it,
we will just switch to grpc for all clients written in Go that
talk to queryservice.
 2015-08-22 16:01:45 -07:00
Dean Yasuda 88edcc204c Merge master changes from youtube/vitess. 2015-08-22 15:57:14 -07:00
Michael Berlin df8457b9df test: Re-adding topo server setup. 
I accidentally removed it here: https://github.com/youtube/vitess/pull/1020

Original commit was:

commit ddbbca247a
Author: Alain Jobart <alainjobart@google.com>
Date:   Thu Aug 20 12:20:56 2015 -0700

    Adding topo server setup to this test, required in some environments.
 2015-08-21 17:33:27 -07:00
Michael Berlin b937dc8a7b Revert recent changes by Dean which were accidentally pushed to the master without review first. 2015-08-21 16:26:20 -07:00
Alain Jobart cdf3a7bb2f Adding utils.py option to skip builds, using it in the Makefile 
(after adding a rule to build before tests).

Global build takes about 2s to build all binaries, and it saves
about 4s in one run of mysqlctl.py, so it's gonna be good!
 2015-08-21 09:51:08 -07:00
Anthony Yeh c664947500 Fix python_client_test for new callerid prefix. 2015-08-20 14:26:47 -07:00
Dean Yasuda 5fa76ff6f5 Merge remote-tracking branch 'upstream/master' 2015-08-20 12:45:05 -07:00
Alain Jobart ddbbca247a Adding topo server setup to this test, required in some environments. 2015-08-20 12:20:56 -07:00
Dean Yasuda 232c6a870d Change Makefile to add individual integration test targets. 
Hook up effective_caller_id and add a simple test in python_client_test.py.
 2015-08-20 11:57:33 -07:00
sougou 1393f52249 Merge pull request #1006 from youtube/suguwork 
tabletserver: INSERT IGNORE support
 2015-08-18 23:11:51 -07:00
Michael Berlin 411f518d72 Merge branch 'master' into wait_for_filtered_replication 2015-08-18 14:14:04 -07:00
Alain Jobart a0ab9f5c48 Merge pull request #1004 from alainjobart/resharding 
Resharding
 2015-08-18 14:12:05 -07:00
Sugu Sougoumarane faa84e2afc Merge branch 'suguwork2' into suguwork 2015-08-18 13:59:33 -07:00
Sugu Sougoumarane 3685adf293 Merge branch 'master' into suguwork 2015-08-18 13:30:56 -07:00
Sugu Sougoumarane 1e37ca1a68 tabletserver: INSERT IGNORE support 
This feature is needed for improving the usability of
ExecuteBatch. People may want a dup key error from insert
to be ignored so that they can follow it with other statements.
In the implementation, there's one caveat I had to handle, which
is that the ignore should be removed if it's an upsert. Otherwise,
we never get the dup key errror.
 2015-08-18 13:22:24 -07:00
Michael Berlin 66146f631f vtctl: Revert that "master" is allowed as type for RunHealthCheck. 
A tablet is never started with -tablet_type master and therefore the manual health check shouldn't support this as well. Nonetheless, any "replica" can become a "master".
 2015-08-18 13:12:45 -07:00
Alain Jobart 81571296b8 Making vertical_split.py always and only work with vtgate. 2015-08-18 08:09:56 -07:00
Alain Jobart 1f45dd97a9 Merge branch 'master' into resharding 2015-08-18 07:06:45 -07:00
Michael Berlin 0d7f7a0f1a test: Verify that QueryService.StreamHealth returns correct stats of the filtered replication. 
We do so by comparing the BinlogPlayer vars against the output of vtctl VtTabletStreamHealth.
 2015-08-17 18:58:41 -07:00
Michael Berlin f1e25c869b test: Remove unused imports from tabletmanager.py. 2015-08-17 18:51:16 -07:00
Shengzhe 230f6b1f8e Merge pull request #993 from yaoshengzhe/fix_support_string_column_in_splitquery 
make SplitQuery support string column.
 2015-08-17 17:19:34 -07:00
Alain Jobart b50890c915 Removing leftover file. 2015-08-17 15:51:33 -07:00
Alain Jobart 6d36cd24c4 update_stream.py now uses only vtctl / vtgate. 
Adding missing APIs to vtctl/s query.go.
 2015-08-17 15:49:12 -07:00
Shengzhe Yao 3736372fee fix splitquery related tests. 
1. bind variables in SplitQuery should not contain leading ':'.
2. fix custom_sharding.py test to pass through returned bind variables from VTGate.
3. fix SplitQuery tests in Java.
 2015-08-17 15:36:07 -07:00
Michael Berlin fe3dde842d test: Replace assertTrue(member in collection) constructs with assertIn(member, collection). 
assertIn has a nicer error formatting.
 2015-08-17 13:09:47 -07:00
Alain Jobart 08d6933e23 Fixing this test after my hook change. 2015-08-17 11:59:29 -07:00
Alain Jobart 1251842c04 Fixing rowcache_invalidator.py to not depend on zkocc nor sleep. 2015-08-17 11:34:25 -07:00
Alain Jobart d172b1e637 Refactoring rowcache_invalidator.py to not use python vttablet client. 2015-08-17 09:28:53 -07:00
Alain Jobart 78d5dc67f0 This test is not using zkocc any more. 2015-08-17 08:42:56 -07:00
Alain Jobart ca77d9a5d2 Retiring zkclient2. toposerver API is going away. 2015-08-17 08:34:18 -07:00
Alain Jobart 9df8070ac2 Removing jscfg package. It wasn't doing much, except 
using panics to return errors.
 2015-08-17 08:13:58 -07:00
Sugu Sougoumarane f788a6f1d7 tabletserver: upsert disallow values function 
This is a more correct fix for disallowing values function.
I've verified that this function is only supported for the
upsert construct:
https://dev.mysql.com/doc/refman/5.5/en/miscellaneous-functions.html#function_values.
So, it's safe to disallow this at the parser level, just as we
do for last_insert_id.
 2015-08-15 15:14:10 -07:00
Sugu Sougoumarane 4aba6f3b7a tabletserver: upsert handles dup key error better 
Found a good way to know which index caused a dup key error.
The error message itself contains that info. This is better than
looking at RowsAffected because there are other conditions that cause
that field to be 0.
 2015-08-14 23:02:44 -07:00
Sugu Sougoumarane d3fc76999f tabletserver: upsert WIP 
Upsert is mostly feature complete. There are still two
issues to resolve:
1. RowsAffected is returned as 0 if no rows were modified. So, we cannot
assume that no rows were matched. So, the value cannot be used to verify
if the dup key matched a pk or a unique key.
2. I mistooke VALUES to be VALUE, and coded defense against using that
construct in the parser. It turns out that the parser already allows use
of VALUES. So, I'll have to find a different way to prevent that usage
for upserts.
 2015-08-14 13:23:33 -07:00
Anthony Yeh eb5141992f Add Java tests to Docker test runner. 2015-08-12 01:56:37 -07:00
Michael Berlin 4daa9c6c56 automation: Extend Task interface to support optional parameters. 
Make parameter validation more strict: Unknown parameters are no longer
supported.

Fixed minor issues e.g. error messages must not be capitalized.
 2015-08-11 13:17:48 -07:00
Alain Jobart dcd7039b9f Fixing this test. 2015-08-11 09:28:47 -07:00
Alain Jobart a79192c887 Fixing vtctld_test to pass. 2015-08-10 12:59:12 -07:00
Alain Jobart 26210aa7ce Fixing a bunch of things. Most integration tests now pass. 2015-08-10 09:33:07 -07:00
Alain Jobart 8fae3c7c30 Merge pull request #972 from alainjobart/resharding 
Adding integration test to make sure master health is correct
 2015-08-07 13:23:57 -07:00
Alain Jobart ad4dcbd726 Replacing all python %u by %d. Apparently it's what we need to do now. 2015-08-07 12:42:36 -07:00
Alain Jobart 612d4a0d6c Re-applying fixes from: 
48808a8003
wrt tablets dying at startup being detected faster.
 2015-08-07 12:25:11 -07:00
Alain Jobart fcc8d71570 Merge branch 'master' into resharding 2015-08-07 12:15:26 -07:00
Alain Jobart 067bbcbb9a Fixing this test to work with grpc. 2015-08-07 12:00:08 -07:00
Alain Jobart b3b24d62f1 Fixing flaky tests. 
With N being integer:
base_time = N.2
terTime = N.4 rounded to N
so terTime < base_time, even though it's not.
 2015-08-07 11:11:54 -07:00
Alain Jobart 43efd74c43 Adding integration test to make sure master health is correct 
after external reparent.
 2015-08-07 09:41:16 -07:00
Alain Jobart 3b14f54fd2 Now using proto's SrvShard. 2015-08-05 08:00:28 -07:00
Alain Jobart be25f8949a Few fixes from review. 2015-08-04 17:12:36 -07:00
Alain Jobart f1b1c1ea18 Now using proto3's Shard record. 2015-08-04 12:44:19 -07:00