Граф коммитов

5840 Коммитов

Автор SHA1 Сообщение Дата
Liang Guo 0d4b8fe87b Merge branch 'master' into status 2016-05-05 14:22:07 -07:00
Liang Guo b5196b6a96 Fix end2end vttest. 2016-05-05 14:19:16 -07:00
Anthony Yeh 4767c832d0 Merge pull request #1680 from enisoc/sqlstate
java: Populate MySQL errno and SQLSTATE in SQLExceptions
2016-05-05 11:03:29 -07:00
Alain Jobart c96850635f Tabletserver doesn't generate sessionid any more. 2016-05-05 07:25:28 -07:00
Alain Jobart a0c8f9e8e8 Removing GetSessionId from queryservice.
And session_id from RPC calls. We switched to target everywhere now.
2016-05-05 07:25:28 -07:00
Anthony Yeh f6d2c0dfc2 Transmit SQLSTATE error info from MySQL client to Vitess clients. 2016-05-04 21:03:05 -07:00
Liang Guo 71838711e8 Merge branch 'master' into status 2016-05-04 18:44:17 -07:00
Liang Guo 48e7708c6d Delete shard gateway/conn related code, and also fix unit tests to use discoverygateway. 2016-05-04 15:20:36 -07:00
Michael Berlin 9135fdf38d Merge pull request #1677 from michael-berlin/throttler_vtworker_plumbing
worker: Support fixed rate throttling for SplitClone and VerticalSplitClone
2016-05-04 14:20:59 -07:00
Alain Jobart 9198c66920 Merge pull request #1675 from alainjobart/testbegincommit
Adding unit test for BeginExecute{,Batch}. Removing tabletconn.Cancelled.
2016-05-03 10:20:20 -07:00
Michael Berlin f43d6375ee worker: Add -max_tps flag to SplitClone and VerticalSplitClone.
The flag controls the throttling of the copy.

By default, throttling is disabled.

Any throttling rate will be evenly distributed across all writer threads.
2016-05-03 09:53:50 -07:00
Michael Berlin a739b89f65 throttler: If maxRate is below threadCount, increase it to threadCount.
- Added magic constant ZeroRateNoProgess to better reflect in the code that maxRate == 0 means that the Throttler won't let anything through.
2016-05-03 09:18:15 -07:00
Michael Berlin d68d5ec3dc throttler: Do not allow more than one request per request interval for rates below 10k.
- Added more benchmark configurations with different QPS rates.
- Updated the example output from the benchmark.
2016-05-03 09:17:59 -07:00
Michael Berlin 0809025d1e vtworker: Move execution state into "executor" struct.
This way, we don't have to pass all required parameters between functions all the time.
2016-05-02 14:49:34 -07:00
Michael Berlin 7c1bdf4999 throttler: First version which supports throttling by a fixed rate across multiple threads.
Next steps (outside of this PR) will be:
- use Throttler in vtworker and filtered replication
- implement MaxReplicationLag module
- implement a read-only webinterface which displays the actual and throttled rate
- make the webinterface read-write e.g. to override the fixed rate limit
2016-05-02 12:08:55 -07:00
Alain Jobart 6b0b84421b Removing tabletconn.Canceled.
Instead, we just use context.Canceled. That way if it's triggered in
some other case, we're still handling it right.
2016-05-02 11:24:47 -07:00
Alain Jobart a087f4c008 Adding unit test for BeginExecute{,Batch}.
In the discovery gateway code.
2016-05-02 11:04:35 -07:00
Michael Berlin a2a0d900ca worker: Resolve destination master using the discovery module.
- Removed previous Resolver interface and respective implementations.
- Removed respective stat vars and flags (--resolve_ttl).

- Added unit test for case where vtworker fails over to a different replica.
- Added unit test for case when healthcheck retries because currently no master is available.
- Extended FakePoolConnection to support these tests:
  - can define callback (AfterFunc) when expected query was received
  - infinite mode where the last request may be received over and over again

- discovery: Introduce DefaultTopoReadConcurrency to avoid duplication.
- discovery: Added EndPointStats.Alias() to avoid duplication.
- discovery: Added EndPointStats.String() to have pretty printed arrays.

Adapted end-to-end tests:
- binlog.py, merge_sharding.py, resharding.py: Enabled healthcheck for master tablets.
- worker.py: Updated stat vars check
2016-04-29 22:58:00 -07:00
Michael Berlin 0cb22ce7e8 stats: Fix typo. 2016-04-29 18:40:34 -07:00
Michael Berlin 61926ba79b vtgate: Fix wrong call to log.Warning() where log.Warningf() was necessary. 2016-04-29 18:25:45 -07:00
Michael Berlin bc6938bcd3 worker: Refactored FakePoolConnection.
- Moved out into own file.
- More code reuse between tests.
- Split split_clone_test.go into multiple tests to better sepearate between a test which tests the normal case and tests which test specific error situations.
2016-04-29 15:18:52 -07:00
Michael Berlin 2c148d7661 tabletserver: Move QueryService fakes into subpackage.
Moved StreamHealthQueryService fake out of wait_for_drain_test.go and use it in other tests as well.
2016-04-29 13:40:04 -07:00
Michael Berlin 1dfbac6b55 Merge pull request #1661 from michael-berlin/discovery_filter_helpers
discovery: Provide filter helper functions to reduce code duplication.
2016-04-29 13:34:42 -07:00
Michael Berlin f2c9937aa7 discovery: Added package comment. 2016-04-29 11:49:45 -07:00
Michael Berlin 1f218bcf82 discovery: GetCurrentMaster(): Return nil instead of an empty list. 2016-04-29 11:49:38 -07:00
Sugu Sougoumarane 9eb47e2de2 bug fix: code improvement
Now that we don't mix mysql flags with MySQL types, there's
no need to bit-shift them. Makes the code more readable.
2016-04-28 16:31:02 -07:00
Sugu Sougoumarane 0eecb3c753 sqltypes bug: safer fix for type resolution
The previous fix was still vulnerable to unexpected MySQL
behavior. The new fix checks the flags based on the type.
So, it's more robust.
2016-04-28 16:22:33 -07:00
Sugu Sougoumarane 939ec958b2 sqltypes bug: incorrect vitess type returned
Bug: b/28409793
MySQL sometimes sets stray flags that have to be ignored.
Specifically, for this query:
select max(uint) from t
MySQL sets the binary flag along with the unsigned flag.
The bug fix changes the behavior where vitess only looks at
the relevant flags and ignores all others.
2016-04-28 14:47:40 -07:00
Michael Berlin 4aa8402b50 BinlogPlayer: Do not use unhealthy REPLICA tablets as source. 2016-04-27 21:15:49 -07:00
Michael Berlin 0434ae24a2 discovery: Provide filter helper functions to reduce code duplication.
Note: The GetCurrentMaster() function which will be used by vtworker when we switch to the discovery package.
2016-04-27 21:15:39 -07:00
sougou b44ed6c0ac Merge pull request #1660 from sougou/v3
v3: support sqltypes.Value for string vindexes
2016-04-27 16:47:01 -07:00
Sugu Sougoumarane 35c5aa2018 v3: support sqltypes.Value for string vindexes
All vindexes must accept sqltypes.Value because that's
how values are supplied for cross-shard joins.
I've also updated the example and the v3 e2e test.
2016-04-27 15:17:19 -07:00
sougou 55b6665b35 Merge pull request #1654 from sougou/v3
v3: revised vindexes
2016-04-27 14:52:45 -07:00
Alain Jobart da0b99b597 Merge pull request #1658 from alainjobart/v3error
V3error
2016-04-27 14:13:18 -07:00
Michael Berlin 0beecb01da Merge pull request #1659 from michael-berlin/worker_refactor
vtworker: Do not reuse the mutex from StatusWorker in implementations.
2016-04-27 14:04:44 -07:00
Alain Jobart b7655d2278 Better version of the keyspace watcher in v3.
We now wait synchronously for the first version, when we get them all we
build the initial vschema. We then process changes incrementally as we
did before.
2016-04-27 13:06:34 -07:00
Alain Jobart 1f8f722528 Only log v3 schema errors after all keyspaces.
Otherwise at startup, some keyspace vschemas may depend on others.
We will only log:
- an error when loading vschema once we have all keyspaces.
- any refresh error.
2016-04-27 13:06:34 -07:00
Alain Jobart 6ace21037a Merge pull request #1656 from alainjobart/combo
Add BeginExecute{,Batch} to gRPC, removing tabletconn.ERR*
2016-04-27 13:05:50 -07:00
Michael Berlin 49d7fa3333 vtworker: Do not reuse the mutex from StatusWorker in implementations.
Removing this dependency makes it easier to reason about which components need synchronization and use the lock.

StatusWorker is now fully isolated and can be accessed only through its exported methods.

Other changes:
- Unexported remaining fields in StatusWorker 
- Added State() to worker interface to deduplicate code in unit tests.
- created TabletStatusList which aggregates TableStatus objects and has its own lock now. By resing it, we reduce code duplication in SplitClone and VerticalSplitClone.
- Changed Status* functions to not use the previous global mutex. Instead, StatusWorker and TabletStatusList are called independently now.
2016-04-27 13:01:06 -07:00
Sugu Sougoumarane ef3f030c6e v3 vindexes: handle space padding
Ref: http://dev.mysql.com/doc/refman/5.7/en/char.html
MySQL ignores trailing spaces. So, vindexes should reflect this.
2016-04-27 12:48:41 -07:00
Alain Jobart 8c015b0b10 Removing Err* from tabletserver.
We now use canonical errors. Note over the RPC channel we were always
sending the canonical errors already, and not really using the old style
errors anyway.
2016-04-27 11:35:16 -07:00
Erez Louidor be79afda7b Merge pull request #1657 from erzel/fixing_equal_splits_value_to_big_rat
Added a workaround for b/28409793
2016-04-27 11:22:09 -07:00
Erez Louidor 915a3ef9de Added a workaround for b/28409793 2016-04-27 10:37:57 -07:00
Alain Jobart 7d38d2a036 Removing tabletconn error codes.
They are now unused, we just depend on canonical errors.
2016-04-27 09:37:49 -07:00
Alain Jobart c1189537bb vtgate doesn't use tabletconn.ServerError.Code.
Instead, it just relies on tabletconn.ServerError.ServerCode, which is
the vtrpcpb version of the code.
2016-04-27 09:30:15 -07:00
Alain Jobart 6fbfebc2a4 resolver.isConnError -> resolver.isRetryableError.
It's all it's used for anyway, so it's just clearer.
2016-04-27 09:08:32 -07:00
Alain Jobart 894a34a53e Removing Code from ScatterConnError.
This is part of the next wave of clean-up, to remove tabletconn.ERR_*
values and replace them with vtrpcpb.ErrorCode.
2016-04-27 08:57:33 -07:00
Alain Jobart af6a18b817 Adding BeginExecute{,Batch} rpc to gRPC code.
It is behind a flag for now, until it's been around for long enough, and
then I'll just enable it by default.
2016-04-27 08:11:32 -07:00
Michael Berlin faa3f2f2c2 vtworker: Log head of stacktrace in case of a panic. 2016-04-26 23:52:06 -07:00
Michael Berlin dd05dec73c vtworker: Abort when filtered replication is already enabled. 2016-04-26 23:50:19 -07:00