github
/
vitess-gh
зеркало из https://github.com/github/vitess-gh.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
8 136 коммитов 101 ветка 22 Теги 482 MiB
Граф коммитов

950 Коммитов

Автор SHA1 Сообщение Дата
Dominic Hamon fd1e0b8fff Introduce error for types that can't be bound to proto3. 
Plumb the error all the way through the stack.
 2015-10-22 12:44:50 -07:00
Ammar Aijazi 0e403f82c0 Slightly improve an error message for DMLs outside of transactions 2015-10-20 19:22:10 -07:00
Ammar Aijazi 8541f8e1a5 Reverrt TerseErrors so that they default to False 2015-10-20 15:57:11 -07:00
Ammar Aijazi 152cb697a8 Return a TabletError even when TerseErrors are enabled 2015-10-20 15:12:03 -07:00
Sugu Sougoumarane 3a9476137b tabletserver: lint & vet fixes 2015-10-17 22:27:45 -07:00
Sugu Sougoumarane 1c31fbfd6a tabletserver planbuilder: lint errors fixed 2015-10-17 11:02:28 -07:00
Sugu Sougoumarane 80ddd79fc4 tabletserver planbuilder: simplified tests 2015-10-17 00:16:30 -07:00
Sugu Sougoumarane 09d8970259 sqlparser: ast.go lint warnings 2015-10-16 13:53:00 -07:00
Sugu Sougoumarane 09e8047e69 tabletserver: fix SetServingType 
If the requested serving state exactly matches the required state,
we should not go through a transition.
 2015-10-15 12:59:00 -07:00
Sugu Sougoumarane bde6d88fe1 tabletserver: fix race condition 
We discovered an interesting race condition between Begin
and StopServer:

* Begin gets called: reserves a request, but takes time to
perform the next step, state is SERVING.
* StopService gets called: transitions state to SHUTTING_DOWN.
* StopService calls WaitForTx: the pool is empty. So, it
proceeds to perform other shutdown operations.
* Begin opens a transaction, because it was given permission.
The pool is not empty any more.
* StopService calls txpool.Close, which gets angry because
no transactions should be in it.

One possible solution I considered was to reserve an entry
in the transaction pool during startRequest. Then Begin would
later fill it out with the connection obtained after BEGIN
succeeded.  This would eliminate the race, but it also
introduces weird dependencies between TabletServer and TxPool.
It would also require Begin to un-reserve if the BEGIN failed.
Overall, I felt the above approach would lead to unreadable code.

So, I went with the following approach:
* Introduce a new 'begins' waitgroup that counts number of Begin
calls in flight.
* Increment & Decrement it in startRequest and endRequest.
* Make those who want to stop query service wait for begins to
be zero before waiting on tx pool to go empty.

So, if there are any Begins in flight when we go into shut down,
we wait for them to complete. As they complete, they all reserve
a connection in the tx pool. Once the Begins are done, no new begins
are allowed, and those that completed all have created transactions.
So, at this point, it's safe to wait for tx pool to go empty.

This is not really testable. So, we have to rely on the above proof
and verification that the code was written accordingly.
 2015-10-14 23:38:01 -07:00
Sugu Sougoumarane bd486ba51f tabletserver endtoend: deflake some tests 2015-10-14 12:00:11 -07:00
Alain Jobart 37ab2fae49 Merge pull request #1207 from alainjobart/resharding 
Removing python object library, and python direct tablet access.
 2015-10-13 12:36:07 -07:00
Alain Jobart 28f36d6576 Adding unit test for new flag. 2015-10-13 06:56:44 -07:00
Sugu Sougoumarane 371c95c853 tabletserver endtoend: delete TestStatus 
The test was a nuisance and wasn't really doing much.
 2015-10-12 21:41:20 -07:00
Sugu Sougoumarane 5f0a05ae86 tabletserver: fixed rowcache flags 
EnableRowcache flag has been moved from dbconfigs.DBConfigs to
tabletserver.Config.
EnableRowcacheInvalidator has become obsolete because that decision
is dictated by tablet type. It's been deleted from dbconfigs.DBConfigs.
 2015-10-12 21:37:50 -07:00
Sugu Sougoumarane dfad470fde tabletserver: TabletServer API change 
Removed StartService & StopService from control API.
Also made target & dbconfigs as value parameters and mandatory.
There are still a couple of plances where there is a nil check.
For now, I've changed them to IsZero check because I don't know
if something could break otherwise.
 2015-10-12 21:02:15 -07:00
Alain Jobart 8fd2543994 Merge pull request #1201 from alainjobart/resharding 
Switching tabletmanager to the new tabletserver API.
 2015-10-12 13:17:59 -07:00
Alain Jobart 5d0b747661 Adding the 'serving' flag to StreamHealthCheck. 2015-10-12 12:05:43 -07:00
Alain Jobart de93a3131e Now verifying the contents of the broadcast health messages. 2015-10-12 10:23:43 -07:00
Alain Jobart 99dbfd5987 Switching tabletmanager to the new tabletserver API. 2015-10-12 08:55:36 -07:00
Sugu Sougoumarane ecfa61b02e tabletserver endtoend: fix merge issue 2015-10-11 22:14:29 -07:00
Sugu Sougoumarane 4819a53378 Merge branch 'master' into suguwork 2015-10-11 22:12:11 -07:00
Sugu Sougoumarane 523af9140c tabletserver endtoend: vtocc->vitess 
I believe all references to vtocc have now been deleted.
 2015-10-11 21:41:49 -07:00
Sugu Sougoumarane cbe2f915fa tabletserver endtoend: simplify framework 2015-10-11 21:28:42 -07:00
Sugu Sougoumarane 8afcb3b60d tabletserver endtoend: stream queries done 
All tests have now been ported. It's now time for cleanup.
 2015-10-11 13:36:39 -07:00
Sugu Sougoumarane c6c6be10e5 tabletserver endtoend: some stream tests 2015-10-11 01:40:01 -07:00
Sugu Sougoumarane 97fd5a0c9b tabletserver endtoend: rest of the cache tests 2015-10-11 01:05:23 -07:00
jeffjiang a1d029faa1 Merge pull request #1198 from guokeno0/logz 
Add callerid support for txlogz and querylogz
 2015-10-10 23:43:11 -07:00
Jeff (Zhefu) Jiang f73ff638eb change querylogz_test.go to use context.Background() 2015-10-10 23:28:27 -07:00
Jeff (Zhefu) Jiang 93a57630e1 Add callerid to txlogz and querylogz (backport from internal) 2015-10-10 18:09:49 -07:00
Jeff (Zhefu) Jiang ccd906c1e2 Revert "Let TableACL clear query plan cache on acl reload" 
This reverts commit b7736ff2ac.
 2015-10-10 17:50:35 -07:00
Jeff (Zhefu) Jiang 3bb9f5fe87 Revert "fix tableacl tests" 
This reverts commit a83456f6f4.
 2015-10-10 17:50:17 -07:00
Sugu Sougoumarane 11273730ff tabletserver endtoend: cache_cases2 2015-10-10 16:47:44 -07:00
Sugu Sougoumarane 519819fc62 tabletserver endtoend: cache_cases1 2015-10-10 16:21:43 -07:00
Sugu Sougoumarane 3553772c61 tabletserver endtoend: nocache cases 
I've written a code generator that converts the python cases
into go. Some massage is needed after the conversion.
The downside is that it's a brainless port. No re-interpretation,
regrouping or gap analysis.
 2015-10-10 14:02:20 -07:00
Sugu Sougoumarane 4f0def0a34 tabletserver endtoend: some nocache cases 2015-10-09 20:45:18 -07:00
Jeff (Zhefu) Jiang a83456f6f4 fix tableacl tests 2015-10-09 18:55:06 -07:00
Sugu Sougoumarane 382cf18203 tabletserver endtoend: testcase framework 2015-10-09 18:54:23 -07:00
Jeff (Zhefu) Jiang b7736ff2ac Let TableACL clear query plan cache on acl reload 2015-10-09 18:20:30 -07:00
Sugu Sougoumarane c3b2901e2c tabletserver endtoend: most cache tests done. 2015-10-09 00:02:02 -07:00
Sugu Sougoumarane 88da603501 tabletserver endtoend: a few cache tests 2015-10-08 18:59:02 -07:00
Ammar Aijazi 8f032c6d0b Demote one more MySQL error from the logs 2015-10-07 16:44:42 -07:00
Sugu Sougoumarane 467c03ce93 Merge branch 'master' into suguwork 2015-10-05 21:16:45 -07:00
Sugu Sougoumarane a9bc7b9867 tabletserver endtoend: regroup tests 2015-10-05 21:04:19 -07:00
Sugu Sougoumarane a28b120196 tabletserver endtoend: nocache_tests all done 2015-10-05 20:56:54 -07:00
Sugu Sougoumarane 971f5a94c4 tabletserver endtoend: health check 
Also changed DebugVars fetch delimiter to be '/' because some tags
may contain '.'
 2015-10-05 16:43:18 -07:00
Alain Jobart 9df18a5908 Adding comment to fix import into google3. 2015-10-05 15:59:36 -07:00
Sugu Sougoumarane 2c0518ea48 Merge branch 'master' into suguwork 2015-10-04 23:24:18 -07:00
Sugu Sougoumarane 2dd0f919a1 tabletserver endtoend: autocommit off 
The test was missing in the python tests.
 2015-10-04 22:24:18 -07:00
Sugu Sougoumarane 7c99b152ed tabletserver endtoend: more tests migrated 
TypeLimits, BindInSelect, QueryRules
 2015-10-04 18:53:24 -07:00
Sugu Sougoumarane 5c84087b0c tabletserver endtoend: ExecuteBatch tests 2015-10-04 11:36:29 -07:00
Sugu Sougoumarane 59a20a7a57 streamlog: increase subscriber channel size 
The go runtime schedules goroutines in any order. This means that
multiple sends can happen on a buffered channel before a runnable
receiver goroutine gets scheduled. This could cause dropped messages
even on an uncontended system.
This seems to happen very frequently for streamlog where the channel
size is 1. For now, I'm increasing this to match the original sender
channel size. Hopefully, this will lead to fewer dropped messages.
 2015-10-03 20:40:46 -07:00
Sugu Sougoumarane d51a37779b tabletserver endtoend: consoildation test 
The race detector found a bug in the QueryExecutor. It probably
hasn't failed on production because the racy code paths were always
trying to set the variable to the same value.
 2015-10-03 19:26:23 -07:00
Sugu Sougoumarane 1d78911dbc tabletserver endtoend: regroup tests 
Moved config tests to a different file.
 2015-10-03 11:34:40 -07:00
Sugu Sougoumarane f523f628c4 tabletserver endtoend: config tests done 2015-10-02 22:08:02 -07:00
Sugu Sougoumarane 7aa8825fdc tabletserver: event capture reorg 
Reorganized Tx & Query capture to reuse the same code
and made their API uniform.
 2015-10-02 21:32:06 -07:00
Sugu Sougoumarane fac3a448ad tabletserver: deflake some tests 2015-10-02 17:23:49 -07:00
Alain Jobart 383e9b19d2 Changing the code a bit after comments. 2015-10-02 16:57:34 -07:00
Sugu Sougoumarane 37d004837f tabletserver: test cleanup 
remove tests that have become obsolete so far.
I'm also removing the corresponding SET functionality support.
 2015-10-02 16:15:32 -07:00
Sugu Sougoumarane 5add0c8ffd tabletserver: unit tests for config functions 2015-10-02 15:48:13 -07:00
Sugu Sougoumarane abbfaa703f tabletserver test: server reconnect test 
Port the server reconnect test to use the mock API. It's non-trivial
to automate mysql restarts for now.
 2015-10-02 14:14:21 -07:00
sougou f776a4d858 Merge pull request #1163 from youtube/suguwork 
tabletserver: revert the revert
 2015-10-02 12:08:54 -07:00
Sugu Sougoumarane 1604b5d6f9 tabletserver endtoend: fix logtracker 
Looks like unit_race is too slow. So, the wait for events becomes
flaky with a 1 second timeout. Increased it to 5s.
 2015-10-02 10:32:11 -07:00
Sugu Sougoumarane 40e25e9ec6 tabletserver: fix broken build 2015-10-02 10:21:23 -07:00
Alain Jobart afabdaf351 vtcombo fix: errors were not handled properly. To fix this, on the fake 
vtgate -> vttablet link, I now convert the errors to gRPC errors,
and back. That way I use the RPC code, and no need to implement
anything specific.
 2015-10-02 10:18:01 -07:00
Sugu Sougoumarane 512a73ef69 Revert "Revert "Merge branch 'suguwork'"" 
This reverts commit 8062499471.
 2015-10-02 09:54:20 -07:00
Ammar Aijazi 8062499471 Revert "Merge branch 'suguwork'" 
This reverts commit bf185cb9db, reversing
changes made to 347c76ff50.
 2015-10-02 07:44:21 -07:00
Sugu Sougoumarane bf185cb9db Merge branch 'suguwork' 2015-10-02 00:16:14 -07:00
Sugu Sougoumarane 1e24732797 tabletserver endtoend: querylog framework 
and a test that uses it
 2015-10-02 00:02:42 -07:00
Sugu Sougoumarane 7583ade16a tabletserver endtoend: more config tests 2015-10-01 21:37:09 -07:00
Sugu Sougoumarane cc7128c657 tabletserver endtoend: config test 
Also simplified some boiler-plate code.
 2015-10-01 15:00:34 -07:00
sougou b43a7c4419 Merge pull request #1151 from youtube/suguwork 
tabletserver endtoend: WIP
 2015-10-01 10:17:26 -07:00
Sugu Sougoumarane 2aed780180 tabletserver endtoend: more transaction tests 2015-09-30 18:03:10 -07:00
Sugu Sougoumarane 355bed457f tabletserver: reorg some timeouts 
queryTimeout moved from QueryEngine to TabletServer.
poolTimeout moved from TxPool to TabletServer, and renamed to
beginTimeout.
 2015-09-30 15:30:10 -07:00
Sugu Sougoumarane 75cd5721aa tabletserver endtoend: transaction tests WIP 2015-09-29 21:19:29 -07:00
Sugu Sougoumarane 1864f1c5df tabletserver endtoend: tx log tracker 
and a test that uses it.
 2015-09-29 16:44:35 -07:00
Ammar Aijazi db029940ce Merge remote-tracking branch 'upstream/master' into aaijazi_add_error_test_cases_to_vtgateclient 2015-09-28 16:51:21 -07:00
Sugu Sougoumarane b51a9134b3 vttest: LauncMySQL -> LaunchMySQL 2015-09-28 13:36:36 -07:00
Sugu Sougoumarane 79ad20e621 tabletserver endtoend: new framework package 
The utility functions were growing. I've moved them to a separate
framework package.
 2015-09-27 22:33:58 -07:00
Sugu Sougoumarane 7d3ef1bc68 tabletserver endtoend: list args 2015-09-27 21:29:43 -07:00
Sugu Sougoumarane 3d2dab9176 tabletserver endtoend: functions for /debug/vars 
Also added a simple test that uses those functions.
 2015-09-27 15:48:49 -07:00
Sugu Sougoumarane 321fc36057 tabletserver endtoend: NULL type test 2015-09-27 00:53:39 -07:00
Sugu Sougoumarane c6aeeecbe8 tabletserver endtoend: rest of the types 2015-09-26 22:55:21 -07:00
Sugu Sougoumarane b5d1688e20 tabletserver endtoend: ints test 
This change also introduces constants in mysql package for flags
and types. Additionally, I mask out flag bits that are not relevant.
This will help us be deterministic about what we do or don't support.
 2015-09-26 21:38:15 -07:00
Sugu Sougoumarane 16f50123f2 tabletserver: fetch memcached path from vttest 
The path was previously hardcoded in a non-customizable file. It's
now moved to environment.go.
 2015-09-26 11:57:58 -07:00
Sugu Sougoumarane 1d51566fe1 tabletserver: first end to end test in go 2015-09-25 15:26:20 -07:00
Ammar Aijazi f7029a76de Simplify AddTabletError functions 2015-09-25 12:04:52 -07:00
Sugu Sougoumarane 25442d61b8 tabletserver: queryctl review comments addressed 
I've also renamed queryctl.go to config.go, because what's left
is mostly Config. The Controller interface is still there. We
could move it to tabletserver.go, or leave it config.go.
For now, I haven't moved it.
 2015-09-25 10:20:24 -07:00
Sugu Sougoumarane 0b821c71ea queryctl: QueryServiceControl -> Controller 2015-09-25 00:40:57 -07:00
Sugu Sougoumarane 81af6079b0 tabletserver: queryctl revamp 
queryctl was initally created because TabletServer could not
export non-rpc functions. Otherwise, the go rpc framework would
complain about those functions.
However, this has changed because we now have rpc-specific adapters.
So, this means that TabletServer can export any function it pleases.
This has made the objects in queryctl redundant.
This change pushes those thin wrappers straight into TabletServer.
Also, I've moved the mocking interface into its own package. This
prevents test code from intermixing with normal code.
There's not much left in queryctl. So, I'm currently debating if it
should remain as a file. I'm leaving it there for now.
 2015-09-24 21:38:34 -07:00
Sugu Sougoumarane 97c0b39baa queryctl: remove checkMySQL global 2015-09-24 16:38:07 -07:00
Alain Jobart 01d6e62ecb The query rules are not a global variable any more. 
It's prettier this way, and doesn't mess us vtcombo.
 2015-09-24 10:44:41 -07:00
Alain Jobart 8efc861504 WIP on vtcombo: queries now work, if they don't need a db name. 2015-09-23 13:10:26 -07:00
Sugu Sougoumarane d8b897afb0 tabletserver test: Fix up faking scheme 
I found a few problems and shortcomings in the mocks:
1. A lock was being held while connecting to mysql, which should
be avoided.
3. Tests depended on a global variable (DefaultDB) being initialized
to different values at the beginning of each test. This would prevent
parallelization.
2. There was no way to use a real mysql connection in tests once
a fakesqldb was setup, which is needed for the queryservice tests
to be ported to go.

In the new scheme, I've added an Engine field to ConnParams. If it's
empty, then the default (MySQL) connection is used without locks.
If there is a name, then we use the map where other test engines
are registered.

The next change will convert the memcache faking to a similar scheme.
 2015-09-22 16:28:44 -07:00
Alain Jobart ff800b2a18 Retiring SpitQueryPart, using the proto3 version now. 
Note the hadoop java code is broken, as it uses the old
java client and not the new one (and the bson structure changed).
The right fix is probably to switch to new client there.
 2015-09-21 12:17:09 -07:00
Bin Wu 27329263bf Test why timeout. 2015-09-16 16:17:13 -07:00
Bin Wu 13ae48e99f Test why timeout. 2015-09-16 16:07:58 -07:00
Bin Wu 18e8a44a7c Test why timeout. 2015-09-16 15:58:11 -07:00
Bin Wu 28aa68beb0 Merge https://github.com/youtube/vitess 2015-09-16 11:59:55 -07:00
Bin Wu f58e8de832 Another change. 2015-09-15 17:48:59 -07:00