github
/
vitess-gh
зеркало из https://github.com/github/vitess-gh.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
8 136 коммитов 104 Ветки 22 Теги 482 MiB
Граф коммитов

950 Коммитов

Автор SHA1 Сообщение Дата
Sugu Sougoumarane 04325bae34 Formatting tweaks 2014-02-18 20:42:19 -08:00
Sugu Sougoumarane 836aa3174e querylogz 2014-02-18 20:25:55 -08:00
Sugu Sougoumarane dcc7900c92 txlogz first cut. 2014-02-16 21:54:08 -08:00
Sugu Sougoumarane 763a1dd8ee Improved streamlogger. 
Simplified the streamlogger code.
Extended it to allow for local non-http subscribers.
Added tests.
The non-http subscription will be used for additional
monitoring and status reporting.
 2014-02-16 11:02:54 -08:00
Sugu Sougoumarane aa53bd3164 query_rules can now filter by table names. 2014-02-15 15:01:09 -08:00
Sugu Sougoumarane 7c13251cc5 Check dbname in rowcache invalidator. 2014-02-15 13:39:42 -08:00
Sugu Sougoumarane 37d457c2b6 Minor tweaks to the query service API. 2014-02-14 21:14:52 -08:00
Sugu Sougoumarane 3e7db5bc00 Rowcache invalidator launch revamp 
- Rowcache invalidator is now launched as part of query service
- vtocc can also launch rowcache invalidator, with new flag binlog-path
- Explicit flags: enable-rowcache, enable-invalidator
- vtocc also needs dba credentials for invalidator
- New sync2.ServiceManager for managing goroutine start & stop
 2014-02-14 15:14:33 -08:00
Sugu Sougoumarane efa816f87b Better cleanup and improved readability. 
If query_engine fails to start, it doesn't cleanup properly.
Somehow, this hasn't bitten us yet. Fixed now.
Methods checking for nil is not a common code pattern.
To prevent confusion, I've moved the nil checks one level up,
which is what people usually do.
 2014-02-10 16:06:22 -08:00
Sugu Sougoumarane 7c7b770eba Ignore cross-db transactions in rowcache invalidator 2014-02-09 14:10:04 -08:00
Alain Jobart 5dd0582c22 Merge branch 'replication' 2014-02-07 10:23:51 -08:00
Alain Jobart 28ec8c76c7 Closing RPC connection in case of failure. 2014-02-07 10:17:29 -08:00
Sugu Sougoumarane 97487eb575 More resilient Rowcache invalidator 
Changed rowcache invalidator to:
1. Keep retrying on unexpected errors.
2. Not to stop query service. This is done by the tablet manager.
 2014-02-05 17:00:16 -08:00
Alain Jobart 6b580d1ab1 Refactoring mysql passwords for CredentialServer. 
We now ask the CredentialsServer everytime we connect for
an up to date password. Refactor code everywhere to use that
feature, and not duplicate mysql.ConnectionParams everywhere.
 2014-01-31 13:21:33 -08:00
Alain Jobart 42ec640375 Now forwarding the context through vtgate. 2014-01-29 12:45:38 -08:00
Alain Jobart 47346a81e0 Simplifying registration glue. 2014-01-29 11:42:05 -08:00
Alain Jobart 443b8859c8 Merge branch 'master' of github.com:youtube/vitess 2014-01-29 10:42:04 -08:00
Alain Jobart 9ab7218b62 Fixing package name. 2014-01-29 10:41:36 -08:00
Sugu Sougoumarane ccb7c11732 Merge branch 'rowcache' 2014-01-29 09:12:12 -08:00
Sugu Sougoumarane b4a7090051 Disallow PASS_DML plan 
PASS_DML doesn't play well with rowcache, update stream,
and filtered replication. Blacklisting it for now.
We'll need to later clean up the code paths that lead to
it once things stabilize.
 2014-01-29 07:30:54 -08:00
Alain Jobart c71c5a1b33 Moving client side of queryservice to its own lib. 2014-01-28 16:26:45 -08:00
Alain Jobart 428e929fdb Adding context, removing unused params. 2014-01-28 15:48:40 -08:00
Alain Jobart 99dad4d1ff Gorpc service is now a plugin for vttablet/vtocc 2014-01-28 15:30:22 -08:00
Sugu Sougoumarane 50b5cf6bd2 Handle unrecognized DDLs as non-fatal. 
Stopping query service for unrecognized DDLs is too abrupt.
Instead, increment the InternalErrors.Invalidation counter
so we can tie that variable to an alert.
 2014-01-27 15:30:49 -08:00
Alain Jobart 3616ccf727 Typing the UpdateStream replies. 2014-01-27 11:29:43 -08:00
Alain Jobart 8aa9dca8cf Moving binlog into its own package. 2014-01-27 09:49:54 -08:00
Sugu Sougoumarane 2c4ad6c923 Add .9 precision for nanosecond accuracy. 2014-01-03 12:17:25 -08:00
Sugu Sougoumarane 4c691db695 Change streamlogs to not use scientific notation. 2014-01-03 11:56:10 -08:00
Sugu Sougoumarane db9747fa21 Merge branch 'vtgate' 2013-12-24 15:25:43 -08:00
Sugu Sougoumarane 064d69300d go vet cleanup 2013-12-24 15:24:45 -08:00
Liang Guo 9eccebf797 Rename rowcache parameter names 2013-12-18 16:23:44 -08:00
Alain Jobart a272a5b993 Group id is now always an int64. 2013-12-18 09:43:13 -08:00
Sugu Sougoumarane 90591f9ce7 vttablet error stats recategorization. 
InfoErrors: Expected failures from normal app flows, like dup key.
Errors: Errors that cause actual failures in the app. A high rate here
should generate a page.
InternalErrors: Unexpected internal errors, that should generate an
alert (not a page), which need to be escalated to the corresponding
developers.
 2013-12-07 20:16:52 -08:00
Sugu Sougoumarane 9f6b283fc2 Stats & logging for UpdateStream 
UpdateStream now reports:
- Number of streams it's running
- Number of errors it's encountered
- Number of statements or events parsed
- Number of transactions

Also took care of some misc TODOs.
 2013-12-06 11:50:39 -08:00
Sugu Sougoumarane e8cbdc2ce6 Undo some ineffective optimizations. 
Allow streaming queries to complete on shutdown.
Don't throttle streaming queries till first row return. It doesn't
help.
 2013-12-05 21:16:49 -08:00
Sugu Sougoumarane 6b2a922189 Delete reserved connections (deprecated). 2013-12-04 14:50:29 -08:00
Sugu Sougoumarane 3d3d08f87f Custom bson for updatestream protos. 2013-12-02 21:31:51 -08:00
Sugu Sougoumarane 35746ec3fd Move rpc types to proto. 2013-12-01 15:53:16 -08:00
Ryszard Szopa 35f95d31bd Export QPS rates from the query engine. 2013-11-27 16:50:33 -08:00
Ryszard Szopa 7c86a042bd Expose SqlQuery state. 2013-11-27 14:12:35 -08:00
Sugu Sougoumarane 9e4e320cee Merge branch 'master' into new-binlog-streamer 2013-11-26 21:46:38 -08:00
Sugu Sougoumarane d25a430ebc Fix tests after state var name change. 
Submitting ric's change.
 2013-11-26 21:45:32 -08:00
Sugu Sougoumarane 502f884e55 The grand purge: delete old binlog code 2013-11-26 20:49:57 -08:00
Sugu Sougoumarane 4eb40cc6de fix merge error: delete states 2013-11-26 18:55:43 -08:00
Sugu Sougoumarane 88fe17a2bc Merge branch 'new-binlog-streamer' 
Conflicts:
	go/vt/tabletserver/rowcache_invalidator.go
 2013-11-26 18:50:05 -08:00
Sugu Sougoumarane b01386add9 Standardize GroupId type and usage 2013-11-26 10:23:04 -08:00
Ryszard Szopa a3d4b44461 Stop using stats.States in most cases. 2013-11-25 12:19:15 -08:00
Sugu Sougoumarane 92c745a798 Fix data race in conn_pool 2013-11-20 00:40:27 -08:00
Sugu Sougoumarane 78c880dcad Big leap: update_stream uses binlog_streamer 
The update stream API also changes. It accepts a group id
based BinlogPosition as input, and streams events one at a
time. The position is also a separate event at the end of
each transaction.
 2013-11-18 00:41:05 -08:00
Sugu Sougoumarane 402b9dad0b unify return type for EventStreamer 2013-11-17 01:27:03 -08:00
Sugu Sougoumarane 4745bacd33 updatestreamctl uses event_streamer (WIP) 2013-11-17 00:54:55 -08:00
Sugu Sougoumarane 5d1f5238da Suppress tx_pool_full logging, too spammy. 2013-11-06 08:12:42 -08:00
Sugu Sougoumarane 6627763690 improved "in use" error, drop vttopo 
Numbered pool now accepts a purpose when locking resources,
and reports it as part of the error if it's in use.
Also dropping vttopo from Makefile and .gitignore.
 2013-11-04 07:38:53 -08:00
Sugu Sougoumarane 38e6cc5bac Merge branch 'barnacle' 2013-10-25 21:29:07 -07:00
Sugu Sougoumarane 672d706e71 Tests for ExecuteBatch structs. 2013-10-25 21:26:47 -07:00
Liang Guo 254286caf6 Remove false warnings in vttablet 2013-10-24 14:15:56 -07:00
Liang Guo f08c07c171 fix vttablet uncaught panic on commit 2013-10-24 11:04:22 -07:00
Sugu Sougoumarane 9685d740f6 vttablet protocol changes 
Client stops transmitting ConnectionId so we can drop
support for reserved connections (obsolete). Also changed
ExecuteBatch API to separate out session info from query list.
 2013-10-23 07:45:43 -07:00
Sugu Sougoumarane d379561f66 Merge branch 'barnacle' 2013-10-22 21:01:52 -07:00
Sugu Sougoumarane 5c590cbc33 tests for tabletserver bson types 2013-10-22 08:46:54 -07:00
Liang Guo 2bee15f6b3 Merge branch 'master' into config 
Conflicts:
	go/vt/tabletserver/cache_pool.go
 2013-10-21 14:33:40 -07:00
Sugu Sougoumarane e59ea49f58 Merge branch 'master' into barnacle 2013-10-19 09:19:48 -07:00
sougou 7c5168df6a Merge pull request #31 from dgryski/go-vet-lint-fixes 
Some trivial fixes from "go vet" and "golint"
 2013-10-19 09:17:44 -07:00
Damian Gryski 9fe8e50c32 all: for foo, _ := range ... -> for foo := range ... 2013-10-19 14:35:39 +02:00
Damian Gryski 276daeef9b all: format string fixes 2013-10-19 14:32:06 +02:00
Sugu Sougoumarane dd01dafbd8 Simplify bson encode API. 2013-10-18 15:43:26 -07:00
Liang Guo 9b7beac056 Merge branch 'master' into config 2013-10-17 16:59:36 -07:00
Liang Guo b99f367070 Fix memcache stats variable reference 2013-10-17 16:55:38 -07:00
Liang Guo 1223ccd9c9 Merge branch 'master' into config 
Conflicts:
	go/vt/tabletserver/queryctl.go
	test/connection_test.py
 2013-10-17 15:46:30 -07:00
Liang Guo 3b0d0c540e Update config parameter code 2013-10-17 14:04:40 -07:00
Sugu Sougoumarane 12966797ae Merge branch 'master' into barnacle 2013-10-14 21:57:35 -07:00
Sugu Sougoumarane 3f0fd5620b minor tweaks. 2013-10-12 22:54:18 -07:00
Sugu Sougoumarane 42c6dd56eb update stream standardized error handling 2013-10-12 17:07:29 -07:00
Sugu Sougoumarane d173b90335 more tweaks to throttler & commit vars. 2013-10-11 10:22:18 -07:00
Sugu Sougoumarane 2504714061 Track commit time, and further simplified throttler. 2013-10-11 09:54:28 -07:00
Ryszard Szopa 3a94c86870 Fix typo in streamlogger. 2013-10-10 12:51:34 -07:00
Sugu Sougoumarane da5c050d5e Custom bson encoding for rpc objets. 2013-10-04 00:50:15 -07:00
Sugu Sougoumarane c82bc9ce87 Mild improvement in GetSessionId check. 2013-10-03 10:25:36 -07:00
Sugu Sougoumarane c756f9ca57 stats tx -> transaction 2013-10-02 21:00:01 -07:00
Sugu Sougoumarane 3f0652d9b5 Rename Cache related vars more consistently. 2013-10-02 20:29:12 -07:00
Sugu Sougoumarane 31d8e6697e Use better suffixes for new stats vars. 2013-10-02 17:04:50 -07:00
Sugu Sougoumarane 3809d76bc2 Move error constants into mysql package. 2013-09-30 09:35:29 -07:00
Sugu Sougoumarane ee8a454e2d Report deadlocks as separate error stats. 2013-09-29 13:36:05 -07:00
Sugu Sougoumarane badea14914 Better error messages. 2013-09-27 18:40:12 -07:00
Sugu Sougoumarane 2d46d9a7d2 Throttle for streaming queries. 
Also includes changes to semaphore to support timeouts.
 2013-09-27 01:18:37 -07:00
Sugu Sougoumarane a09a30c310 Simplified query service state machine. 
Got rid of CLOSED state, renamed OPEN to SERVING to mirror the
NOT_SERVING state. Also, DisallowQueries does not take a forRestart
flag any more. Query service always returns RETRY during (and after)
shutdown.
 2013-09-21 13:55:44 -07:00
Sugu Sougoumarane 89449e4d1d Improved transaction error handling. 2013-09-20 22:21:30 -07:00
Sugu Sougoumarane 4a046112d4 Add idle timeout tracking to Numbered. 2013-09-19 11:43:06 -07:00
Liang Guo 95844e4d4d Change vttablet and tools to accept dbconfig and queryserver configs in command line 2013-09-17 16:06:43 -07:00
Liang Guo f0e2e5643f Update vttablet vars 2013-09-12 17:13:43 -07:00
Liang Guo 4b597e9597 Replace MySQL vars to Mysql to enable tidy stats 2013-09-10 15:40:18 -07:00
Liang Guo d0092b04f0 Export memcache stats, table stats and query stats through stats package. 2013-09-08 22:22:52 -07:00
Liang Guo 84cada73a2 Merge branch 'master' into liguo 2013-09-05 14:22:44 -07:00
Liang Guo d14606511e Export memcache stats, table level stats, and query level stats through stats package 2013-09-05 14:21:13 -07:00
Sugu Sougoumarane 8562d88484 Stricter check on GetSessionId 2013-09-04 19:46:37 -07:00
Liang Guo 2ed2a83699 Export Memcache internal stats through stats package. 2013-09-03 17:00:43 -07:00
Alain Jobart ed32fa48db Removing unused SessionParams attributes. 
The python code only uses Keyspace and Shard.
That also allows me to remove KeyRange from dbconfigs,
was only used to validate against the connection params.
(the other one, Dbname, is used in other places).
 2013-09-02 22:44:15 -07:00
Sugu Sougoumarane ceb2052c44 revamped rowcache. Drop support for anything that requires nested 
lookups. Use consolidated IN clauses for tables that have simple
primary keys.
 2013-09-02 17:17:23 -07:00
Sugu Sougoumarane 3bdeedf1b4 fix race condition on spot check 2013-08-29 22:42:57 -07:00
Sugu Sougoumarane e59c7de895 abort streaming queries on shut down. 2013-08-26 00:51:21 -07:00
Sugu Sougoumarane eee9608a80 increase grace period for spot checks to 10s 2013-08-25 10:16:11 -07:00
Sugu Sougoumarane dd57b0cce4 improved rocache spot check 
knob for spot-checking
wait longer for memcache & fail fatally on failure
don't store rows with long keys in rowcache
 2013-08-25 00:44:19 -07:00
Sugu Sougoumarane 6cefee5765 special prefix for tx pool error 2013-08-20 14:00:00 -07:00
Sugu Sougoumarane ef506f5c93 rowcache: verifyMode support 2013-08-19 22:21:26 -07:00
Sugu Sougoumarane 1a635578bc expvar -> stats phase 2 2013-08-14 15:26:49 -07:00
Sugu Sougoumarane cfb38ce17a bug fix: typo on pool check. 2013-08-12 11:31:08 -07:00
Sugu Sougoumarane 075a6c2069 Remove VT prefix from exported vars 2013-08-12 10:21:52 -07:00
Sugu Sougoumarane b89eeaefc4 Voltron expvar exported as individual vars 2013-08-12 01:47:35 -07:00
Sugu Sougoumarane 217b2e3061 stats: change states to int64 2013-08-11 18:37:28 -07:00
Sugu Sougoumarane 0fac9bf44a stats: Add support for func based export vars. 2013-08-11 17:49:24 -07:00
Ryszard Szopa 2b414a7d03 Reorganize functionality that's common for servers and query servers. 2013-08-09 20:28:57 -07:00
Ryszard Szopa 59c2fe3bf4 Do not write the debug querylog when the streaming query log is available. 2013-08-09 16:52:47 -07:00
Ryszard Szopa 2ccb58696e tabletserver.IsHealthy. 2013-08-08 23:45:39 -07:00
Ryszard Szopa 66cf2567c4 Convert commands to use glog. 
Conflicts:
	go/cmd/vtocc/vtocc.go
	go/cmd/vttablet/vttablet.go
 2013-08-06 14:58:10 -07:00
Ryszard Szopa cead93c364 Automatic rewrite of relog import paths and calls to use glog. 
Commands run:

  find go -name "*.go" | xargs sed --in-place -r 's,"github.com/youtube/vitess/go/relog",log "github.com/golang/glog",g; s,relog.Info,log.Infof,g; s,relog.Warning,log.Warningf,g; s,relog.Error,log.Errorf,g; s,relog.Fatal,log.Fatalf,g; s,relog.Debug,log.V(6).Infof,g'
  find . -name '*.go' -exec gofmt -w {} \;
 2013-08-06 14:56:00 -07:00
Sugu Sougoumarane 79fb4f179f Prep work for alternate ways to export variables, and more comments and 
tests.
 2013-08-05 11:50:01 -07:00
Alain Jobart d93747081a Now moving BinlogPosition into mysqlctl/proto. 
And unifying it with BlPosition.
 2013-08-01 18:08:10 -07:00
Sugu Sougoumarane b00275025a rowcache: bug fix on closing of CachePool 2013-07-24 13:33:14 -07:00
Sugu Sougoumarane 4ec7191503 rowcache: change to a simplified prefix scheme 2013-07-23 17:27:03 -07:00
Sugu Sougoumarane 0c3e481e8f rowcache as child, a few fixes 2013-07-22 22:29:43 -07:00
Sugu Sougoumarane 38e639d96b rowcache as child, phase 2 2013-07-22 22:24:17 -07:00
Sugu Sougoumarane caa0aa30f9 memcache as child of vttablet, phase 1. 2013-07-22 22:24:17 -07:00
shrutip bf31119e76 Fix for update stream test. 2013-07-19 18:38:13 -07:00
Sugu Sougoumarane 043a1633bb bug fix: Use cachePool.IsClosed to check if rowcache is enabled 2013-07-19 15:17:50 -07:00
Ryszard Szopa 0d046a5984 Replace code.google.com with github.com/youtube/vitess. 2013-07-18 18:18:20 -07:00
Alain Jobart bbc923036c Fixing debug/vars for invalidator. 2013-07-15 11:38:13 -07:00
Sugu Sougoumarane 3ac32cae1a improved compareRow, handles some race conditions 2013-07-14 09:13:58 -07:00
Sugu Sougoumarane c7c2a291d5 Use memcache multi-gets for higher rowcache efficiency 2013-07-14 01:23:53 -07:00
Shruti Patil c9fb5d8aaa Custom bson codec and error report cleanup. 2013-07-12 16:41:41 -07:00
Shruti Patil e9955bf3dd Simplified error types for invalidator. 2013-07-12 00:14:08 -07:00
Shruti Patil bc1ccfac22 Config changes and debug statements. 2013-07-11 17:46:43 -07:00
Shruti Patil 90c2d3e06e Rowcache invalidator and update stream improvements. 2013-07-11 15:09:27 -07:00
Sugu Sougoumarane 8bd9e4d597 support for schema overrides 2013-06-04 14:15:15 -07:00
Sugu Sougoumarane 029eaf8612 match RoundRobin API with ResourcePool and start using ResourcePool 2013-05-14 16:04:23 -07:00
Sugu Sougoumarane 44a999e098 return transaction id in a struct for rpc calls 2013-05-09 14:25:40 -07:00
Sugu Sougoumarane edeec1a84f change mysql API to accept string instead of []byte for queries 2013-04-30 13:00:43 -07:00
Sugu Sougoumarane c7fda52899 add more variables for GetSessionId validation 2013-04-25 13:41:44 -07:00
Sugu Sougoumarane a3c2ce4fd7 bson spot optimizations & test fixes 2013-04-24 12:54:55 -07:00
Sugu Sougoumarane 6fc32af99f new feature: transaction log, and simplified stats urls 2013-04-18 10:34:04 -07:00
Sugu Sougoumarane 85fd9dff0f selflessness and atomic normalization 2013-04-10 15:43:10 -07:00
Sugu Sougoumarane 65b1fd7f94 tweak APIs to make them more robust 2013-04-05 17:19:06 -07:00
Sugu Sougoumarane 32781e4c6c query rules phase 3: feature complete 2013-04-05 13:17:04 -07:00
Sugu Sougoumarane e544e8997a query rules phase 3: rules engine core functionality 2013-04-02 11:55:33 -07:00
Sugu Sougoumarane 085aa919aa query rules phase 1: data structures & convenience functions 2013-03-15 13:15:06 -07:00
Mike Solomon 105e9ee395 break out traceback into its own package 
fix data race on flags
 2013-03-14 12:16:32 -07:00
Alain Jobart b3e7fa6da6 Adding KeyRange to GetSessionId, and enforcing correct values on 
the tabletserver side.

LGTM Mike & Sugu.
 2013-03-11 11:53:08 -07:00
Mike Solomon 81e550ad3e move handy atomic macros to sync2 2013-03-07 19:18:45 -08:00
Sugu Sougoumarane 3e00995696 delete dead code 2013-03-07 14:09:55 -08:00
Shruti Patil f0412f06f4 vt_binlog_server, its api and improvements in update_stream - memory improvements, bug fixes. 2013-02-28 14:52:27 -08:00
Alain Jobart 753b4af2ea Logging query engine state transitions, and using new atomic class. 
LGTM Sugu.
 2013-02-25 16:35:00 -08:00
Sugu Sougoumarane 2ded9d13be more logging and diagnostics for schema reloads 2013-02-22 17:03:49 -08:00
Alain Jobart f571f4ffb1 More efficient memory usage in streaming. 
LGTM Mike, Ric, Sugu.
 2013-02-21 15:59:33 -08:00
Sugu Sougoumarane 806e757605 Record queries executed during transaction for better diagnostics 2013-02-21 10:04:44 -08:00
Sugu Sougoumarane 929c1be6f5 tabletserver bug fix: fast fail on tx pool full. Otherwise, it causes domino effects under high stress. 2013-02-20 21:50:47 -08:00
Sugu Sougoumarane 08f55259f4 More graceful handling of connection error during qfetch 2013-02-01 14:25:31 -08:00
Sugu Sougoumarane 351ee8bb3a tabletserver: qFetch should handle conn pool errors, and guarantee broadcast for consolidation. 2013-01-29 23:34:09 -08:00
Sugu Sougoumarane 3c030c757d stats & diagnostics: query_stats and health checks 2013-01-28 13:33:04 -08:00
Shruti Patil 136063d078 Changes for rowcache_invalidator and the corresponding integration test and some related fixes to tabletserver. 2013-01-25 17:09:32 -08:00
Ric Szopa bcaa39d322 Decouple streamlogger registration from serving logs. 2013-01-09 12:49:24 -08:00
Ric Szopa 86332ed4bb Use fractional seconds uniformly in the stream log. 2013-01-09 12:49:20 -08:00
Ric Szopa cc3c447603 Make streamlog publish unfiltered data. 2013-01-09 12:49:16 -08:00
Alain Jobart d3feb974e8 Adding a simple way to report States to debug/vars, using it for 
sql engine status, zk conn in vttablet, zkocc->zk conns.

LGTM Mike.
 2012-12-26 15:41:42 -08:00
Sugu Sougoumarane ac403f00aa improved error handling, reload catches panics, fix test 2012-12-19 17:48:02 -08:00
Shruti Patil 580a0e2fa1 Changed dbconfigs to tabletserver dependency. 2012-12-12 15:21:25 -08:00
Alain Jobart fc280695ea Making max table count a constant, consistent to 10000. 
LGTM Sugu.
 2012-12-11 15:04:40 -08:00
Sugu Sougoumarane f17182be7a allow bind vars in select list, but log a warning 2012-12-10 21:14:21 -08:00
Alain Jobart 5ed0d42a60 Using time.Second more extensivily. 
LGTM Sugu.
 2012-12-10 12:04:42 -08:00
Sugu Sougoumarane 5c26bf56ae improved state and panic reporting 2012-12-05 16:57:03 -08:00
Sugu Sougoumarane 4dab186aca use a more predictable scheme of obtaining field info for selects 2012-12-05 15:22:24 -08:00
Alain Jobart 3d57d98fb4 Tests now wait for tablets to be serving when required. 
(and various test fixes).

LGTM Mike.
 2012-12-03 12:25:19 -08:00
Ric Szopa 303d6b6065 occ_test runs vttablet 2012-11-28 16:41:43 -08:00
Sugu Sougoumarane dbfde3d6e2 simplify logstats boilerplate code 2012-11-19 16:24:03 -08:00
Sugu Sougoumarane cbffce38d5 prepwork for VM. split SqlQuery into two classes. 2012-11-15 13:47:39 -08:00
Ric Szopa bdb65c58b8 JSON in sensitive data string representation. 2012-11-14 12:53:34 -08:00
Ric Szopa ece66e050e Redact away dbconfig password. 2012-11-14 12:53:32 -08:00
Sugu Sougoumarane a306a055f8 end to end tests for type system & some bug fixes 2012-11-12 11:53:19 -08:00
Sugu Sougoumarane 169f86c653 enforce strict mode & other misc fixes 2012-11-09 11:54:43 -08:00
Alain Jobart f4ef9d3714 Adding a PreflightSchema operation. Using it in ApplySchema on a tablet. 
LGTM Mike / Sugu.
 2012-11-08 11:05:26 -08:00
Sugu Sougoumarane fb806cf4dc tabletserver: introduce a more formal type system 2012-11-05 14:28:37 -08:00
Sugu Sougoumarane 31f5be368a add error stats for unexpected panics so they can be tracked 2012-10-30 14:57:32 -07:00
Sugu Sougoumarane d04ec84572 catch all panics so we don't crash and burn on bugs 2012-10-30 14:27:28 -07:00
Sugu Sougoumarane a037491508 tx pool: if full, wait for connections instead of failing 
disallow queries: accept shutdown reason to return different error categories to clients
 2012-10-26 10:54:02 -07:00
Ric Szopa f4024cdfa5 RPC methods take a context argument. 
* * *
The context object is passed around.
* * *
the context is passed as interface{}
* * *
Bring back old function signatures.
* * *
Context-taking RPC method can declare their arguments using the real context type.
* * *
Use inteface{} for context in rpcplus.
 2012-10-24 12:49:59 -07:00
Alain Jobart 957bd40234 Moving sqlquery API to proto. 
LGTM: Sugu.
 2012-10-15 15:29:42 -07:00
Sugu Sougoumarane de06fdca10 revamped timer 2012-09-27 17:31:19 -07:00
Alain Jobart 912b05e16d Moving three more structs into proto, removing an unused one. 
LGTM Sugu.
 2012-09-24 16:57:23 -07:00
Alain Jobart 1485664c43 Splitting the data structures used in mysql-related RPC into proto submodules. 
LGTM Sugu and Mike.
 2012-09-24 15:14:48 -07:00
Sugu Sougoumarane 1feba3990e tabletserver: allowQueries now retries indefinitely until it can connect to a db. 
tabletserver: removed race conditions & deadlocks around allow & disallowQueries.
 2012-09-20 17:46:35 -07:00
Ric Szopa b24920ae86 Log SqlQuery events using streamlog. 2012-09-19 12:58:50 -07:00
Mike Solomon 8bac1b1f06 add tablet change callback to bridge tablet server and tablet manager 2012-09-19 01:06:46 -07:00
Sugu Sougoumarane 8d927b230b cleaner & more efficient mysql.go 2012-09-17 13:14:17 -07:00
Sugu Sougoumarane 9ac3572bba strip multiple trailing comments instead of just last one for query uniqueness 2012-09-12 11:33:44 -07:00
Alain Jobart 2a115de197 Not putting streaming connections into the active pool, 
so they don't inherit the timeout.

LGTM Sugu.
 2012-09-11 15:58:05 -07:00
Alain Jobart fb70af1311 Renaming a few more 'self' to more meaningfull names. 
LGTM Sugu.
 2012-09-04 16:58:16 -07:00
Alain Jobart 00f740bfa7 Using a different connection pool for streaming SQL queries. 
And making the streaming response size a config variable while
I'm at it.

LGTM Sugu.
 2012-08-31 15:27:42 -07:00
Alain Jobart e4aec22b63 Streaming SQL queries now return multiple responses in one RPC response. 
So we don't use StreamQueryResult, but just QueryResult (simpler
and faster).

Also fixing a bug if using cbson, where the decode_next method
returns -1 instead of the offset at the end of the decode.

LGTMike.
 2012-08-29 17:05:14 -07:00
Alain Jobart 90e0a63589 Code clean-up: renaming a few 'self' into variable names, 
to follow newer code conventions.

LGTM: Sugu
 2012-08-28 16:05:34 -07:00
Alain Jobart 1138c656aa Now using rpcplus package everywhere, and adding support for streaming 
mysql queries using a streaming RPC API. Only the go client side is
in here, the python client code will be next.

LGTM by Mike and Sugu over the last couple weeks.
 2012-08-27 16:26:06 -07:00
Alain Jobart d567446f6a Go-formatting that file. 2012-08-27 14:04:32 -07:00
Sugu Sougoumarane 526a3d7a6b report memcache stats. 2012-08-26 21:33:00 -07:00
Sugu Sougoumarane f42fac26e6 Protocol changes: Change most rpc APIs to use structs. 2012-08-23 16:05:41 -07:00
Alain Jobart dc08f7958f Fixing tabletserver unit test to pass. 
LGTM from Sugu.
 2012-08-17 11:23:59 -07:00
Sugu Sougoumarane 3ff4f463c3 report all time stats in nanoseconds 2012-08-15 14:10:53 -07:00
Sugu Sougoumarane e448f50dcf config revamp: convert dbconfig maps to actual structs 2012-08-08 16:11:15 -07:00
Mike Solomon dc267d8b0c merge 2012-08-02 00:02:03 -07:00
Mike Solomon 5866f155cd gofmt 2012-08-01 23:51:05 -07:00
Ric Szopa aac30617e9 Merge Sugu's changes. 2012-07-31 15:07:54 -07:00
Ric Szopa 5107b7aeff Separate authenticated rpc server. 2012-07-31 14:42:18 -07:00
Sugu Sougoumarane 1440c2fc22 Moving config & dbconfig parsing back to vtocc.go 2012-07-31 14:37:19 -07:00
Sugu Sougoumarane f1843c18c5 New protocol for handling multi-dml updates 2012-06-21 18:28:58 -07:00
Sugu Sougoumarane ad9c110c79 flags reorg: moved some flag initializations to their respective packages. 
port is now specified in the command line instead of config file.
 2012-06-21 16:21:45 -07:00
Sugu Sougoumarane 93669dff3b bug fix in base64 decode 2012-06-13 12:49:51 -07:00
Shruti Patil cb08a7a9a2 Protocol change for control commands to remove the 'Database' field from it, since mysql cannot supply it for 'COMMIT'. 2012-06-11 14:14:12 -07:00
Sugu Sougoumarane a66124fe7c Support for db name verification for row cache invalidation messages. 
Also, changed DDL invalidation to expect base64 encoded strings to avoid binary snafus.
 2012-06-07 14:30:41 -07:00
Ric Szopa 02115e7048 Change the license notice to be shorter and refer to the license file. 2012-06-07 11:55:06 -07:00
Sugu Sougoumarane 97e4e35b64 fix typo in error msg in table_info 
InvalidateForDDL should allow processingin shutdown mode
 2012-06-06 13:49:51 -07:00
Sugu Sougoumarane 3aee0efca2 bug fix: active pool was using the wrong time out value to poll for long running queries. 2012-05-31 15:38:39 -07:00
Sugu Sougoumarane ba7248ed40 bug fix: honor invalidation commits while in shutdown mode. 2012-05-22 16:44:28 -07:00
Mike Solomon 5d11f23016 fix whitespace trimming 2012-05-14 23:01:56 -07:00
Mike Solomon 269bded836 gofmt 2012-05-08 10:34:22 -07:00
Sugu Sougoumarane 30f0fa8f15 tabletserver: ExecuteBatch now returns a QueryResult list 2012-05-14 12:13:26 -07:00
Sugu Sougoumarane f7bb1261bb bug fix: pk column validation for row cache was incorrect. 2012-05-08 16:14:43 -07:00
Sugu Sougoumarane 6e7de66f2e added transaction support for invalidation API. 
changed invalidation API to validate that strings are base64 encoded.
fixed minor dbug/vars reporting problem if vtocc could not initialize correctly.
 2012-05-03 15:11:24 -07:00
Sugu Sougoumarane 16249b4972 bson marshal: Don't use Reserve for string & bytes data, they may be too long. 
resurrected schema reloader. it's still needed.
readable query info error message.
remove ever-changing auto_increment info before hashing table info.
 2012-05-02 23:15:49 -07:00
Sugu Sougoumarane 34e59e452f minor reporting bug fix in query cache. 2012-04-27 15:48:09 -07:00
Sugu Sougoumarane e71dcfe7c8 optimization stagte2: chunked writer for bson 2012-04-25 13:40:45 -07:00
Sugu Sougoumarane 11eb137452 optimzation stage1: 
- Cache & reuse field info for selects
- Optimization in QueryBson for reading
- Move the defer closure out of Execute into its own function
- Delete unused execCacheResult
 2012-04-24 11:22:23 -07:00
Sugu Sougoumarane 929765a059 bug fix: Handle the case of cardinality being NULL. 2012-04-17 14:52:10 -07:00
Sugu Sougoumarane 2cfad8fddf rowcache: change encoding from bson to a custom packed binary format. 2012-04-16 14:47:05 -07:00
Sugu Sougoumarane 7e1a1e4ff7 roundrobin: Revamped to handle deadlocks & race conditions. 
snitch: fix panic creator.
token: Handle error of just ':' as bind var name.
sqlquery: trap random panics to log the query that caused them.
 2012-04-07 16:25:47 -07:00
Sugu Sougoumarane 3d02c8ad78 support for auto-increment & default value columns in pk 2012-04-03 00:51:50 -07:00
Sugu Sougoumarane e205ecfd09 track invalidations by table 2012-04-02 21:46:00 -07:00
Sugu Sougoumarane e2e3e0ef00 copyright deviolation 2012-04-01 21:58:56 -07:00
Sugu Sougoumarane 1f39441929 rowcache: simpler, more robust, and less chatty invalidation scheme. 2012-04-01 21:56:48 -07:00
Sugu Sougoumarane 15c1443004 schema: handle case where cardinality==0 
tabletserver: copy read-only subquery results before normalizing
tabletserver: add function to validate the invalidation keys
tablet_server: bug fix. was not disabling cache if pk types were invalid
 2012-04-01 17:14:58 -07:00
Sugu Sougoumarane cc4d3d2878 tabletserver: InvalidateForDDL 
memcache: export stats
 2012-03-29 16:31:10 -07:00
Sugu Sougoumarane 5c5695e7ec use fnv hash to generate unique prefixes for row cache 
fix some tests that were failing on slow dbs
 2012-03-28 14:23:04 -07:00
Sugu Sougoumarane 35229bf0c1 shorten the life of invalidation objects in memcache 2012-03-27 15:59:19 -07:00
Sugu Sougoumarane db2be9f119 skip pk row cache optimization if there's an order by clause 
display query plans for query_cache stats
rowcache: limit size to 4k
 2012-03-27 12:19:39 -07:00
Sugu Sougoumarane be11a853c0 sqlquery: change invalidation function to use new key encoding scheme. 2012-03-26 13:18:37 -07:00
Sugu Sougoumarane 893b05f289 execution: include pk columns to indexes for covering index queries. 2012-03-26 00:04:47 -07:00
Sugu Sougoumarane 82e589636b schema: track cardinality of index columns 
parser: added support for USE INDEX syntax
execution: Improved plan printing
execution: Use cardinality for more accurate index selection
execution: Add use index to subqueries
tests: More readable exec_cases.txt
 2012-03-25 22:20:08 -07:00
Sugu Sougoumarane bd24342935 memcache: bug fix on large gets 
schema: disallow caching for tables with fixed length binary columns
update_stream: allow char columns in primary key
inserts: allow inserts with unspecified columns lists
more reliable ddl throttle scheme
stats: track row not found separately from misses
 2012-03-24 12:57:06 -07:00
Sugu Sougoumarane db4497300d row cache: unit tests done. 2012-03-22 01:01:44 -07:00
Sugu Sougoumarane 6831be96fa rowcache: almost done! 2012-03-20 17:33:11 -07:00
Sugu Sougoumarane 69b96741f9 refactor: created new reusable pools package by pulling code out of connection pools. 2012-03-16 17:07:27 -07:00
Sugu Sougoumarane a082e88409 updated to go's new build system and weekly.2012-03-04 2012-03-11 17:53:40 -07:00
Sugu Sougoumarane 2899c45aea tabletserver: big change. Fix all unprotected reads, standardize APIs and naming conventions. 2012-03-08 15:13:53 -08:00
Sugu Sougoumarane 412916748b Remove generated files _testmain.go, and undo manual changes of test files. 2012-02-29 14:10:06 -08:00
Sugu Sougoumarane 72a862e6b3 initial source code check-in. 2012-02-24 23:30:03 -08:00