vitess-gh/go/mysql
Rohit Nayak 86e0cf822b
Partial Movetables: allow moving a table one shard at a time (#9987)
* Initial Implementation of partial movetables.  Implement cross-shard queries that obey shard routing rules.

Refactor rss updation with routed shard and add to multi col resolution. Run vtgate and reservedconn tests for both regular and with a partially moved shard.

Add ability to run tests with partial keyspaces by adding to a list in the CI generator. Implement that in test.go and cluster setup. Revert previous iteration of running partial keyspace tests in parallel. One test is failing: TestSetSystemVarAutocommitWithConnError.

Add partial vtgate misc test

Add partial keyspace tests to vtgate transaction

Add vtgate queries partial keyspace tests

Review and fix skipped tests where applicable. Refactor to allow conditional skipping of tests for partial keyspaces only so that other tests work exactly as earlier without specifying the DbName, for example

Add move shards example

Cache routing tables and minor refactor. Move move_shards example

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Add lookup index to move shards example. Comment error logs while trying to delete from missing _vt.vdiff table

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Remove move shards demo example

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Add vtgate flag to enable shard routing. Update tests to get vtparams based on partial keyspace setting, so that global routing continues to be tested as earlier without DbName being specified

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

To create subsequent partial movetables workflows we should ignore previous partial movetables (now frozen)

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Remove vdiff2 deletes since they were failing unit tests etc

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Use vtgate flag to enable shard routing rules

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Fix incorrect test setup

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Expect modified frozen query

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

set vtgate param --enable_shard_routing in partial movetables test

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Fix failing tests

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Fix flags test

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Use VTParams helper for partial keyspace tests

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Specify vtParams.DBName for partial keyspace queries

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Self-review. Remove bypass caching.

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Report partial SwitchTraffic results

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Try to address most of the upstream PR review comments

From: https://github.com/vitessio/vitess/pull/9987

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Enforce proper routing for partial migrations

Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Minor changes after self review

Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Account for local timezone in check for vdiff progress

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Update go version in workflows. Fix gofmt issue

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Fix failing CI tests

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Comment failing test for partial keyspaces

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Fix TestVtGateVtExplain by specifying dbname

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Fix leftover debug code

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Move partial keyspace tests to a single shard. Refactor test.go to use multiple config.json files

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Address review comments

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Address review comments

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Improve test skip comments

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* gofmt

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Fix failing tests

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Address review comments

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Try TestFunctionInDefault for partial keyspaces. Update skip comment.

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Delete temporary log lines

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: Matt Lord <mattalord@gmail.com>
2022-09-23 12:15:12 +02:00
..
collations [cli] [servenv] Migrate `--mysql_server_version` to pflag (#11190) 2022-09-09 11:29:31 -04:00
endtoend Upgrade to Ubuntu 20.04 for endtoend tests (#11113) 2022-08-30 15:51:56 +05:30
fakesqldb Partial Movetables: allow moving a table one shard at a time (#9987) 2022-09-23 12:15:12 +02:00
fuzzdata added fuzzer test case for handleNextCommand 2021-05-27 14:58:42 +05:30
ldapauthserver Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
vault Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
auth_server.go Wrong length creating the buffer needed to ask for full auth (#10767) 2022-08-29 16:18:29 +05:30
auth_server_clientcert.go [go/mysql/*] Move all authserver–related flags off of global flagset (#10752) 2022-07-25 15:58:38 -07:00
auth_server_clientcert_test.go [go/mysql/*] Move all authserver–related flags off of global flagset (#10752) 2022-07-25 15:58:38 -07:00
auth_server_none.go mysql: Pass mysql.Conn through {Hash,PlainText,Caching}Storage interfaces 2021-11-18 11:29:56 -08:00
auth_server_static.go Move go/mysql flags to pflags (#11272) 2022-09-23 10:09:35 +05:30
auth_server_static_flaky_test.go tests: remove metrics workflow, _flaky on some tests, add retries 2021-10-22 15:16:56 -07:00
auth_server_test.go Refactor authentication server plugin mechanism 2021-07-21 15:25:54 +02:00
binlog_dump.go add copyright header 2022-02-01 14:14:59 +02:00
binlog_event.go Binlog event parsing: better analysis and support for semi-sync 2022-01-30 10:44:55 +02:00
binlog_event_common.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
binlog_event_common_test.go fix unit tests 2022-01-31 12:16:21 +02:00
binlog_event_filepos.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
binlog_event_json.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
binlog_event_json_test.go all: gofmt -w -r 'interface{} -> any' src 2022-03-22 09:39:55 +01:00
binlog_event_make.go rename and deprecate RPCs, rename vars and files 2020-07-02 11:22:18 -07:00
binlog_event_make_test.go User defined and sys variables (#10547) 2022-06-20 13:52:11 +02:00
binlog_event_mariadb.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
binlog_event_mariadb_test.go fix unit tests 2022-01-31 13:13:32 +02:00
binlog_event_mysql56.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
binlog_event_mysql56_test.go fix unit tests 2022-01-31 13:13:32 +02:00
binlog_event_rbr.go VReplication: Handle DECIMAL 0 Value Edge Case (#11212) 2022-09-13 15:21:45 -04:00
binlog_event_rbr_test.go Take MySQL column type into account in vstreamer 2021-12-08 12:10:54 -05:00
binlog_event_test.go protobuf: upgrade 2021-05-19 11:13:59 +02:00
charset.go add license in /go file 2019-10-21 22:30:02 +08:00
client.go Fix client session state tracking logic (#10871) 2022-07-28 19:45:34 +05:30
client_test.go unit test: fix mysql tests to run on MacOS (#10613) 2022-06-30 11:11:59 +02:00
conn.go Move go/mysql flags to pflags (#11272) 2022-09-23 10:09:35 +05:30
conn_flaky_test.go Fix OK packet parsing logic (#11176) 2022-09-07 11:53:58 +05:30
conn_params.go mysql: add a flag to handle EnableQueryInfo 2022-03-03 17:43:49 +01:00
conn_params_test.go Add additional options for configuring SSL modes as a client 2021-08-04 12:16:22 +02:00
constants.go Implement date, time and timestamp literals (#10921) 2022-08-08 15:12:29 +05:30
constants_test.go Address review feedback 2021-10-19 15:28:39 -07:00
doc.go add license in /go file 2019-10-21 22:30:02 +08:00
encoding.go fix COM_STMT_PREPARE_OK packet 2020-11-12 10:07:35 +05:30
encoding_test.go Fix prepared statment buffer overwriting bug 2019-12-12 15:39:47 -08:00
filepos_gtid.go all: gofmt -w -r 'interface{} -> any' src 2022-03-22 09:39:55 +01:00
filepos_gtid_test.go Fixes bug in filepos flavor 2020-01-10 13:15:12 -08:00
flavor.go Address review comments 2022-07-27 18:24:47 -04:00
flavor_filepos.go Adds RPCs to vttablet that vtorc requires (#10464) 2022-06-20 10:53:04 +05:30
flavor_filepos_test.go Adds RPCs to vttablet that vtorc requires (#10464) 2022-06-20 10:53:04 +05:30
flavor_mariadb.go Adds RPCs to vttablet that vtorc requires (#10464) 2022-06-20 10:53:04 +05:30
flavor_mariadb_binlog_playback.go added mariadb flavor 2021-02-06 01:03:42 +05:30
flavor_mariadb_test.go Adds RPCs to vttablet that vtorc requires (#10464) 2022-06-20 10:53:04 +05:30
flavor_mysql.go Improve schema reload performance by pre-filtering joined rows. (#11043) 2022-08-24 14:17:32 -06:00
flavor_mysql_test.go Adds RPCs to vttablet that vtorc requires (#10464) 2022-06-20 10:53:04 +05:30
flavor_mysqlgr.go Add underlying flavor capability and leverage in unit tests 2022-07-27 02:22:40 -04:00
flavor_mysqlgr_test.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
flavor_test.go Add underlying flavor capability and leverage in unit tests 2022-07-27 02:22:40 -04:00
gtid.go all: gofmt -w -r 'interface{} -> any' src 2022-03-22 09:39:55 +01:00
gtid_set.go added last method to gtid 2020-06-29 00:17:39 +05:30
gtid_test.go all: gofmt -w -r 'interface{} -> any' src 2022-03-22 09:39:55 +01:00
handshake_test.go test: use `T.TempDir` to create temporary test directory (#10433) 2022-06-06 06:47:15 +02:00
innodb_constants.go Refactoring 2022-07-27 14:37:39 -04:00
mariadb_gtid.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
mariadb_gtid_test.go all: gofmt -w -r 'interface{} -> any' src 2022-03-22 09:39:55 +01:00
mysql56_gtid.go all: gofmt -w -r 'interface{} -> any' src 2022-03-22 09:39:55 +01:00
mysql56_gtid_set.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
mysql56_gtid_set_test.go The difference between two identical GTIDSet values should be empty, not 2021-08-30 14:43:34 -07:00
mysql56_gtid_test.go all: gofmt -w -r 'interface{} -> any' src 2022-03-22 09:39:55 +01:00
mysql_fuzzer.go mysql: Add UnimplementedHandler struct 2022-01-18 12:12:49 -08:00
mysql_fuzzer_test.go refactor: move from io/ioutil to io and os package 2021-10-05 08:05:23 +08:00
primary_status.go naming: remove MasterStatus proto and use PrimaryStatus everywhere 2021-07-21 10:17:58 -07:00
query.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
query_benchmark_test.go Move go/mysql flags to pflags (#11272) 2022-09-23 10:09:35 +05:30
query_test.go Run latest gofmt on everything & address linter warnings (#11008) 2022-08-16 13:53:42 -07:00
replication.go refactor: moving constants to appropriate file 2022-01-31 12:16:47 +02:00
replication_constants.go Binlog event parsing: better analysis and support for semi-sync 2022-01-30 10:44:55 +02:00
replication_position.go docs: added comments explaining the functioning of the AtLeast function 2022-01-05 09:14:24 +05:30
replication_position_test.go feat: fix Atleast function to handle both nils properly, since the underlying contains interface doesn't 2021-12-17 19:27:20 +05:30
replication_status.go Simply Replication Status proto conversions (#10926) 2022-08-08 19:52:38 +05:30
replication_status_test.go Manage MySQL Replication Status States Properly (#9853) 2022-03-11 09:56:40 -08:00
replication_test.go Binlog event parsing: better analysis and support for semi-sync 2022-01-30 10:44:55 +02:00
schema.go Improve performance of the `DetectSchemaChange` query by only accessing `information_schema.columns` once. (#10416) 2022-06-07 11:15:08 +05:30
server.go Binlog event parsing: better analysis and support for semi-sync 2022-01-30 10:44:55 +02:00
server_flaky_test.go Move go/mysql flags to pflags (#11272) 2022-09-23 10:09:35 +05:30
sql_error.go Implement date, time and timestamp literals (#10921) 2022-08-08 15:12:29 +05:30
sql_error_test.go PR #6630 translated gRPC error message to ERNetPacketTooLarge, but only 2021-09-20 12:24:23 -07:00
streaming_query.go Fix client session state tracking logic (#10871) 2022-07-28 19:45:34 +05:30