* [grpclient] Migrate `--grpc_auth_static_client_creds` to pflag
This was missed in #10806.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* [grpcclient] Migrate test setters for `--grpc_auth_static_client_creds`
- Relates to #10934.
- Relates to #10979.
- Closes#10891.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
This:
- Corrects an initial migration of the `DualFormat*` functions to
_actually_ install the flag on the passed in flagset, rather than the
global one.
- Updates `StringListVar` to fully-implement `pflag.Value`, mimicking
pflag's `StringSlice` implementation for this method.
- Removes the compile-time type-check on `StringSetFlag` implementing
`flag.Value` (which is a subset of `pflag.Value` anyway), as well as in
the test code.
Closes#11296.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* feat: remove cli code which isn't used in vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: port over the /api/problems endpoint to the http server of VTOrc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: added comments to api/problems/ endpoint explaining the future scope
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove api endpoints registered in old VTOrc ui
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove gracefulTakeover tests since the api is removed and add tests for the api/problems API
Signed-off-by: Manan Gupta <manan@planetscale.com>
* refactor: fix license year and imports
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add apis for disabling and enabling global recoveries
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add api for reading replication-analysis
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add health api as well
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: add tests for the api's ported over
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: add liveness endpoint to tests too
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: return an internal server error when VTOrc isn't healthy
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: reduce flakiness in api tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
* [vtctlclient] Migrate `vtctl_client_protocol` to pflag
Closes#10892.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* forgot to update testdata
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Move vttlstest to pflag
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Address review comments
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Intermediate changes, before going full pflag
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Move vttlstest to Cobra
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Minor improvements after external and self review
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Correct Use for createCACmd
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Correct CLI syntax in TestSecure e2e test
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Rotate binary log before for snapshot connections
Rotate the binary logs to limit the GTID auto positioning overhead.
This is needed as the currently open binary log (which can be up to 1G in size
by default) will need to be scanned and empty GTID events will be streamed for
those GTIDs in the log that we are skipping. In total, this can add a lot of
overhead on both the mysqld instance and the tablet.
Rotating the log ensures that we are processing a fresh binary log that will
be minimal in size and GTID events.
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Don't cause VStream failure if we can't rotate the binary log
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Limit binlog flushes and add e2e test
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Minor changes after self review
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add flag and Get/Set methods for the rotation threshold
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Use dashes for new flags
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Minor changes after self review
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Setup 2 healthy replicas, confirm semi-sync and repl state
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Cleanup after self review
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Reduce backup job time so that transform at least gets a chance to run
Signed-off-by: Matt Lord <mattalord@gmail.com>
* More reliably clear out tablet data
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* vtcombo mutates options, make a copy to avoid this
Signed-off-by: Olga Shestopalova <oshestopalova@hubspot.com>
* move copying and add e2e vtcombo test
Signed-off-by: Olga Shestopalova <oshestopalova@hubspot.com>
* undo incorrect comment update
Signed-off-by: Olga Shestopalova <oshestopalova@hubspot.com>
* fix import order
Signed-off-by: Olga Shestopalova <oshestopalova@hubspot.com>
Signed-off-by: Olga Shestopalova <oshestopalova@hubspot.com>
Co-authored-by: Olga Shestopalova <oshestopalova@hubspot.com>
* Add mysql_server_pool_conn_read_buffers flag and thread through to listener and connection
Signed-off-by: Brian Ramos <brirams@users.noreply.github.com>
* review feedback: code style
Signed-off-by: Brian Ramos <brirams@users.noreply.github.com>
* fix endtoend test
Signed-off-by: Brian Ramos <brirams@users.noreply.github.com>
* run benchmarks with read buffer pooling, too
Signed-off-by: Brian Ramos <brirams@users.noreply.github.com>
* update flag name to match new conventions
Signed-off-by: Brian Ramos <brirams@users.noreply.github.com>
* update 15.0.0 summary with reference to new flag
Signed-off-by: Brian Ramos <brirams@users.noreply.github.com>
* an attempt at cleaning up branch for rebasing
Signed-off-by: Brian Ramos <brirams@users.noreply.github.com>
Signed-off-by: Brian Ramos <brirams@users.noreply.github.com>
* [cli] Migrate `vtctld` flags to `pflag`
Closes#11282.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* add new flag to transition to proper DurationVar flags
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* add vtctlclient help to endtoend data
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* [cmd/vtctlclient] Migrate to pflag
Closes#11281.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* feat: add vttestserver flags
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix parsing of flags in vttestserver to use pflags
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix parsing of flags that take in lists of strings
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: update flags help output
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix tests to not pass the same argument twice
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: fix help output after merging main
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
* Initial Implementation of partial movetables. Implement cross-shard queries that obey shard routing rules.
Refactor rss updation with routed shard and add to multi col resolution. Run vtgate and reservedconn tests for both regular and with a partially moved shard.
Add ability to run tests with partial keyspaces by adding to a list in the CI generator. Implement that in test.go and cluster setup. Revert previous iteration of running partial keyspace tests in parallel. One test is failing: TestSetSystemVarAutocommitWithConnError.
Add partial vtgate misc test
Add partial keyspace tests to vtgate transaction
Add vtgate queries partial keyspace tests
Review and fix skipped tests where applicable. Refactor to allow conditional skipping of tests for partial keyspaces only so that other tests work exactly as earlier without specifying the DbName, for example
Add move shards example
Cache routing tables and minor refactor. Move move_shards example
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Add lookup index to move shards example. Comment error logs while trying to delete from missing _vt.vdiff table
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Remove move shards demo example
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Add vtgate flag to enable shard routing. Update tests to get vtparams based on partial keyspace setting, so that global routing continues to be tested as earlier without DbName being specified
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
To create subsequent partial movetables workflows we should ignore previous partial movetables (now frozen)
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Remove vdiff2 deletes since they were failing unit tests etc
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Use vtgate flag to enable shard routing rules
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Fix incorrect test setup
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Expect modified frozen query
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
set vtgate param --enable_shard_routing in partial movetables test
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Fix failing tests
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Fix flags test
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Use VTParams helper for partial keyspace tests
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Specify vtParams.DBName for partial keyspace queries
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Self-review. Remove bypass caching.
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Report partial SwitchTraffic results
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Try to address most of the upstream PR review comments
From: https://github.com/vitessio/vitess/pull/9987
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Enforce proper routing for partial migrations
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Minor changes after self review
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Account for local timezone in check for vdiff progress
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Update go version in workflows. Fix gofmt issue
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Fix failing CI tests
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Comment failing test for partial keyspaces
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Fix TestVtGateVtExplain by specifying dbname
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Fix leftover debug code
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Move partial keyspace tests to a single shard. Refactor test.go to use multiple config.json files
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Address review comments
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Address review comments
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Improve test skip comments
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* gofmt
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Fix failing tests
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Address review comments
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Try TestFunctionInDefault for partial keyspaces. Update skip comment.
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Delete temporary log lines
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: Matt Lord <mattalord@gmail.com>
* Make mysql_server_flush_delay vtgate only pflag
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add comment for go test flag
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Address review comments
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Remove mysql_server_flush_delay from vtorc help output
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* feat: fix 101_initiail_cluster to not run PRS, but instead wait for tablets to be up and primary promotion by vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix remaining binaries in local examples to not call ISP at all and instead rely on VTOrc to elect a primary
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix region-sharding example too to not call ISP
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: use _ instead of i in for loop since i is unused
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
* Fix complex predicates being pulled into `ON` conditions for `LEFT JOIN` statements.
Also convert `LEFT JOIN` statements with complex predicates into `INNER JOIN` when possible.
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
* Allow outer join simplification for `IS NOT NULL` expressions.
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
* feat: rewrite column names in HAVING
Signed-off-by: Andres Taylor <andres@planetscale.com>
* fix: make sure to figure out dependencies of count(*) when used in HAVING
Signed-off-by: Andres Taylor <andres@planetscale.com>
* test: update expectations
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
* docs: fix logging in vtorc http
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add servenv and required flags to vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add vtorc port to local and region sharding example
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: add vtorc to flag help test
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: simplify vtorc parsing of flags
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add redirect for unrecognized paths to /debug/status to vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add a component to debug/status page of vtorc to display recent recoveries
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: simplify a condition in vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix command used for invoking vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove adding help flag and let servenv handle it
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add vtorc servenv page to docker-compose example too
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add prometheus plugin like we do for other binaries
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: change vtorc debug port to not conflict with vttablet's port
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix year in file license
Signed-off-by: Manan Gupta <manan@planetscale.com>
* docs: add summary docs
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Rameez Sajwani
Andrew Mason
Manan Gupta
Shlomi Noach
Harshit Gangal
FlorentP
Matt Lord
Rohit Nayak
Andres Taylor
Deepthi Sigireddi
Olga Shestopalova
Brian Ramos
Frances Thai
Arthur Schreiber