* Prevent orphaned VDiffs in two ways...
1. When opening the engine, restart any vdiffs that are in the
started state as this indicates it did not complete and was
unable to save the final state and must be restarted.
2. When a vdiff run fails, retry saving the error state with an
exponential backoff until the engine shuts down. This way
the normal retry mechanism will kick in OR #1 will kick in
when the engine is next opened on the primary tablet.
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Handle failures before vdiff_table records are created
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add more ephemeral client errors
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Show vdiff state of error even if no vdiff_table records
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Minor cleanup
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add vdiff2 unit tests
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add unit test for retry
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Small cleanup
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Addressing review comments and other improvements
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Use warning log for ... warnings :-)
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Minor touch ups
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* bugfix: allow predicates on top of derived tables to be handled without dependencies
Signed-off-by: Andres Taylor <andres@planetscale.com>
* stop normalizer from changing literals into bindvars
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Co-authored-by: Andres Taylor <andres@planetscale.com>
* make sure planner works well with more than 64 tables
Signed-off-by: Andres Taylor <andres@planetscale.com>
* semantics: use an immutable bitset
Signed-off-by: Vicent Marti <vmg@strn.cat>
* bitset: add documentation
Signed-off-by: Vicent Marti <vmg@strn.cat>
* bitset: add license
Signed-off-by: Vicent Marti <vmg@strn.cat>
* use IsEmpty and NonEmpty more
Signed-off-by: Andres Taylor <andres@planetscale.com>
* rename file
Signed-off-by: Andres Taylor <andres@planetscale.com>
* bitset: better documentation
Signed-off-by: Vicent Marti <vmg@strn.cat>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Vicent Marti <vmg@strn.cat>
Co-authored-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Vicent Marti <vmg@strn.cat>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Vicent Martí <42793+vmg@users.noreply.github.com>
Co-authored-by: Andres Taylor <andres@planetscale.com>
* Update test runners to run all tests including outside package
Sometimes it's necessary to have a separate _test package inside another
package. Our current test runner doesn't run such tests though, leading
to tests being broken without us knowing.
Worst case it hides real bugs. So we should always ensure to also run
these tests as well.
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
* Fix tests
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
On MariaDB, it's possible that an OK packet has
`SERVER_SESSION_STATE_CHANGED` set to true, but it has no additional
state change information as the end of the packet is reached.
We should not treat this as an error condition and treat this as a
proper OK packet.
Fixes https://github.com/vitessio/vitess/issues/11795
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
* Switch ApplySchema `--sql` argument to be `StringArray` instead of `StringSlice`
Fixes#11785.
See [this comment](https://github.com/vitessio/vitess/issues/11785#issuecomment-1323641942)
for an explanation.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* add endtoend test
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Co-authored-by: Andrew Mason <andrew@planetscale.com>
* Addition of a CI tool to detect dead links in test/config.json (#11668)
* Addition of a tool to detect unrequireed CI clusters in test/config.json
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* move mysql80 derived to queries/derived in config.json
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* test: clean the user table as well
Signed-off-by: Andres Taylor <andres@planetscale.com>
* Fix flakyness in TestDerivedTableWithHaving
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Skip TestDerivedAggregationOnRHS
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Go mod tidy
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: Andres Taylor <andres@planetscale.com>
* apply go mod tidy
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: Andres Taylor <andres@planetscale.com>
* improve handling of ORDER BY/HAVING rewriting
Fixes#11641
Signed-off-by: Andres Taylor <andres@planetscale.com>
* Go fmt file
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
* Fix ordering when error happens during stream setup (#11592)
* Fix ordering when error happens during stream setup
When we already have an error during the stream setup, that errors needs
to be returned immediately.
Otherwise the stream will already be started and the error message is
never returned to the client and it silently looks like a success even
if it's a failure.
This in turn can result in an empty result for example when an error
would have been expected.
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
* Add a bunch of testing to verify stream execute behavior
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
* Send errors in stream instead of a grpc error from streaming rpcs when transaction or reserved connection is acquired (#11656)
* wip
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* feat: send errors in stream instead of a grpc error from streaming rpcs
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>
* VTexplain topology only uses serving shards
This addresses isse #11632 , which
causes vtexplain to sometimes give bad results if the keyspace is being
resharded, because sometimes it picks source shards and other times target
shards, for routing the query.
The issue is that the `VTExplain.buildTopolog()` adds both source and
destination shards to the map that holds shards per keyspace, when only one of
them is actually serving traffic at any point in time. Later on, vtexplain
loops over this map. Because looping over the map gives a non-deterministic
order, sometimes the results are correct, and sometimes incorrect - that is,
sometimes it gives the result of the shard that is serving, and other times,
the shard that is not serving.
This change ensures that only the serving shards are added to the shards per
keyspace map, thus avoiding the incorrect vtexplain.
Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
* This addresses issue #11632 , which
causes vtexplain to sometimes give bad results if the keyspace is being
resharded, because sometimes it picks source shards and other times target
shards, for routing the query.
The issue is that the VTExplain.buildTopolog() adds both source and
destination shards to the map that holds shards per keyspace, when only one of
them is actually serving traffic at any point in time. Later on, vtexplain
loops over this map. Because looping over the map gives a non-deterministic
order, sometimes the results are correct, and sometimes incorrect - that is,
sometimes it gives the result of the shard that is serving, and other times,
the shard that is not serving.
This change ensures that only the serving shards are added to the shards per
keyspace map, thus avoiding the incorrect vtexplain.
Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
* Fix check_make_vtadmin_authz_testgen
Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
* empty commit to trigger CI
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
Co-authored-by: Andres Taylor <andres@planetscale.com>
* Do not multiply `AggregateRandom` in `JOIN`s (#11633)
* Do not multiply AggregateRandom in JOIN
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Update the releases notes with new known issue
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Addition of the release notes for v15.0.1
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Fix format issue in the release notes summary
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* set sql mode in TestAggregateRandom
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Instead of relying on a specific version check, detect if the new redo
log location is present or not.
This makes the backup logic independent from the version check against
MySQL. With this change, 8.0.30 can be backed up just as well as any
other version.
Solves the backup part of
https://github.com/vitessio/vitess/issues/11554 by removing the
dependency on the version check. There might be still other places where
that issue can crop up, but it solves the immediate breaking issue with
backups.
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
* Update logic for parsing test flags to run unit tests within GoLand. Parse test flags in vtgate to allow running unit tests
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Minor changes after self-review
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* don't rewrite HAVING predicates that use table columns
Signed-off-by: Andres Taylor <andres@planetscale.com>
* Revert the changes made in #11306
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
* Fix early rewriter test
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Co-authored-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Co-authored-by: Andres Taylor <andres@planetscale.com>
* Only perform vindex logic for deletes against sharded keyspaces.
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
* Add a testcase covering deletes on an unsharded reference table.
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
* Check if target keyspace is unsharded instead of checking opcode.
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
Co-authored-by: Arthur Schreiber <arthurschreiber@github.com>
* Allow version to be accessible via the -v shorthand
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* update help text
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* fix nonidempotence for tests
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Signed-off-by: Andrew Mason <andrew@planetscale.com>
I accidentally stumbled over the behavior outlined in
`flag.PrintDefaults` [1], which `pflag` replicates, specifically:
> The listed type, here int, can be changed by placing a back-quoted
> name in the flag's usage string; the first such item in the message is
> taken to be a parameter name to show in the message and the back quotes
> are stripped from the message when displayed.
[1]: https://pkg.go.dev/flag#PrintDefaults.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Co-authored-by: Andrew Mason <andrew@planetscale.com>
* feat: added test for vtorc not being able to handle mutliple failures and fix it
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: fix code to delete rdonly tablet from the correct list
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: added failing e2e test
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* log txID and reserveID in stream execute
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* fix: maintain list of qd per key on the map and check for current connection id while removing
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* added additional comments
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* Plan order by Count()
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Clean up the new aggregation E2E test
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Push more order by needs to the select list
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Remove unrequired code in TestOrderByCount
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* remove unwanted directory
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Use JSON for plan tests
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* move vschemas into single directory
Signed-off-by: Andres Taylor <andres@planetscale.com>
* indentation
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: FlorentP <35779988+frouioui@users.noreply.github.com>
* Testing....
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Setup reverse routing rules at the start of the first partial movetables. Enhance e2e test to test for this as well as switch the remaining shard so that all shards are switched
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Update display representation of state for partially moved keyspaces
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Test improvements
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Cleanup shard routing rules on cancel/complete
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Cleanup intermediate changes
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Fix DryRun TrafficSwitcher and its tests
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Support concurrent partial movetables for N keyspaces
And other minor changes after self review
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Avoid unnecessary save + rebuild steps
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add secondary ordering for shard routing rules on to_keyspace
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Minor changes after self review
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: Rohit Nayak <rohit@planetscale.com>
* feat: call the logic for waiting for shard locks on synchronous on-term hooks and add tests for it
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix comment for acceptSighupSignal function
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>
* When restarting workflow, use new context
Otherwise the restart will also fail due to the
expired/canceled context:
E0929 14:06:08.618848 85788 vdiff.go:247] Could not restart workflow commerce2customer: rpc error: code = Canceled desc = context canceled, please restart it manually
E0929 14:06:08.619983 85788 vtctl.go:2630] vdiff returning with error: rpc error: code = Canceled desc = context canceled
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add retry when restarting workflows on ephemeral errors
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Correct mistake that prevented compilation :face-palm:
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Move towards MySQL 8.0 as the default template generation
This upgrades the remaining things to Ubuntu 20.04 and makes MySQL 8.0
the default we run tests against. We still have tests for MySQL 5.7 but
those are now explicitly opted into.
This should finish up the Ubuntu 20.04 upgrade and also makes things
easier for the future when we need to upgrade again.
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
* CI: rename shard vtorc_8.0 to vtorc_5.7, change expected test output for 8.0
Signed-off-by: deepthi <deepthi@planetscale.com>
* CI: increase timeout for 8.0 tests on the actual test step from 30 to 45 mins
Signed-off-by: deepthi <deepthi@planetscale.com>
* CI: increase timeout to 45 minutes for mysql57 tests too. We really only need this for vtorc, but I've made the change to the template so all tests get it.
Signed-off-by: deepthi <deepthi@planetscale.com>
* CI: fix vtorc test to work with both 5.7 and 8.0
Signed-off-by: deepthi <deepthi@planetscale.com>
* CI: missed docker flag in mysql57 template, one more fix to vtorc test
Signed-off-by: deepthi <deepthi@planetscale.com>
* removing spaces from pb file
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
* removing spaces in pb file part 2
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
Co-authored-by: deepthi <deepthi@planetscale.com>
Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com>
* flags etc: delete old flags and stats, add deprecation notice to release notes
Signed-off-by: deepthi <deepthi@planetscale.com>
* vrepl: fix tests to run SwitchReads with tablet_types flag instead of deprecated tablet_type
Signed-off-by: deepthi <deepthi@planetscale.com>
* pitr: fix tests to run SwitchReads with tablet_types flag instead of deprecated tablet_type
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
* Migrate dbconfigs to pflag
And remove deprecated flags
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Remove no longer needed function
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Address review comments and test failures
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Revert this bit as it changed behavior and caused some failures
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Fix vttestserver unit tests
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Fix vttestserver unit tests
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Update vttestserver help output
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* feat: remove configurations that aren't needed like raft and initialization of database
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: use waitreplicastimeout for all timeouts related to waiting for execution on a replica
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove agents in vtorc since we don't use them
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove more unused configurations
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: cleanup post and pre processes that VTOrc doesn't intend to support
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove unused code and config for shell process
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove unused configurations
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: convert flags to pflag architecture
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove cli flags which aren't required
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove more unused configs
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove the old UI of orchestrator and cleanup app and http packages
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove web/vtorc files and fix Dockerfiles to not try to copy them
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix all the examples to not pass in the ListenAddress and not export port for the old UI
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove raft code and command applier
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove unused code and configs
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove more unused code
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove http configs that are no longer required
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove more unused configs and dead code
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove more unused configs
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove access tokens since they are unused
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove RecoveryPeriodBlockMinutes which was only for backward compatibility
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove unused package
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove replicas in instance since they weren't used anywhere
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: copy over vtorc things that vtgr was using to remove vtgr's dependency on vtorc package
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: change restart function to use tmc rpc calls
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove detection queries for vtorc since we read these from topo-server
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: improve read topology test to also verify the errantGTID set
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: use internal function to find difference in gtid sets instead of using the MySQL connection
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove population of group replication information
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove configs related to connecting to MySQL instances since it is no longer required
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove code to connect to MySQL backend since VTOrc now only uses SQLite3
Signed-off-by: Manan Gupta <manan@planetscale.com>
* cleanup: go mod tidy
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove unused/redundant flags, we are down to 0 :)
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove filtering configs
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove write buffering and associated config parameters
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove more miscellaneous configs
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove one more rejection parameter
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: move discovery queue parameters to be constants
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: move some more configs to be constants
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add flags for all the configurations we have kept
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix imports in main.go
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: fix vtorc test output after so many changes
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add release-notes docs for config changes
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove index definitions for tables that are already deleted
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add API removal documentation as well to release notes
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove duplication of debug pages in release notes
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: use some flags in e2e test to verify they work
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: use sliceVar for the flag definition of cluster_to_watch
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
* Improved logic for subquery merging
This is a squash of a number of commits created by Arthur.
Co-authored-by: Arthur Schreiber <arthurschreiber@github.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
* refactor: use existing functionality in ExtractedSubquery
Signed-off-by: Andres Taylor <andres@planetscale.com>
* refactor: extract the ColName only once
Signed-off-by: Andres Taylor <andres@planetscale.com>
* comments
Signed-off-by: Andres Taylor <andres@planetscale.com>
* turn off linter that is wrong
Signed-off-by: Andres Taylor <andres@planetscale.com>
* cleanup and commenting
Signed-off-by: Andres Taylor <andres@planetscale.com>
* refactor and comments
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: Andres Taylor <andres@planetscale.com>
* Don't block on vschema channel in case of heavy vschema changes and vstream load
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Prevent VStreamer engine deadlocks during state transitions
The VStreamer engine is somewhat unusual in two ways:
1. It is open and running on replica tablets rather than only
running on primary tablets.
2. It has no controllers so the main engine mutex is widely shared.
Because of this, when a tablet has open vstreams (direct binary log
streams) performing work and a state transition starts, it can
deadlock with the tabletmanager's state lock when checking if
the engine is open or not.
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Address review comments
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: Matt Lord <mattalord@gmail.com>
* Move acl package to pflag
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Stragglers
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Gofmt after GH merge conflict resolution
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Merge in new vtbackup work
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Address linter errors:
context-as-argument: context.Context should be the first parameter of a function (revive)
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* tests: add more plan_tests for derived tables and subqueries
Co-authored-by: Arthur Schreiber <arthurschreiber@github.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
* refactor: make switch pretty
Signed-off-by: Andres Taylor <andres@planetscale.com>
* feat: push derived table under the route
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* simplify logic
Signed-off-by: Andres Taylor <andres@planetscale.com>
* check for valid derived tables also when doing JOIN on the vtgate
Signed-off-by: Andres Taylor <andres@planetscale.com>
* comments
Signed-off-by: Andres Taylor <andres@planetscale.com>
* add more tests with derived tables
Signed-off-by: Andres Taylor <andres@planetscale.com>
* handle HAVING in derived tables correctly
Signed-off-by: Andres Taylor <andres@planetscale.com>
* merge tests into single file
Signed-off-by: Andres Taylor <andres@planetscale.com>
* rename test keypspace
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Co-authored-by: Arthur Schreiber <arthurschreiber@github.com>
Co-authored-by: Harshit Gangal <harshit@planetscale.com>
* [cli] Begrudgingly shim `flag.Parse` call to trick glog
I applied this to everywhere I think needs it.
- Closes#11283.
- Closes#11276.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* update test data after rebase
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* [vtexplain] Migrate to pflags
Signed-off-by: Andres Taylor <andres@planetscale.com>
* refactor for readability
Signed-off-by: Andres Taylor <andres@planetscale.com>
* fixup vtexplain cmd: make use variables for defaults in all places
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
* Migrate mysqlctld from flag to pflag
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Minor flag help text cleanup
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Pedant can't stop, won't stop
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Update mysqlctld for new grpc client flag handling
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Move vtaclcheck to pflags
Signed-off-by: Matt Lord <mattalord@gmail.com>
* SetOutput on correct FlagSet
Co-authored-by: Andrew Mason <amason@hey.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* gofmt file after accepting suggestion via GH PR
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: Andrew Mason <amason@hey.com>
* [grpclient] Migrate `--grpc_auth_static_client_creds` to pflag
This was missed in #10806.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* [grpcclient] Migrate test setters for `--grpc_auth_static_client_creds`
- Relates to #10934.
- Relates to #10979.
- Closes#10891.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
This:
- Corrects an initial migration of the `DualFormat*` functions to
_actually_ install the flag on the passed in flagset, rather than the
global one.
- Updates `StringListVar` to fully-implement `pflag.Value`, mimicking
pflag's `StringSlice` implementation for this method.
- Removes the compile-time type-check on `StringSetFlag` implementing
`flag.Value` (which is a subset of `pflag.Value` anyway), as well as in
the test code.
Closes#11296.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* feat: remove cli code which isn't used in vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: port over the /api/problems endpoint to the http server of VTOrc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: added comments to api/problems/ endpoint explaining the future scope
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove api endpoints registered in old VTOrc ui
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove gracefulTakeover tests since the api is removed and add tests for the api/problems API
Signed-off-by: Manan Gupta <manan@planetscale.com>
* refactor: fix license year and imports
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add apis for disabling and enabling global recoveries
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add api for reading replication-analysis
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add health api as well
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: add tests for the api's ported over
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: add liveness endpoint to tests too
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: return an internal server error when VTOrc isn't healthy
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: reduce flakiness in api tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
* [vtctlclient] Migrate `vtctl_client_protocol` to pflag
Closes#10892.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* forgot to update testdata
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Matt Lord
FlorentP
vitess-bot[bot]
Manan Gupta
Dirkjan Bussink
Rameez Sajwani
Harshit Gangal
Rohit Nayak
Andrew Mason
Frances Thai
Andres Taylor
Max Englander
Deepthi Sigireddi
Michael
Arthur Schreiber
Phani Raj
Shlomi Noach