* Fix ordering when error happens during stream setup (#11592)
* Fix ordering when error happens during stream setup
When we already have an error during the stream setup, that errors needs
to be returned immediately.
Otherwise the stream will already be started and the error message is
never returned to the client and it silently looks like a success even
if it's a failure.
This in turn can result in an empty result for example when an error
would have been expected.
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
* Add a bunch of testing to verify stream execute behavior
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
* Send errors in stream instead of a grpc error from streaming rpcs when transaction or reserved connection is acquired (#11656)
* wip
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* feat: send errors in stream instead of a grpc error from streaming rpcs
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>
* VTexplain topology only uses serving shards
This addresses isse #11632 , which
causes vtexplain to sometimes give bad results if the keyspace is being
resharded, because sometimes it picks source shards and other times target
shards, for routing the query.
The issue is that the `VTExplain.buildTopolog()` adds both source and
destination shards to the map that holds shards per keyspace, when only one of
them is actually serving traffic at any point in time. Later on, vtexplain
loops over this map. Because looping over the map gives a non-deterministic
order, sometimes the results are correct, and sometimes incorrect - that is,
sometimes it gives the result of the shard that is serving, and other times,
the shard that is not serving.
This change ensures that only the serving shards are added to the shards per
keyspace map, thus avoiding the incorrect vtexplain.
Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
* This addresses issue #11632 , which
causes vtexplain to sometimes give bad results if the keyspace is being
resharded, because sometimes it picks source shards and other times target
shards, for routing the query.
The issue is that the VTExplain.buildTopolog() adds both source and
destination shards to the map that holds shards per keyspace, when only one of
them is actually serving traffic at any point in time. Later on, vtexplain
loops over this map. Because looping over the map gives a non-deterministic
order, sometimes the results are correct, and sometimes incorrect - that is,
sometimes it gives the result of the shard that is serving, and other times,
the shard that is not serving.
This change ensures that only the serving shards are added to the shards per
keyspace map, thus avoiding the incorrect vtexplain.
Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
* Fix check_make_vtadmin_authz_testgen
Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
* empty commit to trigger CI
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: Eduardo J. Ortega U <5791035+ejortegau@users.noreply.github.com>
Co-authored-by: Andres Taylor <andres@planetscale.com>
* Do not multiply `AggregateRandom` in `JOIN`s (#11633)
* Do not multiply AggregateRandom in JOIN
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Update the releases notes with new known issue
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Addition of the release notes for v15.0.1
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Fix format issue in the release notes summary
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* set sql mode in TestAggregateRandom
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Instead of relying on a specific version check, detect if the new redo
log location is present or not.
This makes the backup logic independent from the version check against
MySQL. With this change, 8.0.30 can be backed up just as well as any
other version.
Solves the backup part of
https://github.com/vitessio/vitess/issues/11554 by removing the
dependency on the version check. There might be still other places where
that issue can crop up, but it solves the immediate breaking issue with
backups.
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com>
* Update logic for parsing test flags to run unit tests within GoLand. Parse test flags in vtgate to allow running unit tests
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Minor changes after self-review
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* don't rewrite HAVING predicates that use table columns
Signed-off-by: Andres Taylor <andres@planetscale.com>
* Revert the changes made in #11306
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
* Fix early rewriter test
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Co-authored-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Co-authored-by: Andres Taylor <andres@planetscale.com>
* Only perform vindex logic for deletes against sharded keyspaces.
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
* Add a testcase covering deletes on an unsharded reference table.
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
* Check if target keyspace is unsharded instead of checking opcode.
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
Signed-off-by: Arthur Schreiber <arthurschreiber@github.com>
Co-authored-by: Arthur Schreiber <arthurschreiber@github.com>
* Allow version to be accessible via the -v shorthand
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* update help text
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* fix nonidempotence for tests
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Signed-off-by: Andrew Mason <andrew@planetscale.com>
I accidentally stumbled over the behavior outlined in
`flag.PrintDefaults` [1], which `pflag` replicates, specifically:
> The listed type, here int, can be changed by placing a back-quoted
> name in the flag's usage string; the first such item in the message is
> taken to be a parameter name to show in the message and the back quotes
> are stripped from the message when displayed.
[1]: https://pkg.go.dev/flag#PrintDefaults.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Signed-off-by: Andrew Mason <andrew@planetscale.com>
Co-authored-by: Andrew Mason <andrew@planetscale.com>
* feat: added test for vtorc not being able to handle mutliple failures and fix it
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: fix code to delete rdonly tablet from the correct list
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: added failing e2e test
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* log txID and reserveID in stream execute
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* fix: maintain list of qd per key on the map and check for current connection id while removing
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* added additional comments
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
Signed-off-by: Harshit Gangal <harshit@planetscale.com>
* Plan order by Count()
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Clean up the new aggregation E2E test
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Push more order by needs to the select list
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Remove unrequired code in TestOrderByCount
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* remove unwanted directory
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* Use JSON for plan tests
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
* move vschemas into single directory
Signed-off-by: Andres Taylor <andres@planetscale.com>
* indentation
Signed-off-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: Andres Taylor <andres@planetscale.com>
Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr>
Signed-off-by: Andres Taylor <andres@planetscale.com>
Co-authored-by: FlorentP <35779988+frouioui@users.noreply.github.com>
* Testing....
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Setup reverse routing rules at the start of the first partial movetables. Enhance e2e test to test for this as well as switch the remaining shard so that all shards are switched
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Update display representation of state for partially moved keyspaces
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
* Test improvements
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Cleanup shard routing rules on cancel/complete
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Cleanup intermediate changes
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Fix DryRun TrafficSwitcher and its tests
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Support concurrent partial movetables for N keyspaces
And other minor changes after self review
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Avoid unnecessary save + rebuild steps
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add secondary ordering for shard routing rules on to_keyspace
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Minor changes after self review
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: Rohit Nayak <rohit@planetscale.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: Rohit Nayak <rohit@planetscale.com>
* feat: call the logic for waiting for shard locks on synchronous on-term hooks and add tests for it
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix comment for acceptSighupSignal function
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
Co-authored-by: Manan Gupta <manan@planetscale.com>
* When restarting workflow, use new context
Otherwise the restart will also fail due to the
expired/canceled context:
E0929 14:06:08.618848 85788 vdiff.go:247] Could not restart workflow commerce2customer: rpc error: code = Canceled desc = context canceled, please restart it manually
E0929 14:06:08.619983 85788 vtctl.go:2630] vdiff returning with error: rpc error: code = Canceled desc = context canceled
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add retry when restarting workflows on ephemeral errors
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Correct mistake that prevented compilation :face-palm:
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Move towards MySQL 8.0 as the default template generation
This upgrades the remaining things to Ubuntu 20.04 and makes MySQL 8.0
the default we run tests against. We still have tests for MySQL 5.7 but
those are now explicitly opted into.
This should finish up the Ubuntu 20.04 upgrade and also makes things
easier for the future when we need to upgrade again.
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
* CI: rename shard vtorc_8.0 to vtorc_5.7, change expected test output for 8.0
Signed-off-by: deepthi <deepthi@planetscale.com>
* CI: increase timeout for 8.0 tests on the actual test step from 30 to 45 mins
Signed-off-by: deepthi <deepthi@planetscale.com>
* CI: increase timeout to 45 minutes for mysql57 tests too. We really only need this for vtorc, but I've made the change to the template so all tests get it.
Signed-off-by: deepthi <deepthi@planetscale.com>
* CI: fix vtorc test to work with both 5.7 and 8.0
Signed-off-by: deepthi <deepthi@planetscale.com>
* CI: missed docker flag in mysql57 template, one more fix to vtorc test
Signed-off-by: deepthi <deepthi@planetscale.com>
* removing spaces from pb file
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
* removing spaces in pb file part 2
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>
Co-authored-by: deepthi <deepthi@planetscale.com>
Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com>
* flags etc: delete old flags and stats, add deprecation notice to release notes
Signed-off-by: deepthi <deepthi@planetscale.com>
* vrepl: fix tests to run SwitchReads with tablet_types flag instead of deprecated tablet_type
Signed-off-by: deepthi <deepthi@planetscale.com>
* pitr: fix tests to run SwitchReads with tablet_types flag instead of deprecated tablet_type
Signed-off-by: deepthi <deepthi@planetscale.com>
Signed-off-by: deepthi <deepthi@planetscale.com>
Harshit Gangal
vitess-bot[bot]
FlorentP
Rameez Sajwani
Manan Gupta
Rohit Nayak
Andrew Mason
Frances Thai
Andres Taylor
Matt Lord
Dirkjan Bussink
Max Englander
Deepthi Sigireddi