* feat: fix 101_initiail_cluster to not run PRS, but instead wait for tablets to be up and primary promotion by vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix remaining binaries in local examples to not call ISP at all and instead rely on VTOrc to elect a primary
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix region-sharding example too to not call ISP
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: use _ instead of i in for loop since i is unused
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
* docs: fix logging in vtorc http
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add servenv and required flags to vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add vtorc port to local and region sharding example
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: add vtorc to flag help test
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: simplify vtorc parsing of flags
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add redirect for unrecognized paths to /debug/status to vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add a component to debug/status page of vtorc to display recent recoveries
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: simplify a condition in vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix command used for invoking vtorc
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove adding help flag and let servenv handle it
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add vtorc servenv page to docker-compose example too
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add prometheus plugin like we do for other binaries
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: change vtorc debug port to not conflict with vttablet's port
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix year in file license
Signed-off-by: Manan Gupta <manan@planetscale.com>
* docs: add summary docs
Signed-off-by: Manan Gupta <manan@planetscale.com>
Signed-off-by: Manan Gupta <manan@planetscale.com>
* Run latest gofmt on everything & address linter warnings
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Rebuild proto impls with new protoc
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Field name is needed for zk
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Make protos with 1.18.4
Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
* feat: use the latest operator yaml file
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add initial script for vtop example
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add script to install kubectl and kind and add a github workflow
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add code to verify vtadmin and vtorc are running correctly
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add move tables and reshard part to the example
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove the inserted row in vtorc check so that move tables isn't affected
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: run the ci test outside docker and skip build since it is already done in a previous step
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: source build.env before running the test in ci
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: use custom image names in the test so as to not add image pull policies to the configuration
Signed-off-by: Manan Gupta <manan@planetscale.com>
* ci: remove setup steps that aren't used
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: use the common files instead of creating new ones
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: increase timeout of the ci
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: turned off semi-sync for the test
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: insertion should be retried since VTOrc might take a few seconds to fix it
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove the workflow file since it is too heavy
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add double dashed flags and remove configuration for ci
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix readme for cluster example
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: copy init_db.sql file to 101 yaml
Signed-off-by: Manan Gupta <manan@planetscale.com>
* docs: fix comment in helper file
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: deprecate enable_semi_sync flag
Signed-off-by: Manan Gupta <manan@planetscale.com>
* docs: add the deprecation change to the summary
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: fix test expectation for vttablet flags
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix cnf files to not mention the deprecated flag
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove enable_semi_sync from scripts of examples
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: remove setting enable_semi_sync flag in testlib tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove enable_semi_sync from a bunch of tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: refactor setup to take in durability policy instead of a boolean and add a test for cross cell durability policy
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add cross cell durability policy to the docs
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix flag help output
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: we shouldn't remove enable_semi_sync from tests which are testing upgrades
Signed-off-by: Manan Gupta <manan@planetscale.com>
* Delete all legacy sharding related code
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Move used until functions from initialsharding to cluster
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Remove vtctl commands
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Kill vtworker and SetKeyspaceServedFrom cmd
Signed-off-by: Matt Lord <mattalord@gmail.com>
* WaitForDrain related stragglers
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Legacy local straggler workflow
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Get rid of SetKeyspaceShardingInfo & wait for drain stragglers
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Remove vtworker stragglers
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Update throttlerservice protobuf
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Rename test 24, add hashicorp vault test to it (now mysql_server_vault)
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Remove last mentiones of legacy sharding in vtctl
Signed-off-by: Matt Lord <mattalord@gmail.com>
* remove binlog_use_v3_resharding_mode
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Address review comments
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Address review comments
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Correct vtgate help output
Signed-off-by: Matt Lord <mattalord@gmail.com>
* examples/compose: Set keyspace durability policy to help vtorc recognize vttablets
Signed-off-by: yoheimuta <yoheimuta@gmail.com>
* Move SetKeyspaceDurabilityPolicy commands to a more appropriate place.
Signed-off-by: yoheimuta <yoheimuta@gmail.com>
* Add a set_keyspace_durability_policy container to docker-compose.beginners.yml
Signed-off-by: yoheimuta <yoheimuta@gmail.com>
* Adjust a generated docker-compose.yml by vtcompose/vtcompose.go to work
Signed-off-by: yoheimuta <yoheimuta@gmail.com>
* Tweak docker-compose.beginners.yml to use the environment variable
Signed-off-by: yoheimuta <yoheimuta@gmail.com>
* Fix a gofmt error
Signed-off-by: yoheimuta <yoheimuta@gmail.com>
* Halve request and double limit for RAM in the example cluster
1. We don't need/want everything to have guaranteed QoS (request == limit)
2. It's common for mysqld (especially 8.0) to go beyond the old limit
and end up in a crash loop
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Use consistent requests,limits ordering in yaml
Signed-off-by: Matt Lord <mattalord@gmail.com>
* feat: store keyspace information in the vtorc database
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add code for refreshing keyspaces
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove the durability config from VTOrc and instead use the durability stored in the topo server
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: remove usage of Durability config from the tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: add a new test for checking that VTOrc works with keyspaces having different durability policies and verify it failed previously
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove usage of durability config in examples
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: parallel execution of test causes data race
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add warning if no durability is specified
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: export the durabler interface and its methods
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add function to get a durability policy
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add a durability parameter to the exported durability functions, and pass nils from all the places calling it
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: fix the durability tests to reflect the new changes
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix PlannedReparentShard to get the durability policy and use it instead of relying on the global policy
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix EmergencyReparentShard to get the durability policy and use it instead of relying on the global policy
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix all places other than vtorc using durability_policies
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove use of SetDurabilityPolicy
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove setdurabilitypolicy function
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix VTOrc to store its global durability policy separately
Signed-off-by: Manan Gupta <manan@planetscale.com>
* docs: fix case of a comment
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: fix vtorc tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fail if durability policies don't match
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: set the correct durability_policy for the keyspaces in vtorc tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: set the correct durability_policy for the keyspace in TestSemiSync
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: set the correct durability_policy for the keyspaces in reparent tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: set the correct durability_policy for the keyspaces in backup tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: set the correct durability_policy for the keyspaces in recovery tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: set the correct durability_policy for the keyspace in initialization test for ERS
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: set the correct durability_policy for the keyspaces in sharding tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: set the correct durability_policy for the keyspaces in vault tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: add commands to set the correct durability_policy in local and legacy_local examples
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: set the correct durability_policies for the unit tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: fix upgrade test for initialization in ers
Signed-off-by: Manan Gupta <manan@planetscale.com>
* Update vtctl help output to use double dashes for long flag names
* Update any spots missed in examples
* Catch any missed spots in tests
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Ensure all legacy sharding commands are clearly deprecated
Then they can be removed in v15
* Add deprecation notice for related flags
* Move single dashes to double in edited files
* Adjust vttablet --help output test
* Document new default in command help output
* Correct vttablet_process printf that I noticed in a test flake
* Add protos for remaining reparent actions and run generators
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Initial reparent-actions implementations
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Add pools and configs for reparent operations
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Update impls to use new pools
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* jk, TER only updates the topo
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Add http wrappers
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Add unit tests
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Update local rbac to allow reparents
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* feat: revert semi_sync change
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove the durability_policy flags usage from vtctld vtctl and vtworker
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: fix remaining function and reintroduce the test
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: skip the tests in ers that rely on the durability_policy being specified correctly
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: remove the usage of durability_policy from all the tests
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: remove the deprecation notice from the enable_semi_sync flag
Signed-off-by: Manan Gupta <manan@planetscale.com>
* test: fix the flags test expectation
Signed-off-by: Manan Gupta <manan@planetscale.com>
This way vreplication is using the same user on the source (streamer) and
target (player).
And set the filtered user config in the binlog server streamer and
the tabletmanager restore.
Signed-off-by: Matt Lord <mattalord@gmail.com>
* Add `--rbac` to example vtadmin-up
This was introduced and made a required flag (with mutually-exclusive `--no-rbac`)
in https://github.com/vitessio/vitess/pull/9972.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Cleanup `Options.Services`, and inspect the ServiceInfo() directly
Instead of requiring users to know in advance (and put into their startup
scripts), which services will run, we can (correctly) inspect the service
names that have been registered on the underlying grpc.Server and mark
them as SERVING in the health service.
_Alsooooo_, this was never bound to a flag, so users couldn't use it
anyway! Whoops.
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Provide option to enable channelz service
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Add flag to enable channelz
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Tidy up flags; label all sections
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Add missing flag for AllowTracing
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* feat: wait for the replica to finish restoring before calling InitShardPrimary to ensure it sets semi-sync correctly
Signed-off-by: Manan Gupta <manan@planetscale.com>
* feat: have a common timeout for all the tablets instead of 300 per tablet
Signed-off-by: Manan Gupta <manan@planetscale.com>
* Migrate remaining vtctldclient commands in local example
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Missed some extra `Apply(V)Schema` calls
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Add consul-topo for local example
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Bash tricks!
Signed-off-by: Andrew Mason <andrew@planetscale.com>
* Add consul topo to local example CI matrix
Signed-off-by: Andrew Mason <andrew@planetscale.com>
The return value was changed from `[]byte` to `([]byte, error)` a little
while ago, making it impossible to run the demo.
This handles the error and returns nil after logging the error in case
the conversion fails.
Signed-off-by: Pierre Jambet <pierre.jambet@gmail.com>
Manan Gupta
Matt Lord
yohei yoshimuta
Andrew Mason
doeg
FlorentP
Vicent Marti
Deepthi Sigireddi
cui fliter
Andrew Mason
Florent Poinsard
Pierre Jambet
Manan Gupta
notfelineit