If a row already exists on the destination with a different keyspace_id on a different destination shard, we cannot use UPDATE. Instead, we must DELETE the old row from the old destination shard and INSERT the new row to the new destination shard.
This unflakes the worker.py test which requires to observe vtworker in the clone state such that it can run the reparent.
The issue was introduced in https://github.com/youtube/vitess/pull/1858 where waitForTablets() was added as part of the clone phase.
The protobuf files are updated with the added
tabletmanagerservice.RestoreFromBackup and mysqlctl.ReinitConfig calls and their
respective Request/Response messages.
Executing the command vttablet will completely delete the existing data and
then replace it from the latest backup. The sequence of actions is basically
the same as vttablet has already used to setup a freshly started replica.
Also mysqlctl interface is extended with ReinitConfig function. It's used by
vttablet to change server_id of the replica before restoring it from backup.
That's necessary to avoid a possible situation when restored replica skips
transactions in the replication stream which have the same server_id.
The new RestoreFromBackup command will be used later in the implementation of
an automated pivot.
This change doesn't add any tests for the new functionality because I didn't
find a place where I could add these tests. Please tell me if there is actually
a place that I missed. Meanwhile I tested this manually on the example
Kubernetes setup.
Thix fixes the problem that SplitClone immediately failed when the destination tablets became unhealthy because a previous chunk pipeline inserted data too fast.
I've reused the existing timeout of 2 hours (from the destination writes) and made it configurable as global flag "--retry_duration".
The old chunk generator code generated "threads+1" chunks. The new code generates "threads" chunks instead. I forgot to adjust the number of threads for the table status.
We'll restart the query by advancing the start of the chunk right after the primary key of the last row read so far.
Other changes:
- Refactored FindChunks. I'm using sqltypes.Result now.
- Added Chunk struct to store a chunk. I'm using sqltypes.Value for the start and end to simplify processing the values.
The experimental VSchema editor UI was originally intended as a learning
tool to help explain VSchema concepts. We are removing it prior to the
General Availability release because we don't want to encourage a workflow
in which new VSchema is pushed instantly without review.
Instead, VSchema configs should be under source control, and changes
should be peer-reviewed before being pushed at an appropriate time by a
DBA. This mirrors industry best practices for regular SQL schema changes.
In a future release, we plan to incorporate an improved, holistic
Schema+VSchema management system. For now, updates should be done through
vtctlclient:
http://vitess.io/reference/vtctl.html#applyvschema
Michael Berlin
Pavel Ivanov
Harshit Gangal
Anthony Yeh
David Slater
Alain Jobart
Alain Jobart