Граф коммитов

313 Коммитов

Автор SHA1 Сообщение Дата
Rafael Chacon 6593e5df30 Merge branch 'upstream-master' into vtshovel-poc 2019-11-22 17:09:45 -08:00
Sugu Sougoumarane 2ef29629d7 tabletmanager: WaitForPosition
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-11-02 17:35:51 -07:00
Sugu Sougoumarane cd858ac621 vdiff: resultStreamer
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-11-02 17:34:13 -07:00
Sugu Sougoumarane 2083acea37 mysqlctl: GetSchema also returns field info
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-11-02 17:34:11 -07:00
Anthony Yeh 9a09f820bc
PlannedReparentShard: Fix more known-recoverable problems. (#5376)
* PlannedReparentShard: Fix more known-recoverable problems.

PlannedReparentShard should be able to fix replication as long as all
tablets are reachable and all replication positions are in a
mutually-consistent state.

PRS also no longer trusts that the shard record contains up-to-date
information on the master, because we update that record asynchronously
now. Instead, it looks at MasterTermStartTime values stored in each
master tablet's record, so it makes the same choice of master as
vtgates.

Signed-off-by: Anthony Yeh <enisoc@planetscale.com>

* PlannedReparentShard: Add -lag_threshold flag.

Signed-off-by: Anthony Yeh <enisoc@planetscale.com>

* Fix expected error in reparent test.

Signed-off-by: Anthony Yeh <enisoc@planetscale.com>

* PRS: Add test case for graceful recovery.

Signed-off-by: Anthony Yeh <enisoc@planetscale.com>

* PRS: Measure replication progress instead of lag.

Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
2019-10-31 12:39:03 -07:00
Anthony Yeh e8dd5a4f08 Merge branch 'master' into reparent-refactor
Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
2019-10-30 12:44:51 -07:00
Deepthi Sigireddi d9fa8cd683
Merge pull request #5316 from planetscale/ds-init-tablet-master-ts
InitTablet should not update master alias on shard record
2019-10-25 16:11:26 -07:00
deepthi 8b6cca05de set masterTermStartTime on tablet correctly whenever tablet type changes
Signed-off-by: deepthi <deepthi@planetscale.com>
2019-10-23 18:46:58 -07:00
deepthi 57a6f4878a store master_term_start_time in tablet record. InitTablet should not update shard master
Signed-off-by: deepthi <deepthi@planetscale.com>
2019-10-22 12:27:45 -07:00
yuxiaobo f886cc2681 add license in helm/misc/proto
Signed-off-by: yuxiaobo <yuxiaobogo@163.com>
2019-10-21 17:35:53 +08:00
Rafael Chacon b56bf67d7a Update approach to not require another binary to run vtshovel
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
2019-10-17 16:27:56 -07:00
Anthony Yeh 191d5e5be6 Merge branch 'master' into reparent-refactor
Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
2019-10-16 16:18:42 -07:00
Anthony Yeh 75f547126e Merge branch 'master' into reparent-refactor
Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
2019-10-14 11:32:50 -07:00
Sugu Sougoumarane 01b6c7a067 stream_migrater: integrate with migrater
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-10-11 19:34:14 -07:00
Rafael Chacon f903605880 Merge branch 'upstream-master' into vtshovel-poc 2019-10-10 13:38:48 -07:00
Anthony Yeh 4b12b1cf37 Merge branch 'master' into reparent-refactor
Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
2019-10-07 15:33:39 -07:00
deepthi ac3efb4072 Replace keyspace_type in vschema with a boolean require_explicit_routing
Signed-off-by: deepthi <deepthi@planetscale.com>
2019-10-06 12:23:42 -07:00
deepthi 060f148188 save restore time and position in local_metadata instead of tablet record, more params cleanup and refactoring from reviews
Signed-off-by: deepthi <deepthi@planetscale.com>
2019-10-03 15:38:12 -07:00
deepthi d689461f97 add fields to Tablet record to store which backup we restored from, and which gtid we restored to
Signed-off-by: deepthi <deepthi@planetscale.com>
2019-10-03 15:33:59 -07:00
deepthi 1732232e6f add base_keyspace param to CreateKeyspace
Signed-off-by: deepthi <deepthi@planetscale.com>
2019-10-03 15:33:59 -07:00
deepthi 2e77b986af CreateKeyspace for snapshot keyspaces
Signed-off-by: deepthi <deepthi@planetscale.com>
2019-10-03 15:33:56 -07:00
Rafael Chacon 489eb3c4ac Adds proto definition to store dml extracted from Query
Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
2019-10-03 15:21:50 -07:00
Nitin Shatma d93239c76e proto-changes
Signed-off-by: Nitin Shatma <nitin.sharma@airbnb.com>
2019-10-02 18:51:16 -07:00
Nitin Shatma b912811977 vstream optional field event
Signed-off-by: Nitin Shatma <nitin.sharma@airbnb.com>
2019-10-02 12:23:16 -07:00
Anthony Yeh 2f5a57eb21 topodata.proto: Add master_term_start_time field to Shard record.
Signed-off-by: Anthony Yeh <enisoc@planetscale.com>
2019-10-01 15:11:19 -07:00
hwdef 28abeb0985 fix some spelling mistakes
Signed-off-by: hwdef <hwdef97@gmail.com>
2019-09-27 11:15:31 +08:00
chentanjun 1a43e41322 fix-up typo
Signed-off-by: chentanjun <2799194073@qq.com>
2019-08-26 11:28:06 +08:00
Sugu Sougoumarane 19f23cc041 migrater: add more fields to Journal
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-08-03 19:52:41 -07:00
Sugu Sougoumarane 3fb294471c migrater: initial cut
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-08-03 19:52:41 -07:00
zwcn 440aa522bc use english quotes instead
Signed-off-by: zwcn <zilongwang61@gmail.com>
2019-06-26 19:13:16 -04:00
Sugu Sougoumarane 4e80ef467b vstream: address review comments
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-06-06 15:40:29 -07:00
Sugu Sougoumarane 0b59741632 vstream: RPC scaffolding
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-05-13 20:30:35 -07:00
Sugu Sougoumarane b4379fa078 consistent lookup: address review comments
The VCursor API is impressively simpler no.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-05-09 20:59:29 -07:00
Sugu Sougoumarane 91ea12e8e5 consistent lookup: commit order in Session
Added two new shard sessions for commit ordering:
pre and post.

Added API to set the commit order and changed
tx conn to honor it.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-05-05 19:36:42 -07:00
Sugu Sougoumarane 08791180dc vschema: routing rules proto changes
This is the first part of the changes to implement #4790.
This part implements all the management functionality for
routing rules.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-04-22 20:00:00 -07:00
Sugu Sougoumarane 856a196bc6 vstreamer: rowstreamer locks the table
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-04-17 11:09:54 -07:00
Sugu Sougoumarane c096533521 vplayer: copyTables initial cut
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-04-17 11:04:32 -07:00
Sugu Sougoumarane 1d2d111be9 vstreamer: VStreamRows rpc method
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-04-17 11:04:32 -07:00
SataQiu 139b49e351 fix spelling mistake: conjonction -> conjunction
Signed-off-by: SataQiu <qiushida@beyondcent.com>
2019-04-17 11:59:27 +08:00
Rafael Chacon 2c63167f11 Refactor region to aliases
* Deprecate the concept of regions.
* Favor aliases instead

Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
2019-03-29 19:28:16 -07:00
Michael Demmer 56c2a938d0
Merge pull request #4739 from tinyspeck/autocommit-batch-execute
Single round trip commit on BatchExecute
2019-03-22 08:47:38 -07:00
Rafael Chacon 20b59bb3ca Single round trip commit on BatchExecute
* When auto-commit is on, passDML is on and ExecuteBatch is in a transaction,
there is no need to explicitly create a transaction. We can forward the DML
directly to the database.
* This optimization yielded significant more throughput in vttablets. We got
around 25-30% improvement. Most of our queries are single point
inserts/updates that already use auto commit  when coming from vtgates, so this
improvement is something that we've been wanting to do for a long time.

Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
2019-03-21 15:35:46 -07:00
Rafael Chacon 13aa747940 Merge branch 'upstream-master' into 4496-topo-serving-shards-refactor 2019-03-07 13:16:12 -08:00
Sugu Sougoumarane 8913fcf081
Merge pull request #4699 from planetscale/ds-allow-master-backup
allow backup on master
2019-03-06 16:20:40 -08:00
Sugu Sougoumarane 31178f975a
Merge pull request #4637 from planetscale/ss-vheart
vreplication: improved lag tracking
2019-03-03 13:28:01 -08:00
deepthi 72689e8872 allow backups on master tablet
Signed-off-by: deepthi <deepthi@planetscale.com>
2019-03-02 16:21:08 -08:00
Kim Bao Long d0063cb907 Trivial fix: Remove the redundant words
Although it is spelling mistakes, it might make an affects while reading.

Co-Authored-By: Nguyen Phuong An <AnNP@vn.fujitsu.com>
Signed-off-by: Kim Bao Long <longkb@vn.fujitsu.com>
2019-02-25 14:39:32 +07:00
Sugu Sougoumarane a6f5da6c6c
Merge pull request #4550 from planetscale/ds-reparent-undo
rollback changes made during PlannedReparent if we encounter errors
2019-02-22 15:14:36 -08:00
Rafael Chacon 9272b1cd73 Add ServedType for backwards compatiblilty
* Also adds more validations and tests when rebuilding serving keyspace

Signed-off-by: Rafael Chacon <rafael@slack-corp.com>
2019-02-18 18:26:00 -08:00
Sugu Sougoumarane 41767d84fb vreplication: improved lag tracking
The new lag tracking introduces the following changes:
* VStreamer sends its current time along with every event. This
  allows for VPlayer to correct for any clock skew that may
  exist between the two machines. This results in a more accurate
  calculation of lag.
* If there are no events to send for a period of time, VStreamer
  sends a heartbeat event. This allows us VPlayer to essentially
  know for sure that it's still caught up.
* If VPlayer receives no event for its wait period, then it updates
  the SecondsBehindMaster stat to indicate that it's actually falling
  behind.

The VStreamer timeout for heartbeat is set slightly lower than the
VPlayer idle timeout. This ensures that Vplayer won't timeout
exactly when it's about to receive the heartbeat event.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2019-02-17 19:35:30 -08:00