Граф коммитов

73 Коммитов

Автор SHA1 Сообщение Дата
Peter Farr 2af7875d21 Finished all major aspects of design, other than
--ignore_unreachable_tablets flag, which I would like some discussion
around.

Signed-off-by: Peter Farr <Peter@PrismaPhonic.com>
2020-07-27 16:24:47 -07:00
Rohit Nayak 6b580b69c3
vtctl vexec/workflow actions: add e2e test, improve list output
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-07-04 17:24:49 +02:00
Rohit Nayak 9afb76fc09
Fix go.mod/sum
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-07-03 23:11:54 +02:00
Rohit Nayak e8d9e8ff04
vtctl vexec/workflow actions: initial commit
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-07-03 23:08:53 +02:00
Sugu Sougoumarane 12b89d3294
Merge pull request #6399 from planetscale/rn-schema-version-ddl-as-blob
Schema Version Table: change ddl column to blob
2020-07-02 16:14:33 -07:00
Rohit Nayak c83201f5e3
Schema Version Table: alter table using withddl avoiding breaking change. Updated tests: extra events can occur if tests are run isolated due to the way withddl works
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-07-01 13:43:05 +02:00
Andres Taylor bd59a4834c union-all: use specific version of x/sync
Signed-off-by: Andres Taylor <andres@planetscale.com>
2020-07-01 09:26:06 +05:30
Andres Taylor 4aff6a85f9 go mod tidy
Signed-off-by: Andres Taylor <andres@planetscale.com>
2020-06-24 13:35:21 +02:00
Andrew Mason ccf496daf0 Upgrade consul to 1.8.0 to reconcile k8s addition with consul upgrade
Signed-off-by: Andrew Mason <amason@slack-corp.com>
2020-06-22 18:04:16 -04:00
deepthi fc3f4df3a6 go mod tidy
Signed-off-by: deepthi <deepthi@planetscale.com>
2020-06-17 16:53:11 -07:00
Rohit Nayak 81512fae85
Historian now subscribes to schema engine. Added ReloadAt in SE/VS to optimize schema loads. Related tests. Fixed TestMessage race
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-05-31 00:16:27 +02:00
Andrew Mason fef8a46437 Upgrade pgzip to v1.2.4
This upgrades pgzip to include a
[fix](https://github.com/klauspost/pgzip/pull/33) to prevent negative
byte counts being returned from `pgzip.Write`, which cause panics in
`bufio`.

These panics manifest when using compressed xtrabackup backups as:

```
Error: <_Rendezvous of RPC that terminated with
(StatusCode.UNKNOWN, caught panic during Backup: bufio: writer returned negative count from Write)>
```

Signed-off-by: Andrew Mason <amason@slack-corp.com>
2020-05-15 13:12:33 -04:00
deepthi 5389714d53 Print MasterTermStartTime in ListAllTablets, ListTablets,
ListShardTablets. In case of an old master, we replace the timestamp
displayed with "defunct".
Also show MasterTermStartTime in output of "show vitess_tablets" from
vtgate.

Signed-off-by: deepthi <deepthi@planetscale.com>
2020-04-30 21:27:18 -07:00
Rohit Nayak b57f24bb10 Added blacklist, drop shard/tables checks. Bug fixes
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-04-20 17:18:48 -07:00
Rohit Nayak 61d1fba5e2 Added reusable log recorder
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-04-20 17:17:34 -07:00
Sugu Sougoumarane 3ebacbd452 yaml: address review comments
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2020-04-19 08:46:22 -07:00
Sugu Sougoumarane 9e29feb390 yaml: start transitioning: PoolSize
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2020-04-18 18:30:22 -07:00
Sugu Sougoumarane ffadb39a60
Merge pull request #5935 from planetscale/rn-vrepl-work
VReplication: End to End tests with a few related bug fixes
2020-03-24 13:05:46 -07:00
Rohit Nayak 1ebf4f9e39
Copied go.mod/go.sum from master since branch was not compiling with the merged ones
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-03-21 00:55:19 +01:00
Rohit Nayak 5f302e1215
Fixed incorrect shard and gtid sorting, vdiff test
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-03-21 00:42:40 +01:00
Rohit Nayak d1b9653d02
Updated tests. VDiff defaults to all tablet types. More stream migration logs
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-03-21 00:41:24 +01:00
Rohit Nayak bcbf6d2dee
Working e2e tests for vreplication flows
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-03-21 00:32:18 +01:00
Rohit Nayak 49ba470bc5
partial commit
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-03-21 00:23:36 +01:00
Morgan Tocker 7a48f72139 Upgrade go minimum to 1.13
Signed-off-by: Morgan Tocker <tocker@gmail.com>
2020-03-20 11:30:34 -06:00
Carson Anderson fcf1473273 Add Kubernetes topo implementation
Signed-off-by: Carson Anderson <ca@carsonoid.net>
2020-03-14 09:49:37 -06:00
Daniel Kozlowski f569401b5c
Merge pull request #5674 from planetscale/dk-add-azblob-backups
Adding Azure Blob backup support
2020-03-06 13:01:05 -08:00
Sugu Sougoumarane 124cc4399b
DBConfigs Refactor (#5833)
* Added wrapper struct, modified usage

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Add go.mod

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Refactor dbconfigs and its calls

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Fix failing vstreamer test

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Fix copy and paste mistakes

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Made withCredentials a private function and refactored the calls

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Fix Tests

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Clean up code

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Extend DbConfigs and change method name

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Add Connect Method

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Pass Context as parameter

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Modify definition of DBConfigs

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Make Connect function part of the Connector definition

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>

* Remove obselete functions

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>
2020-03-05 08:31:25 -08:00
Dan Kozlowski aae83c4f26 Fixing directory paths for azure blobs
Signed-off-by: Dan Kozlowski <koz@planetscale.com>
2020-03-04 14:56:06 -08:00
Dan Kozlowski 7c84818955 Incorporating feedback from pull request.
Giving Env paramaters for secrets, not just CLI.
Rename prefix to be storage_root i.a.w what we do for s3 and gcs

Signed-off-by: Dan Kozlowski <koz@planetscale.com>
2020-03-04 14:56:06 -08:00
Dan Kozlowski d9d0ad6c94 Adding Azure Blob backup support
This adds initial support for Azure blob service as a backup target

Signed-off-by: Dan Kozlowski <koz@planetscale.com>
2020-03-04 14:56:06 -08:00
Sugu Sougoumarane 8383470c15 mysql: flush buffer on inactivity
The mysql server uses buffered I/O. This works fine if the results
are streamed all at once. But if the streaming is intermitent, like
in the case of messages, the client may not get timely responses.

This new change flushes any unsent data if no other data is sent
within mysql_server_flush_delay (100ms).

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2020-03-01 20:49:18 -08:00
pradip parmar 9ef5649e34 Merge remote-tracking branch 'vitess/master' into web-test
Signed-off-by: pradip parmar <prince.soamedia@gmail.com>
2020-02-24 10:36:01 +05:30
Sugu Sougoumarane c051a7051f vrepl: vplayer must rollback on exit
There were code paths where vplayer had an open transaction on
its dbclient connection, which got accidentally continued by
the copier. This caused production issues with some users.

The specific use case that was observed is:
* vreplication performs catchup.
* While it's in the middle of an apply, the context is canceled.
* The copier has no work.
* vreplication is asked to resume from the last saved position.
* It replays the same statements already in the transaction.
* This leads to dup key errors.

This fix conservatively rolls back dbclient before exiting.
The dbclient itself skips rollback if it's not in a transaction.
This change leads to relieable rollbacks where needed. At the
same time, there are no spurious rollbacks if we didn't start
a transaction.

As safety, I've added an extra rollback in the vreplicator loop
just in case the underlying functions accidentally leave an
incomplete transaction open.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2020-02-21 12:07:40 -08:00
Morgan Tocker 5240335511 Fix consul dependency
Signed-off-by: Morgan Tocker <tocker@gmail.com>
2020-02-19 08:42:38 -07:00
pradip parmar 54afe78a10 vtctld web test .
Signed-off-by: pradip parmar <prince.soamedia@gmail.com>
2020-02-18 14:32:34 +05:30
Saif Al-Harthi c0c927a0a9
Support SHOW CREATE TABLE with qualifier (#5585)
* Support SHOW CREATE TABLE with qualifer

Signed-off-by: Saif Alharthi <saif@saifalharthi.me>
2020-02-06 22:30:09 +01:00
Roni HE a2dd7a0b38
Fix: Vitess show character set where
right now the show character set are hardcoded to be filtered with where statement.
since we don't really want to expand the results or forward the query to an underlying mysql because vitess only supports these 2 charsets.

closes #5584

* separate show charset vs show charset in parser
* changed the executor accordingly
* use where_expression_opt in parser, changed AST and executor accordingly

Signed-off-by: roni <herongrong2011@gmail.com>
2020-02-06 17:14:51 +01:00
deepthi 58d4813f24 upgrade aws-sdk-go to 1.28.8
Signed-off-by: deepthi <deepthi@planetscale.com>
2020-01-30 14:17:08 -08:00
Sugu Sougoumarane 46bd5442e2 Fix vreplication panic
Fixes #5752

DBClient was setting the connection to nil after close, which was
causing vreplication to panic because it defers a Rollback on all
connections. The safer thing is to close the connection, but not
set it to nil.

There's no test for DBClient, but I've manually verified that this
won't panic any more.

Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2020-01-23 14:21:14 -08:00
deepthi 2dc1c7861d Deprecate disable_local_gateway flag. It should have been removed along with l2vtgate functionality
Signed-off-by: deepthi <deepthi@planetscale.com>
2020-01-20 12:30:06 -08:00
Sugu Sougoumarane 7fe28224f7
Merge pull request #5670 from princeparmar/prepared-statement
prepared statement testcases
2020-01-18 20:25:08 -08:00
Sugu Sougoumarane c4e06a1e62
Merge pull request #5681 from kalfonso/kalfonso.200109-datadog-tracing-plugin
Add datadog plugin for tracing
2020-01-18 20:20:45 -08:00
prince 0ac67abc27
Merge branch 'master' into prepared-statement 2020-01-17 13:02:48 +05:30
prince 4e46a60908 mysql server testcase. (#5671)
Signed-off-by: pradip parmar <prince.soamedia@gmail.com>
2020-01-16 17:04:40 -08:00
pradip parmar 3ab33f668c prepare statment test case.
Signed-off-by: pradip parmar <prince.soamedia@gmail.com>
2020-01-16 12:50:31 +05:30
Sugu Sougoumarane f87c943ea6 vrepl: CreateLookupVindex tweaks
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
2020-01-14 17:58:47 -08:00
Karel Alfonso Sague 245508242d Add datadog plugin for tracing
Signed-off-by: Karel Alfonso Sague <kalfonso@squareup.com>
2020-01-10 14:04:05 +11:00
Toliver Jue 3df2fd9173 Fix order
Signed-off-by: Toliver Jue <toliver@planetscale.com>
2020-01-09 10:30:37 +01:00
Rohit Nayak 255e321ffb select last_insert_id() working for non-streaming case
Signed-off-by: Rohit Nayak <rohit@planetscale.com>
2020-01-06 11:29:17 -08:00
Deepthi Sigireddi cafd1094ef
Merge branch 'master' into mysql-replication-bugfix 2020-01-03 10:53:22 -08:00