This commit updates all fixed dependencies to their latest version and
mades changes to the default configs to.
1. Increase the defaults for a few key values
2. Move some settings to new files to support MariaDB 10.3
3. Add a sample production configuration to allow easy deployment
Signed-off-by: DK <dk@planetscale.com>
The test was flaky because frequent restarts of mysql were
causing the relay log to go corrupt. Changed the my.cnf settings
to be more crash-safe.
Signed-off-by: Sugu Sougoumarane <ssougou@gmail.com>
Issue #3158
The current value of 0 is definitely incorrect because vitess
treats table names as case-sensitive. It's better we don't set
this variable at all. Let the users decide what it should be.
Fix for `Access denied; you need (at least one of) the RELOAD
privilege(s) for this operation` during master failover process,
which happens when Orchestrator tries to execute RESET SLAVE.
This adds support for an 'allprivs' user which is supposed to have more
privileges than 'app' user, but unlike 'dba' user 'allprivs' won't have SUPER
privilege. New method ExecuteFetchAsAllPrivs() is added to TabletManager
service to be able to execute queries as this new 'allprivs' user.
'allprivs' user will be used for administrative tasks done by Vitess, such as
changes in metadata tables that schema swap process will be doing. 'allprivs'
user shouldn't have SUPER privilege so that schema swap could safely change
replicated metadata on the master without risk to commit the changes when
master has been already demoted to be a slave.
This table will contain a per-shard metadata information similarly to how
_vt.local_metadata contains local per-tablet metadata information. It will be
used later by online schema swap process. This table will be replicated and
changed only on the master. As an exception to that rule creation of the table
is added to the vttablet startup code to make sure that it's created during
vttablet version update.
Copying of the contents of _vt.shard_metadata table is added to horizontal
resharding procedure to make sure that the contents of the table is consistent
across all shards of a single keyspace in cases when we are resharding only part
of the source keyspace leaving another part intact. The check for existence of
_vt.shard_metadata is still added to CopySchemaShard e.g. for a case when new
vtworker code will work with older vttablet code.
I've modified init_db.sql to create both _vt.shard_metadata and
_vt.local_metadata tables because it's just logically correct to have them both
in the tests since they will exist both in production. Tests are also adjusted
to not clobber the _vt database when not needed since resharding code now
depends on _vt.shard_metadata table existence.
This causes mysqld to ignore relay logs on disk at startup,
and instead start replicating from the master wherever the SQL thread
left off.
This is generally recommended for safer crash recovery.
We also do it to avoid problems caused by spurious Previous-GTIDs lists
in relay logs that occur when we restart mysqld in integration tests.
This allows tablets (consisting of vttablet+mysqld) to be restarted
without having to manually re-point slaves at the master and tell them
to start replicating again. This is important for rollout flows, where
each slave is restarted in turn.
We can't just check at vttablet startup, because then we'd miss the case
when only mysqld is restarted. Instead, we check at each healthcheck
interval so replication becomes self-healing. The replication lag check
had to be moved from mysqlctl to tabletmanager and integrated with this
healing, to make sure the healing attempt is synchronized with the slave
status check.
This is also needed for Orchestrator integration, because we are responsible
for giving the slave a master in the first place so Orchestrator knows which
replication tree the slave belongs to.
When using semi-sync with rpl_semi_sync_master_wait_no_slave=1 and
rpl_semi_sync_master_timeout=[effectively infinite], it's necessary to
toggle master-side semi-sync at the proper times to prevent slaves from
getting stuck (since they have no slaves of their own to ACK).
This commit adds a new vttablet flag -enable_semi_sync, which causes the
replication-management features of Vitess (e.g. InitShardMaster and
various reparent operations) to automatically manage semi-sync settings.
With these settings, it becomes (reasonably) safe to reparent away from
an unresponsive master, as long as the following can be verified:
* All semi-sync slaves have stopped replicating from the old master.
* The chosen new master is the farthest ahead in replication.
Currently we only allow slaves marked in Vitess terms as "replica"
tablets to be semi-sync slaves (i.e. to send ACKs). Vitess slaves marked as
"rdonly" will NOT ACK, because they are not eligible to become masters,
so we never want them to be the only slave with the latest update.
This means when using semi-sync, every shard must have at least 2 live
"replica" type tablets (including the master) in order to accept writes.
This gets rid of the opaque mysql-db-dir.tbz archive, replacing it with
a .sql file. The .sql file approach makes it clear what state the DB is
initialized with, and also makes it easy to customize.
This option was renamed in MySQL 5.5, so it has been silently ignored
since we moved to MariaDB 10.0. I found out it was wrong because MySQL
5.6 quits if it sees an unknown option.
Make min rdonly instances configurable for vtworker tasks
Remove unnecessary tablets wherever possible
Use lower buffer pool size in MySQL to save memory
Track performance using New Relic
teejae
Sugu Sougoumarane
Deepthi Sigireddi
deepthi
Dan Kozlowski
Derek Perkins
DK
Rafael Chacon
Daniël van Eeden
Alain Jobart
Martin Fris
Pavel Ivanov
Joshua Thompson
Anthony Yeh
Anand Henry
Anand Henry
Veres Lajos
shrutip