зеркало из https://github.com/github/vitess-gh.git
helm: allow further customization through additional flags and secrets (#4333)
* Add extraFlags and secrets for vitess components to be able to configure transport encryption. * Fixed indentation of volumeMounts. * Fixed whitespace issues in templates, where helm would remove essential newlines. * Fixed minor typo in vtgate template. * Use toJson and trimAll to escape vtctlclient orchestrator flags. * Use override default secrets instead of mounting both. * Reverting accidental change of comments in helper template. * Using inline flags helper for InitShardMaster-jobs. * Added grpc TLS documentation. * Add section about slave replication traffic encryption and percona at rest encryption using the vault plugin. * Minor fixes to README and a few more comments to helm values. * Use bash array to avoid pitfalls providing flags as raw string * Add vtctlclient.extraFlags support to preStop hook.
This commit is contained in:
Nik Voss
Derek Perkins
Родитель
782fec9ad8
Коммит
f5ceb5abc8
|
|
@ -283,3 +283,97 @@ topology:
|
|||
orchestrator:
|
||||
enabled: true
|
||||
```
|
||||
|
||||
### Enable TLS encryption for vitess grpc communication
|
||||
|
||||
Each component of vitess requires a certificate and private key to secure incoming requests and further configuration for every outgoing connection. In this example TLS certificates were generated and stored in several kubernetes secrets:
|
||||
```yaml
|
||||
vttablet:
|
||||
extraFlags:
|
||||
# configure which certificates to use for serving grpc requests
|
||||
grpc_cert: /vt/usersecrets/vttablet-tls/vttablet.pem
|
||||
grpc_key: /vt/usersecrets/vttablet-tls/vttablet-key.pem
|
||||
tablet_grpc_ca: /vt/usersecrets/vttablet-tls/vitess-ca.pem
|
||||
tablet_grpc_server_name: vttablet
|
||||
secrets:
|
||||
- vttablet-tls
|
||||
|
||||
vtctld:
|
||||
extraFlags:
|
||||
grpc_cert: /vt/usersecrets/vtctld-tls/vtctld.pem
|
||||
grpc_key: /vt/usersecrets/vtctld-tls/vtctld-key.pem
|
||||
tablet_grpc_ca: /vt/usersecrets/vtctld-tls/vitess-ca.pem
|
||||
tablet_grpc_server_name: vttablet
|
||||
tablet_manager_grpc_ca: /vt/usersecrets/vtctld-tls/vitess-ca.pem
|
||||
tablet_manager_grpc_server_name: vttablet
|
||||
secrets:
|
||||
- vtctld-tls
|
||||
|
||||
vtctlclient: # configuration used by both InitShardMaster-jobs and orchestrator to be able to communicate with vtctld
|
||||
extraFlags:
|
||||
vtctld_grpc_ca: /vt/usersecrets/vitess-ca/vitess-ca.pem
|
||||
vtctld_grpc_server_name: vtctld
|
||||
secrets:
|
||||
- vitess-ca
|
||||
|
||||
vtgate:
|
||||
extraFlags:
|
||||
grpc_cert: /vt/usersecrets/vtgate-tls/vtgate.pem
|
||||
grpc_key: /vt/usersecrets/vtgate-tls/vtgate-key.pem
|
||||
tablet_grpc_ca: /vt/usersecrets/vtgate-tls/vitess-ca.pem
|
||||
tablet_grpc_server_name: vttablet
|
||||
secrets:
|
||||
- vtgate-tls
|
||||
```
|
||||
|
||||
### Slave replication traffic encryption
|
||||
|
||||
To encrypt traffic between slaves and master additional flags can be provided. By default MySQL generates self-signed certificates on startup (otherwise specify `ssl_*` settings within you `extraMyCnf`), that can be used to encrypt the traffic:
|
||||
```
|
||||
vttablet:
|
||||
extraFlags:
|
||||
db_flags: 2048
|
||||
db_repl_use_ssl: true
|
||||
db-config-repl-flags: 2048
|
||||
|
||||
```
|
||||
|
||||
### Percona at rest encryption using the vault plugin
|
||||
|
||||
To use the [percona at rest encryption](https://www.percona.com/doc/percona-server/LATEST/management/data_at_rest_encryption.html) several additional settings have to be provided via an `extraMyCnf`-file. This makes only sense if the traffic is encrypted as well (see above sections), since binlog replication is unencrypted by default.
|
||||
```
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: vttablet-extra-config
|
||||
namespace: vitess
|
||||
data:
|
||||
extra.cnf: |-
|
||||
early-plugin-load=keyring_vault=keyring_vault.so
|
||||
# this includes default rpl plugins, see https://github.com/vitessio/vitess/blob/master/config/mycnf/master_mysql56.cnf for details
|
||||
plugin-load=rpl_semi_sync_master=semisync_master.so;rpl_semi_sync_slave=semisync_slave.so;keyring_udf=keyring_udf.so
|
||||
keyring_vault_config=/vt/usersecrets/vttablet-vault/vault.conf # load keyring configuration from secret
|
||||
innodb_encrypt_tables=ON # encrypt all tables by default
|
||||
encrypt_binlog=ON # binlog encryption
|
||||
master_verify_checksum=ON # necessary for binlog encryption
|
||||
binlog_checksum=CRC32 # necessary for binlog encryption
|
||||
encrypt-tmp-files=ON # use temporary AES keys to encrypt temporary files
|
||||
```
|
||||
|
||||
An example vault configuration, which is provided by the `vttablet-vault`-Secret in the above example:
|
||||
```
|
||||
vault_url = https://10.0.0.1:8200
|
||||
secret_mount_point = vitess
|
||||
token = 11111111-1111-1111-1111111111
|
||||
vault_ca = /vt/usersecrets/vttablet-vault/vault-ca-bundle.pem
|
||||
```
|
||||
|
||||
At last add the secret containing the vault configuration and the additional MySQL-configuration to your helm values:
|
||||
```
|
||||
vttablet:
|
||||
flavor: "percona" # only works with percona
|
||||
mysqlImage: "percona:5.7.23"
|
||||
extraMyCnf: vttablet-extra-config
|
||||
secrets:
|
||||
- vttablet-vault
|
||||
```
|
||||
|
|
|
|||
|
|
@ -11,6 +11,17 @@
|
|||
{{end -}}
|
||||
{{- end -}}
|
||||
|
||||
############################
|
||||
# Format a flag map into a command line (inline),
|
||||
# as expected by the golang 'flag' package.
|
||||
# Boolean flags must be given a value, such as "true" or "false".
|
||||
#############################
|
||||
{{- define "format-flags-inline" -}}
|
||||
{{- range $key, $value := . -}}
|
||||
-{{$key}}={{$value | quote}}{{" "}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
||||
#############################
|
||||
# Repeat a string N times, where N is the total number
|
||||
# of replicas. Len must be used on the calling end to
|
||||
|
|
@ -114,7 +125,7 @@ fi
|
|||
export EXTRA_MY_CNF="$FLAVOR_MYCNF:/vtdataroot/tabletdata/report-host.cnf:/vt/config/mycnf/rbr.cnf"
|
||||
|
||||
{{ if . }}
|
||||
for filename in /vt/userconfig/*; do
|
||||
for filename in /vt/userconfig/*.cnf; do
|
||||
export EXTRA_MY_CNF="$EXTRA_MY_CNF:$filename"
|
||||
done
|
||||
{{ end }}
|
||||
|
|
@ -350,3 +361,32 @@ cat $CEPH_CREDENTIALS_FILE
|
|||
{{ end }}
|
||||
|
||||
{{- end -}}
|
||||
|
||||
#############################
|
||||
# user secret volumes - expects list of secret names
|
||||
#############################
|
||||
{{- define "user-secret-volumes" -}}
|
||||
|
||||
{{ if . }}
|
||||
{{- range . }}
|
||||
- name: user-secret-{{ . }}
|
||||
secret:
|
||||
secretName: {{ . }}
|
||||
{{- end }}
|
||||
{{ end }}
|
||||
|
||||
{{- end -}}
|
||||
|
||||
#############################
|
||||
# user secret volumeMounts - expects list of secret names
|
||||
#############################
|
||||
{{- define "user-secret-volumeMounts" -}}
|
||||
|
||||
{{ if . }}
|
||||
{{- range . }}
|
||||
- name: user-secret-{{ . }}
|
||||
mountPath: /vt/usersecrets/{{ . }}
|
||||
{{- end }}
|
||||
{{ end }}
|
||||
|
||||
{{- end -}}
|
||||
|
|
|
|||
|
|
@ -6,12 +6,13 @@
|
|||
{{- $orc := index . 0 -}}
|
||||
{{- $namespace := index . 1 -}}
|
||||
{{- $enableHeartbeat := index . 2 -}}
|
||||
{{- $defaultVtctlclient := index . 3 }}
|
||||
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: orchestrator-cm
|
||||
data:
|
||||
data:
|
||||
orchestrator.conf.json: |-
|
||||
{
|
||||
"ActiveNodeExpireSeconds": 5,
|
||||
|
|
@ -77,7 +78,7 @@ data:
|
|||
],
|
||||
"PostMasterFailoverProcesses": [
|
||||
"echo 'Recovered from {failureType} on {failureCluster}. Failed: {failedHost}:{failedPort}; Promoted: {successorHost}:{successorPort}' >> /tmp/recovery.log",
|
||||
"vtctlclient -server vtctld.{{ $namespace }}:15999 TabletExternallyReparented {successorAlias}"
|
||||
"vtctlclient {{ include "format-flags-inline" $defaultVtctlclient.extraFlags | toJson | trimAll "\"" }} -server vtctld.{{ $namespace }}:15999 TabletExternallyReparented {successorAlias}"
|
||||
],
|
||||
"PostponeSlaveRecoveryOnLagMinutes": 0,
|
||||
"PostUnsuccessfulFailoverProcesses": [
|
||||
|
|
@ -123,7 +124,7 @@ data:
|
|||
"ReplicationLagQuery": "SELECT unix_timestamp() - floor(ts/1000000000) FROM `_vt`.heartbeat ORDER BY ts DESC LIMIT 1;",
|
||||
{{ else }}
|
||||
"ReplicationLagQuery": "",
|
||||
{{ end }}
|
||||
{{ end }}
|
||||
"ServeAgentsHttp": false,
|
||||
"SkipBinlogEventsContaining": [
|
||||
],
|
||||
|
|
@ -148,4 +149,4 @@ data:
|
|||
"UseSSL": false,
|
||||
"VerifyReplicationFilters": false
|
||||
}
|
||||
{{ end }}
|
||||
{{ end }}
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@
|
|||
{{- define "orchestrator" -}}
|
||||
# set tuple values to more recognizable variables
|
||||
{{- $orc := index . 0 -}}
|
||||
{{- $defaultVtctlclient := index . 1 }}
|
||||
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
|
|
@ -58,7 +59,7 @@ spec:
|
|||
serviceName: orchestrator-headless
|
||||
replicas: {{ $orc.replicas }}
|
||||
podManagementPolicy: Parallel
|
||||
updateStrategy:
|
||||
updateStrategy:
|
||||
type: RollingUpdate
|
||||
selector:
|
||||
matchLabels:
|
||||
|
|
@ -116,7 +117,7 @@ spec:
|
|||
mountPath: /conf/
|
||||
- name: tmplogs
|
||||
mountPath: /tmp
|
||||
|
||||
{{ include "user-secret-volumeMounts" $defaultVtctlclient.secrets | indent 12 }}
|
||||
env:
|
||||
- name: VTCTLD_SERVER_PORT
|
||||
value: "15999"
|
||||
|
|
@ -145,6 +146,7 @@ spec:
|
|||
emptyDir: {}
|
||||
- name: tmplogs
|
||||
emptyDir: {}
|
||||
{{ include "user-secret-volumes" $defaultVtctlclient.secrets | indent 8 }}
|
||||
|
||||
{{- end -}}
|
||||
|
||||
|
|
@ -154,7 +156,7 @@ spec:
|
|||
{{- define "orchestrator-statefulset-service" -}}
|
||||
# set tuple values to more recognizable variables
|
||||
{{- $orc := index . 0 -}}
|
||||
{{- $i := index . 1 -}}
|
||||
{{- $i := index . 1 }}
|
||||
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
|
|
|
|||
|
|
@ -76,7 +76,7 @@ spec:
|
|||
{{ include "backup-env" $config.backup | indent 12 }}
|
||||
volumeMounts:
|
||||
{{ include "backup-volumeMount" $config.backup | indent 12 }}
|
||||
|
||||
{{ include "user-secret-volumeMounts" (.secrets | default $defaultVtctld.secrets) | indent 12 }}
|
||||
resources:
|
||||
{{ toYaml (.resources | default $defaultVtctld.resources) | indent 12 }}
|
||||
command:
|
||||
|
|
@ -102,11 +102,13 @@ spec:
|
|||
-topo_global_server_address="etcd-global-client.{{ $namespace }}:2379"
|
||||
-topo_global_root=/vitess/global
|
||||
{{ include "backup-flags" (tuple $config.backup "vtctld") | indent 16 }}
|
||||
{{ include "format-flags-all" (tuple $defaultVtctld.extraFlags .extraFlags) | indent 16 }}
|
||||
END_OF_COMMAND
|
||||
)
|
||||
|
||||
volumes:
|
||||
{{ include "backup-volume" $config.backup | indent 8 }}
|
||||
{{ include "user-secret-volumes" (.secrets | default $defaultVtctld.secrets) | indent 8 }}
|
||||
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
|
@ -125,4 +127,4 @@ affinity:
|
|||
{{ include "node-affinity" $region | indent 2 }}
|
||||
{{- end -}}
|
||||
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
|
|
|||
|
|
@ -88,7 +88,7 @@ spec:
|
|||
volumeMounts:
|
||||
- name: creds
|
||||
mountPath: "/mysqlcreds"
|
||||
|
||||
{{ include "user-secret-volumeMounts" (.secrets | default $defaultVtgate.secrets) | indent 12 }}
|
||||
resources:
|
||||
{{ toYaml (.resources | default $defaultVtgate.resources) | indent 12 }}
|
||||
|
||||
|
|
@ -121,7 +121,7 @@ spec:
|
|||
volumes:
|
||||
- name: creds
|
||||
emptyDir: {}
|
||||
|
||||
{{ include "user-secret-volumes" (.secrets | default $defaultVtgate.secrets) | indent 8 }}
|
||||
---
|
||||
###################################
|
||||
# vtgate PodDisruptionBudget
|
||||
|
|
@ -205,7 +205,7 @@ affinity:
|
|||
|
||||
###################################
|
||||
# init-container to set mysql credentials file
|
||||
# it loops through the users and pulls out their
|
||||
# it loops through the users and pulls out their
|
||||
# respective passwords from mounted secrets
|
||||
###################################
|
||||
{{- define "init-mysql-creds" -}}
|
||||
|
|
@ -246,4 +246,4 @@ affinity:
|
|||
echo $creds > /mysqlcreds/creds.json
|
||||
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
{{- end -}}
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
###################################
|
||||
{{- define "vttablet-service" -}}
|
||||
# set tuple values to more recognizable variables
|
||||
{{- $pmm := index . 0 -}}
|
||||
{{- $pmm := index . 0 }}
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
|
|
@ -33,7 +33,7 @@ spec:
|
|||
{{- end -}}
|
||||
|
||||
###################################
|
||||
# vttablet
|
||||
# vttablet
|
||||
###################################
|
||||
{{- define "vttablet" -}}
|
||||
# set tuple values to more recognizable variables
|
||||
|
|
@ -43,11 +43,12 @@ spec:
|
|||
{{- $shard := index . 3 -}}
|
||||
{{- $tablet := index . 4 -}}
|
||||
{{- $defaultVttablet := index . 5 -}}
|
||||
{{- $namespace := index . 6 -}}
|
||||
{{- $config := index . 7 -}}
|
||||
{{- $pmm := index . 8 -}}
|
||||
{{- $orc := index . 9 -}}
|
||||
{{- $totalTabletCount := index . 10 -}}
|
||||
{{- $defaultVtctlclient := index . 6 -}}
|
||||
{{- $namespace := index . 7 -}}
|
||||
{{- $config := index . 8 -}}
|
||||
{{- $pmm := index . 9 -}}
|
||||
{{- $orc := index . 10 -}}
|
||||
{{- $totalTabletCount := index . 11 -}}
|
||||
|
||||
# sanitize inputs to create tablet name
|
||||
{{- $cellClean := include "clean-label" $cell.name -}}
|
||||
|
|
@ -75,7 +76,7 @@ spec:
|
|||
serviceName: vttablet
|
||||
replicas: {{ .replicas | default $defaultVttablet.replicas }}
|
||||
podManagementPolicy: Parallel
|
||||
updateStrategy:
|
||||
updateStrategy:
|
||||
type: RollingUpdate
|
||||
selector:
|
||||
matchLabels:
|
||||
|
|
@ -105,7 +106,7 @@ spec:
|
|||
|
||||
containers:
|
||||
{{ include "cont-mysql" (tuple $topology $cell $keyspace $shard $tablet $defaultVttablet $uid) | indent 8 }}
|
||||
{{ include "cont-vttablet" (tuple $topology $cell $keyspace $shard $tablet $defaultVttablet $vitessTag $uid $namespace $config $orc $totalTabletCount) | indent 8 }}
|
||||
{{ include "cont-vttablet" (tuple $topology $cell $keyspace $shard $tablet $defaultVttablet $defaultVtctlclient $vitessTag $uid $namespace $config $orc $totalTabletCount) | indent 8 }}
|
||||
{{ include "cont-logrotate" . | indent 8 }}
|
||||
{{ include "cont-mysql-generallog" . | indent 8 }}
|
||||
{{ include "cont-mysql-errorlog" . | indent 8 }}
|
||||
|
|
@ -116,7 +117,8 @@ spec:
|
|||
- name: vt
|
||||
emptyDir: {}
|
||||
{{ include "backup-volume" $config.backup | indent 8 }}
|
||||
{{ include "user-config-volume" $defaultVttablet.extraMyCnf | indent 8 }}
|
||||
{{ include "user-config-volume" (.extraMyCnf | default $defaultVttablet.extraMyCnf) | indent 8 }}
|
||||
{{ include "user-secret-volumes" (.secrets | default $defaultVttablet.secrets) | indent 8 }}
|
||||
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
|
|
@ -162,6 +164,8 @@ spec:
|
|||
containers:
|
||||
- name: init-shard-master
|
||||
image: "vitess/vtctlclient:{{$vitessTag}}"
|
||||
volumeMounts:
|
||||
{{ include "user-secret-volumeMounts" $defaultVtctlclient.secrets | indent 10 }}
|
||||
|
||||
command: ["bash"]
|
||||
args:
|
||||
|
|
@ -172,9 +176,10 @@ spec:
|
|||
VTCTLD_SVC=vtctld.{{ $namespace }}:15999
|
||||
SECONDS=0
|
||||
TIMEOUT_SECONDS=600
|
||||
VTCTL_EXTRA_FLAGS=({{ include "format-flags-inline" $defaultVtctlclient.extraFlags }})
|
||||
|
||||
# poll every 5 seconds to see if vtctld is ready
|
||||
until vtctlclient -server $VTCTLD_SVC ListAllTablets {{ $cellClean }} > /dev/null 2>&1; do
|
||||
until vtctlclient ${VTCTL_EXTRA_FLAGS[@]} -server $VTCTLD_SVC ListAllTablets {{ $cellClean }} > /dev/null 2>&1; do
|
||||
if (( $SECONDS > $TIMEOUT_SECONDS )); then
|
||||
echo "timed out waiting for vtctlclient to be ready"
|
||||
exit 1
|
||||
|
|
@ -184,8 +189,8 @@ spec:
|
|||
|
||||
until [ $TABLETS_READY ]; do
|
||||
# get all the tablets in the current cell
|
||||
cellTablets="$(vtctlclient -server $VTCTLD_SVC ListAllTablets {{ $cellClean }})"
|
||||
|
||||
cellTablets="$(vtctlclient ${VTCTL_EXTRA_FLAGS[@]} -server $VTCTLD_SVC ListAllTablets {{ $cellClean }})"
|
||||
|
||||
# filter to only the tablets in our current shard
|
||||
shardTablets=$( echo "$cellTablets" | awk 'substr( $5,1,{{ len $shardName }} ) == "{{ $shardName }}" {print $0}')
|
||||
|
||||
|
|
@ -197,7 +202,7 @@ spec:
|
|||
fi
|
||||
|
||||
# check for a master tablet from the GetShard call
|
||||
master_alias=$(vtctlclient -server $VTCTLD_SVC GetShard {{ $keyspace.name }}/{{ $shard.name }} | jq '.master_alias.uid')
|
||||
master_alias=$(vtctlclient ${VTLCTL_EXTRA_FLAGS[@]} -server $VTCTLD_SVC GetShard {{ $keyspace.name }}/{{ $shard.name }} | jq '.master_alias.uid')
|
||||
if [ $master_alias != "null" ]; then
|
||||
echo "'$master_alias' is already the master tablet, exiting without running InitShardMaster"
|
||||
exit
|
||||
|
|
@ -205,7 +210,7 @@ spec:
|
|||
|
||||
# count the number of newlines for the given shard to get the tablet count
|
||||
tabletCount=$( echo "$shardTablets" | wc | awk '{print $1}')
|
||||
|
||||
|
||||
# check to see if the tablet count equals the expected tablet count
|
||||
if [ $tabletCount == {{ $totalTabletCount }} ]; then
|
||||
TABLETS_READY=true
|
||||
|
|
@ -214,7 +219,7 @@ spec:
|
|||
echo "timed out waiting for tablets to be ready"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
# wait 5 seconds for vttablets to continue getting ready
|
||||
sleep 5
|
||||
fi
|
||||
|
|
@ -223,16 +228,18 @@ spec:
|
|||
|
||||
# find the tablet id for the "-replica-0" stateful set for a given cell, keyspace and shard
|
||||
tablet_id=$( echo "$shardTablets" | awk 'substr( $5,1,{{ add (len $shardName) 10 }} ) == "{{ $shardName }}-replica-0" {print $1}')
|
||||
|
||||
|
||||
# initialize the shard master
|
||||
until vtctlclient -server $VTCTLD_SVC InitShardMaster -force {{ $keyspace.name }}/{{ $shard.name }} $tablet_id; do
|
||||
until vtctlclient ${VTCTL_EXTRA_FLAGS[@]} -server $VTCTLD_SVC InitShardMaster -force {{ $keyspace.name }}/{{ $shard.name }} $tablet_id; do
|
||||
if (( $SECONDS > $TIMEOUT_SECONDS )); then
|
||||
echo "timed out waiting for InitShardMaster to succeed"
|
||||
exit 1
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
|
||||
volumes:
|
||||
{{ include "user-secret-volumes" (.secrets | default $defaultVtctlclient.secrets) | indent 8 }}
|
||||
|
||||
{{- end -}}
|
||||
|
||||
{{- end -}}
|
||||
|
|
@ -340,12 +347,13 @@ spec:
|
|||
{{- $shard := index . 3 -}}
|
||||
{{- $tablet := index . 4 -}}
|
||||
{{- $defaultVttablet := index . 5 -}}
|
||||
{{- $vitessTag := index . 6 -}}
|
||||
{{- $uid := index . 7 -}}
|
||||
{{- $namespace := index . 8 -}}
|
||||
{{- $config := index . 9 -}}
|
||||
{{- $orc := index . 10 -}}
|
||||
{{- $totalTabletCount := index . 11 -}}
|
||||
{{- $defaultVtctlclient := index . 6 -}}
|
||||
{{- $vitessTag := index . 7 -}}
|
||||
{{- $uid := index . 8 -}}
|
||||
{{- $namespace := index . 9 -}}
|
||||
{{- $config := index . 10 -}}
|
||||
{{- $orc := index . 11 -}}
|
||||
{{- $totalTabletCount := index . 12 -}}
|
||||
|
||||
{{- $cellClean := include "clean-label" $cell.name -}}
|
||||
{{- with $tablet.vttablet -}}
|
||||
|
|
@ -368,7 +376,8 @@ spec:
|
|||
- name: vtdataroot
|
||||
mountPath: "/vtdataroot"
|
||||
{{ include "backup-volumeMount" $config.backup | indent 4 }}
|
||||
{{ include "user-config-volumeMount" $defaultVttablet.extraMyCnf | indent 4 }}
|
||||
{{ include "user-config-volumeMount" (.extraMyCnf | default $defaultVttablet.extraMyCnf) | indent 4 }}
|
||||
{{ include "user-secret-volumeMounts" (.secrets | default $defaultVttablet.secrets) | indent 4 }}
|
||||
|
||||
resources:
|
||||
{{ toYaml (.resources | default $defaultVttablet.resources) | indent 6 }}
|
||||
|
|
@ -397,8 +406,9 @@ spec:
|
|||
set -x
|
||||
|
||||
VTCTLD_SVC=vtctld.{{ $namespace }}:15999
|
||||
VTCTL_EXTRA_FLAGS=({{ include "format-flags-inline" $defaultVtctlclient.extraFlags }})
|
||||
|
||||
master_alias_json=$(/vt/bin/vtctlclient -server $VTCTLD_SVC GetShard {{ $keyspace.name }}/{{ $shard.name }})
|
||||
master_alias_json=$(/vt/bin/vtctlclient ${VTCTL_EXTRA_FLAGS[@]} -server $VTCTLD_SVC GetShard {{ $keyspace.name }}/{{ $shard.name }})
|
||||
master_cell=$(jq -r '.master_alias.cell' <<< "$master_alias_json")
|
||||
master_uid=$(jq -r '.master_alias.uid' <<< "$master_alias_json")
|
||||
master_alias=$master_cell-$master_uid
|
||||
|
|
@ -423,7 +433,7 @@ spec:
|
|||
until [ $DONE_REPARENTING ]; do
|
||||
|
||||
# reparent before shutting down
|
||||
/vt/bin/vtctlclient -server $VTCTLD_SVC PlannedReparentShard -keyspace_shard={{ $keyspace.name }}/{{ $shard.name }} -avoid_master=$current_alias
|
||||
/vt/bin/vtctlclient ${VTCTL_EXTRA_FLAGS[@]} -server $VTCTLD_SVC PlannedReparentShard -keyspace_shard={{ $keyspace.name }}/{{ $shard.name }} -avoid_master=$current_alias
|
||||
|
||||
# if PlannedReparentShard succeeded, then don't retry
|
||||
if [ $? -eq 0 ]; then
|
||||
|
|
@ -443,7 +453,7 @@ spec:
|
|||
|
||||
# delete the current tablet from topology. Not strictly necessary, but helps to prevent
|
||||
# edge cases where there are two masters
|
||||
/vt/bin/vtctlclient -server $VTCTLD_SVC DeleteTablet $current_alias
|
||||
/vt/bin/vtctlclient ${VTCTL_EXTRA_FLAGS[@]} -server $VTCTLD_SVC DeleteTablet $current_alias
|
||||
|
||||
command: ["bash"]
|
||||
args:
|
||||
|
|
@ -451,9 +461,9 @@ spec:
|
|||
- |
|
||||
set -ex
|
||||
|
||||
{{ include "mycnf-exec" $defaultVttablet.extraMyCnf | indent 6 }}
|
||||
{{ include "mycnf-exec" (.extraMyCnf | default $defaultVttablet.extraMyCnf) | indent 6 }}
|
||||
{{ include "backup-exec" $config.backup | indent 6 }}
|
||||
|
||||
|
||||
eval exec /vt/bin/vttablet $(cat <<END_OF_COMMAND
|
||||
-topo_implementation="etcd2"
|
||||
-topo_global_server_address="etcd-global-client.{{ $namespace }}:2379"
|
||||
|
|
@ -482,6 +492,7 @@ spec:
|
|||
-orc_discover_interval "5m"
|
||||
{{ end }}
|
||||
{{ include "backup-flags" (tuple $config.backup "vttablet") | indent 8 }}
|
||||
{{ include "format-flags-all" (tuple $defaultVttablet.extraFlags .extraFlags) | indent 8 }}
|
||||
END_OF_COMMAND
|
||||
)
|
||||
{{- end -}}
|
||||
|
|
@ -516,7 +527,8 @@ spec:
|
|||
mountPath: /vtdataroot
|
||||
- name: vt
|
||||
mountPath: /vt
|
||||
{{ include "user-config-volumeMount" $defaultVttablet.extraMyCnf | indent 4 }}
|
||||
{{ include "user-config-volumeMount" (.extraMyCnf | default $defaultVttablet.extraMyCnf) | indent 4 }}
|
||||
{{ include "user-secret-volumeMounts" (.secrets | $defaultVttablet.secrets) | indent 4 }}
|
||||
resources:
|
||||
{{ toYaml (.mysqlResources | default $defaultVttablet.mysqlResources) | indent 6 }}
|
||||
env:
|
||||
|
|
@ -555,8 +567,7 @@ spec:
|
|||
- "-c"
|
||||
- |
|
||||
set -ex
|
||||
|
||||
{{ include "mycnf-exec" $defaultVttablet.extraMyCnf | indent 6 }}
|
||||
{{ include "mycnf-exec" (.extraMyCnf | default $defaultVttablet.extraMyCnf) | indent 6 }}
|
||||
|
||||
eval exec /vt/bin/mysqlctld $(cat <<END_OF_COMMAND
|
||||
-logtostderr=true
|
||||
|
|
@ -678,7 +689,7 @@ affinity:
|
|||
cell: {{ $cellClean | quote }}
|
||||
keyspace: {{ $keyspaceClean | quote }}
|
||||
shard: {{ $shardClean | quote }}
|
||||
|
||||
|
||||
# prefer to stay away from any vttablets
|
||||
- weight: 10
|
||||
podAffinityTerm:
|
||||
|
|
|
|||
|
|
@ -12,10 +12,10 @@
|
|||
{{ if $.Values.orchestrator.enabled }}
|
||||
|
||||
# create orchestrator global services and StatefulSet
|
||||
{{ include "orchestrator" (tuple $.Values.orchestrator) }}
|
||||
{{ include "orchestrator" (tuple $.Values.orchestrator $.Values.vtctlclient) }}
|
||||
---
|
||||
# create orchestrator config map
|
||||
{{ include "orchestrator-config" (tuple $.Values.orchestrator $.Release.Namespace $.Values.vttablet.enableHeartbeat) }}
|
||||
{{ include "orchestrator-config" (tuple $.Values.orchestrator $.Release.Namespace $.Values.vttablet.enableHeartbeat $.Values.vtctlclient) }}
|
||||
---
|
||||
# create a Service per StatefulSet replica
|
||||
{{ range $i := until (int $.Values.orchestrator.replicas) }}
|
||||
|
|
@ -59,7 +59,7 @@
|
|||
# now range through the tablets again to set them up
|
||||
{{ range $tablet := $shard.tablets }}
|
||||
---
|
||||
{{ include "vttablet" (tuple $.Values.topology $cell $keyspace $shard $tablet $.Values.vttablet $.Release.Namespace $.Values.config $.Values.pmm $.Values.orchestrator $totalTabletCount) }}
|
||||
{{ include "vttablet" (tuple $.Values.topology $cell $keyspace $shard $tablet $.Values.vttablet $.Values.vtctlclient $.Release.Namespace $.Values.config $.Values.pmm $.Values.orchestrator $totalTabletCount) }}
|
||||
|
||||
{{ end }} # range $tablet
|
||||
{{ end }} # range $shard
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ config:
|
|||
# Backup flags will be applied to components that need them.
|
||||
# These are defined globally since all components should agree.
|
||||
backup:
|
||||
|
||||
|
||||
enabled: false
|
||||
|
||||
# choose a backup service - valid values are gcs/s3
|
||||
|
|
@ -50,10 +50,10 @@ config:
|
|||
|
||||
# S3 bucket to use for backups
|
||||
# s3_backup_storage_bucket: "vitess-backups"
|
||||
|
||||
|
||||
# root prefix for all backup-related object names
|
||||
# s3_backup_storage_root: "vtbackups"
|
||||
|
||||
|
||||
# server-side encryption algorithm (e.g., AES256, aws:kms)
|
||||
# s3_backup_server_side_encryption: "AES256"
|
||||
|
||||
|
|
@ -109,6 +109,8 @@ vtctld:
|
|||
limits:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
extraFlags: {}
|
||||
secrets: [] # secrets are mounted under /vt/usersecrets/{secretname}
|
||||
|
||||
# Default values for vtgate resources defined in 'topology'
|
||||
vtgate:
|
||||
|
|
@ -118,12 +120,20 @@ vtgate:
|
|||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
extraFlags: {}
|
||||
secrets: [] # secrets are mounted under /vt/usersecrets/{secretname}
|
||||
|
||||
# Default values for vtctlclient resources defined in 'topology'
|
||||
vtctlclient:
|
||||
extraFlags: {}
|
||||
secrets: [] # secrets are mounted under /vt/usersecrets/{secretname}
|
||||
|
||||
|
||||
# Default values for vttablet resources defined in 'topology'
|
||||
vttablet:
|
||||
vitessTag: "latest"
|
||||
|
||||
# valid values are
|
||||
# valid values are
|
||||
# - mysql56 (for MySQL/Percona 5.6 or 5.7)
|
||||
# - mariadb (for MariaDB <= 10.2)
|
||||
# - mariadb103 (for MariaDB >= 10.3)
|
||||
|
|
@ -145,6 +155,12 @@ vttablet:
|
|||
extraMyCnf: ""
|
||||
# extraMyCnf: extra-my-cnf
|
||||
|
||||
# Additional flags that will be appended to the vttablet command
|
||||
extraFlags: {}
|
||||
|
||||
# User secrets that will be mounted under /vt/usersecrets/{secretname}/
|
||||
secrets: []
|
||||
|
||||
resources:
|
||||
# common production values 2-4CPU/4-8Gi RAM
|
||||
limits:
|
||||
|
|
@ -203,8 +219,8 @@ pmm:
|
|||
metricsResolution: 1s
|
||||
|
||||
# METRICS_RETENTION (Option)
|
||||
# This option determines how long metrics are stored at PMM Server.
|
||||
# The value is passed as a combination of hours, minutes, and seconds, such as 720h0m0s.
|
||||
# This option determines how long metrics are stored at PMM Server.
|
||||
# The value is passed as a combination of hours, minutes, and seconds, such as 720h0m0s.
|
||||
# The minutes (a number followed by m) and seconds (a number followed by s) are optional.
|
||||
metricsRetention: 720h
|
||||
|
||||
|
|
@ -216,10 +232,10 @@ pmm:
|
|||
# NOTE: The value must be passed in kilobytes
|
||||
# NOTE: Make sure to quote this value so it isn't converted into scientific notation
|
||||
|
||||
# By default, Prometheus in PMM Server uses up to 768 MB of memory for storing the most recently used data chunks.
|
||||
# By default, Prometheus in PMM Server uses up to 768 MB of memory for storing the most recently used data chunks.
|
||||
# Depending on the amount of data coming into Prometheus, you may require a higher limit to avoid throttling data ingestion,
|
||||
# or allow less memory consumption if it is needed for other processes.
|
||||
# The limit affects only memory reserved for data chunks. Actual RAM usage by Prometheus is higher.
|
||||
# The limit affects only memory reserved for data chunks. Actual RAM usage by Prometheus is higher.
|
||||
# It is recommended to set this limit to roughly 2/3 of the total memory that you are planning to allow for Prometheus.
|
||||
metricsMemory: "600000"
|
||||
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче