We can now add annotations to the service accounts in a generic
way. This allows for example to add Workflow Identitty in GKE
environment but it is not limited to it.
Co-authored-by: Kamil Breguła <kamil.bregula@polidea.com>
Co-authored-by: Jacob Ferriero <jferriero@google.com>
Co-authored-by: Kamil Breguła <kamil.bregula@polidea.com>
* Multi-Namespace mode for helm chart
Users should not REQUIRE a ClusterRole/ClusterRolebinding
to run airflow via helm. This change will allow "single" and "multi"
namespace modes so users can add airflow to managed kubernetes clusters
* add namespace to role
* add rolebinding too
* add docs
* add values.schema.json change
Some of the users of Airflow are using Kerberos to authenticate
their worker workflows. Airflow has a basic support for Kerberos
for some of the operators and it has support to refresh the
temporary Kerberos tokens via `airflow kerberos` command.
This change adds support for the Kerberos side-car that connects
to the Kerberos Key Distribution Center and retrieves the
token using Keytab that should be deployed as Kubernetes Secret.
It uses shared volume to share the temporary token. The nice
thing about setting it up as a sidecar is that the Keytab
is never shared with the workers - the secret is only mounted
by the sidecar and the workers have only access to the temporary
token.
Depends on #11129
* add git sync sidecars
* add a helm test
* add more tests
* allow users to provide git username and pass via a k8s secrets
* set default values for airflow worker repository & tag
* change ci timeout
* fix link
* add credentials_secret to airflow.cfg configmap
* set GIT_SYNC_ADD_USER on kubernetes worker pods, set uid
* add fsGroup to webserver and kubernete workers
* move gitSync to dags.gitSync
* rename valueFields
* turn off git sync and dag persistence by default
* provide option to specify known_hosts
* add git-sync details into the chart documentation
* Update .gitignore
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
* make git sync max failures configurable
* Apply suggestions from code review
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
* add back requirements.lock
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
The Kubernetes tests are now run using Helm chart
rather than the custom templates we used to have.
The Helm Chart uses locally build production image
so the tests are testing not only Airflow but also
Helm Chart and a Production image - all at the
same time. Later on we will add more tests
covering more functionalities of both Helm Chart
and Production Image. This is the first step to
get all of those bundle together and become
testable.
This change introduces also 'shell' sub-command
for Breeze's kind-cluster command and
EMBEDDED_DAGS build args for production image -
both of them useful to run the Kubernetes tests
more easily - without building two images
and with an easy-to-iterate-over-tests
shell command - which works without any
other development environment.
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Co-authored-by: Daniel Imberman <daniel@astronomer.io>
Jarek Potiuk
Daniel Imberman
Can Güney Aksakalli
David Cavaletto
Vicken Simonian
Aneesh Joseph
Bryant Larsen
Kaxil Naik
Ash Berlin-Taylor
Greg Neiheisel