During testing v1-10-test backport for Breeze the
--github-repository flag did not work. It turned out that
the lowercase variable was not re-set when the flag was
provided by Breeze.
This change causes the lowercasing to be run just before it
is used to make sure that the GITHUB_REPOSITORY value
is used after it's been overwritten.
When we ported the new CI mechanism to v1-10-test it turned out
that we have to correct the retrieval of DEFAULT BRANCH
and DEFAULT_CONSTRAINTS_BRANCH.
Since we are building the images using the "master" scripts, we need to
make sure the branches are retrieved from _initialization.sh of the
incoming PR, not from the one in the master branch.
Additionally versions 2.7 and 3.5 builds have to be merged to
master and excluded when the build is run targeting master branch.
TestApiKerberos::test_trigger_dag previously was dependent that the `example_bash_operator` exist in the Database.
If one of the other tests didn't write it to the DB or if one of the other tests cleared it from the DB, this test failed.
it consists of CeleryExecutor and KubernetesExecutor, which allows users
to route their tasks to either Kubernetes or Celery based on the queue
defined on a task
The cache in Github Actions is immutable - once you create it
it cannot be modified. That's why cache keys should contain
hash of all files that are used to create the cache.
Kubernetes cache key did not contain it, and as a side effect
the cache from master kubernetes setup.py was used in the v1-10-test
after the breeze changes were cherry-picked.
- Instead of supporting only an Admin user in the base test class, you can also use a normal User or Viewer
- Only add users when they are being used so we can do a little less in the setup phase (minor speedup in TestDagACLView)
Currently we run the following tests too when checking spell-check errors
- check_guide_links_in_operator_descriptions()
- check_class_links_in_operators_and_hooks_ref()
- check_guide_links_in_operators_and_hooks_ref()
- check_enforce_code_block()
- check_exampleinclude_for_example_dags()
- check_google_guides()
__lshift__ and __rshift__ methods should return other not self.
This PR fixes XComArg implementation to support chain like this one:
BaseOprator >> XComArg >> BaseOperator
Related to: #10153
If a task failed hard on celery, _before_ being able to execute the
airflow code the task would end up stuck in queued state. This change
makes it get retried.
This was discovered in load testing the HA work (but unrelated to HA
changes), where I swamped the kube-dns pod, meaning the worker was
sometimes unable to resolve the db name via DNS, so the state in the DB
was never updated
This was added when we first migrated to pytest, but pytest is now
complaining that this setting is unknown.
Since it doesn't do anything, lets remove it
If an argument is not provided to `shift` it by default uses `shift 1`. The commands "shift 1" and "shift" (with no argument) do the same thing. This PR makes the usage consistent as there were occurences of both
"airflow.providers.amazon.aws.secrets.secrets_manager." "SecretsManagerBackend.get_conn_uri"
to
"airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend.get_conn_uri"
Our migration scripts are written in the way that if you
are running them to reset the db and you have some files in
example_dags or dags, or some plugins in your plugins folder,
you will get various kinds of errors.
Those errors are printed to the user and even if they are
ultimately ignored by the SqlAlchemy migration process,
they are confusing whether the database reset
(and thus the whole initialize-virtualenv) were successful
or not.
This change disables any DAGs that could be loaded during the
DB reset (in case of running './breeze initialize-virtualenv'.
Fixes#10894
Since we are running the airflow image as airflow user, the
entrypoint and clear-logs scripts should also be set as airflow.
This had no impact if you actually run this as root user or
when your group was root (which was recommended).
Snakebite's kerberos support relied on a python-krbV
which has been removed from PyPI. It did not work
completely anyway due to snakebite not being officially
supported in python3 (snakebite-py3 did not work with
SSL which made Kerberos pretty much unusable.
This commit removes the snakebite's kerberos support
from setup.py so that you still can install kerberos
as extra for other uses.
Recent releases of FAB and Celery caused our installation to
fail. Luckily we have protection so that regular PRs are not
affected, however we need to update the setup.py to exclude
those dependencies that cause the problem.
Those are:
* vine - which is used by Celery Sensor (via kombu) - 5.0.0
version breaks celery-vine feature
* Flask-OauthLib and flask-login - combination of the current
requirements caused a conflict by forcing flask login to
be 0.5.0 which is not compatible with Flask Application Builder
Denis Evseev
John Bampton
Jonathan Stacks
Jarek Potiuk
Kaxil Naik
Sumit Maheshwari
Jacob Ferriero
Ping Zhang
Jed Cunningham
Ryan Hamilton
Daniel Imberman
Tomek Urbaszek
Dmytro Usenko
Cooper Gillan
Ash Berlin-Taylor
tszerszen
Daniel Cohen
dependabot[bot]
mucio