Tests requiring Kubernetes Cluster are now moved out of
the regular CI tests and moved to "kubernetes_tests" folder
so that they can be run entirely on host without having
the CI image built at all. They use production image
to run the tests on KinD cluster and we add tooling
to start/stop/deploy the application to the KinD cluster
automatically - for both CI testing and local development.
This is a pre-requisite to convert the tests to convert the
tests to use the official Helm Chart and Docker images or
Apache Airflow.
It closes#8782
Breeze had --push-images switch to also push images to repo
but it was often needed to build and push images separately.
We have now a possibility to push an already built image with
separate push-image command instead and also you can choose
to push to cache registry in GitHub rather than to DockerHub
with --registry-cache switch.
* Use production image for k8s tests
The CI image has become too large to load into KinD,
it also only really makes sense to use the production image for
integration tests
* nit
Co-authored-by: Daniel Imberman <daniel@astronomer.io>
* Push CI images to Docker packcage cache for v1-10 branches
This is done as a commit to master so that we can keep the two branches
in sync
Co-Authored-By: Ash Berlin-Taylor <ash_github@firemirror.com>
* Run Github Actions against v1-10-stable too
Co-authored-by: Ash Berlin-Taylor <ash_github@firemirror.com>
All PRs will used cached "latest good" version of the python
base images from our GitHub registry. The python versions in
the Github Registry will only get updated after a master
build (which pulls latest Python image from DockerHub) builds
and passes test correctly.
This is to avoid problems that we had recently with Python
patchlevel releases breaking our Docker builds.
By default github actions checks out only latest commit but in order to
see if there are any changes since the last readme generated
we need to see the whole history so we need to fetch it all.
We also skip generating the new README in case there is only one
commit in the history since the last release. The nature of readme
generation is that the commit with the README itself will never
be in the list of commits for the previous release so there is
always at least one commit more than the one listed in the readme.
Right now requirements will be only checked during the
CI build if the setup.py has changed and if yes, clear instructions
will be given. The diff will still be printed otherwise but
it will not cause the job to fail
The CRON job from previous runs did not have everything working
after the emergency migration to Github Actions.
This change brings back following improvements:
* rebuilding images from the scratch in CRON job
* automatically upgrading all requirements to test if they are new
* pushing production images to github packages as cache
* pushing nightly tag to github
* tests are not executed for doc-only changes
* images will be (once merged) downloaded from GitHub Registry so likely much faster
* we have a "scheduled" nightly build that will build everything from scratch and check if no requirements have been broken
* improved split of static checks between two static check jobs - to utilise parallelism better.
* reorganised some fast jobs (requirements, prod image) that do not depend on tests so that they can run earlier
* shorter names for jobs so that they are nicer to view in the actions view
* matrix definitions of the jobs so that we can manage them better
Trailing slash causes weird problem. When you enter the issue
via this link, your URL in browser gets updated to an empty one
which results in Error Page when you resolve the ticket.
Issue link in PR will be now automatically inserted by the boring
cyborg probot integration. It also handles the case of document-only
changes and it will keep the link updated if the title of the PR
is updated.
The periodic labeler had a bug that it was not using pagination.
In effect the labeler was checking only 30 oldest request and
if they all contained proper labels, it was not checking any more
PRs.
Details are available in
https://github.com/paulfantom/periodic-labeler/pull/4
But until it is merged, we switch to our own fixed version.
This commit adds a .github/SECURITY.md file that defines the
contents of the "Policy" tab in the new "Security" section of
the GitHub interface.
Currently the Policy tab obtains its content from the
docs/security.rst file, which contains technical, non-policy
related information. This commit retains the
"Reporting Vulnerabilities" section of docs/security.rst, which
is relevant, and strips the extraneous content.
Many operators and hooks are not listed in the
config, and therefore
not picked up by the docks generator.
Closes#3201 from Fokko/airflow-2302-add-missing-
docs
Remove example reference to AIRFLOW-[one] because
it confuses merge tools. In addition, simplify
the checkboxes because Github displays how many
of them have been checked off.
Closes#2163 from jlowin/pr-template-2
Please accept this PR that addresses the following
issues:
-
https://issues.apache.org/jira/browse/AIRFLOW-919
I also made the airflow PR template a little bit
less verbose (requires less edits when creating a
PR).
Testing Done:
- Ran a webserver with this case and made sure
that the DAG page loaded
Closes#2110 from
aoen/ddavydov/fix_running_task_with_no_start_date
For all PRs with UI changes, it is a requirement
that screenshots be included in the PR. Update
PULL_REQUEST_TEMPLATE with this PR guideline for
UI changes.
Closes#1829 from r39132/master
James Timmins
Kamil Breguła
Jarek Potiuk
Kaxil Naik
Daniel Imberman
Ash Berlin-Taylor
Jiajie Zhong
Tomek Urbaszek
Bas Harenslak
Tomek Urbaszek
Xinbin Huang
Philippe Gagnon
Felix
Tao Feng
Fokko Driesprong
r39132
Sid Anand
Taylor D. Edmiston
Fokko Driesprong
Bolke de Bruin
Jeremiah Lowin
Dan Davydov
Jeremiah Lowin
Siddharth Anand