Originally Breeze was used to run unit and integration tests, recently system
tests and finally we make it a bit more friendly to test your DAGs there. You
can now install any older airflow version in Breeze via
--install-airflow-version switch and "files/dags" folder is mounted to
"/files/dags" and this folder is used to read the dags from.
This change introduces sub-commands in breeze tool.
It is much needed as we have many commands now
and it was difficult to separate commands from flags.
Also --help output was very long and unreadable.
With this change help it is much easier to discover
what breeze can do for you as well as navigate with it.
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Co-authored-by: Kamil Breguła <mik-laj@users.noreply.github.com>
We will run system test on back-ported operators for 1.10* series of airflow
and for that we need to have support for running system tests using pytest's
markers and reading environment variables passed from HOST machine (to pass
credentials).
This is the first step to automate system tests execution.
* Add DAG return type
* Add airflow dag test CLI command for running a complete DAG with CLI
* Added new command note to updating.md
* Add airflow dags test example to testing.rst
* Fix rst inline code mistake
You can now choose which integration you want to start when you run
Breeze as well as when CI tests are run. Now by default Breeze
and CI runs without integrations, but you can add them via
Breeze flags or by environment variables when CI is executed.
We have pytest markers now that mark tests that can be run for
integration, backend and runtime selected.
Also we have now more test jobs - we have separate test run
for all non-integration tests (with less memory used by the
integrations) and separate jobs that run integration tests
only (more memory used for integrations but far less number
of tests to run)
We have far too much bash code around that is not automatically tested.
This is the first step to change it (simplifications and more tests are coming
soon).
Adds new executor that is meant to be used mainly
for debugging and DAG development purposes. This
executor executes single task instance at time and
is able to work with SQLLite and sensors.
Xinbin Huang
Jarek Potiuk
yajna pandith
Kamil Breguła
Matt Buell
Bas Harenslak
Tomek
Philippe Gagnon