github
/
incubator-airflow
зеркало из https://github.com/github/incubator-airflow.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Fix typos and minor simplification in TESTING.rst (#13194)

This commit is contained in:
Kamil Breguła 2020-12-21 19:05:57 +01:00 коммит произвёл GitHub
Родитель 925153f5a5
Коммит 97eee350e4
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
1 изменённых файлов: 68 добавлений и 64 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

132
TESTING.rst
Просмотреть файл

@ -26,12 +26,11 @@ Airflow Test Infrastructure
* **Integration tests** are available in the Breeze development environment
that is also used for Airflow CI tests. Integration tests are special tests that require
additional services running, such as Postgres, MySQL, Kerberos, etc. Currently, these tests are not
marked as integration tests but soon they will be separated by ``pytest`` annotations.
additional services running, such as Postgres, MySQL, Kerberos, etc.
* **System tests** are automatic tests that use external systems like
Google Cloud. These tests are intended for an end-to-end DAG execution.
The tests can be executed on both the current version of Apache Airflow and any of the older
The tests can be executed on both the current version of Apache Airflow and any older
versions from 1.10.* series.
This document is about running Python tests. Before the tests are run, use
@ -53,7 +52,7 @@ Follow the guidelines when writing unit tests:
rather than ``unittest`` ones. See `Pytest docs <http://doc.pytest.org/en/latest/assert.html>`_ for details.
* Use a parameterized framework for tests that have variations in parameters.
**NOTE:** We plan to convert all unit tests to standard "asserts" semi-automatically but this will be done later
**NOTE:** We plan to convert all unit tests to standard "asserts" semi-automatically, but this will be done later
in Airflow 2.0 development phase. That will include setUp/tearDown/context managers and decorators.
Running Unit Tests from IDE
@ -82,13 +81,13 @@ Running Unit Tests
To run unit, integration, and system tests from the Breeze and your
virtualenv, you can use the `pytest <http://doc.pytest.org/en/latest/>`_ framework.
Custom ``pytest`` plugin run ``airflow db init`` and ``airflow db reset`` the first
Custom ``pytest`` plugin runs ``airflow db init`` and ``airflow db reset`` the first
time you launch them. So, you can count on the database being initialized. Currently,
when you run tests not supported **in the local virtualenv, they may either fail
or provide an error message**.
There are many available options for selecting a specific test in ``pytest``. Details can be found
in the official documentation but here are a few basic examples:
in the official documentation, but here are a few basic examples:
.. code-block:: bash
@ -130,7 +129,7 @@ for debugging purposes, enter:
Running Tests for a Specified Target Using Breeze from the Host
---------------------------------------------------------------
If you wish to only run tests and not to drop into shell, apply the
If you wish to only run tests and not to drop into the shell, apply the
``tests`` command. You can add extra targets and pytest flags after the ``--`` command. Note that
often you want to run the tests with a clean/reset db, so usually you want to add ``--db-reset`` flag
to breeze.
@ -155,46 +154,46 @@ You can also specify individual tests or a group of tests:
Running Tests of a specified type from the Host
-----------------------------------------------
You can also run tests for a specific test type. For the stability and performance point of view
we separated tests to different test types so that they can be run separately.
You can also run tests for a specific test type. For the stability and performance point of view,
we separated tests into different test types to be run separately.
You can select the test type by adding ``--test-type TEST_TYPE`` before the test command. There are two
kinds of test types:
* Per-directories types are added to select subset of the tests based on sub-directories in ``tests`` folder.
Example test types there - Core, Providers, CLI. The only action that happens when you choose the right
test folders are pre-selected. For those types of tests it is only useful to choose the test type
test folders are pre-selected. It is only useful for those types of tests to choose the test type
when you do not specify test to run.
Runs all core tests:
Runs all core tests:
.. code-block:: bash
.. code-block:: bash
./breeze --test-type Core --db-reset tests
./breeze --test-type Core --db-reset tests
Runs all provider tests:
Runs all provider tests:
.. code-block:: bash
.. code-block:: bash
./breeze --test-type Providers --db-reset tests
./breeze --test-type Providers --db-reset tests
* Special kinds of tests - Integration, Heisentests, Quarantined, Postgres, MySQL which are marked with pytest
* Special kinds of tests - Integration, Heisentests, Quarantined, Postgres, MySQL, which are marked with pytest
marks and for those you need to select the type using test-type switch. If you want to run such tests
using breeze, you need to pass appropriate ``--test-type`` otherwise the test will be skipped.
Similarly to the per-directory tests if you do not specify the test or tests to run,
all tests of a given type are run
Run quarantined test_task_command.py test:
Run quarantined test_task_command.py test:
.. code-block:: bash
.. code-block:: bash
./breeze --test-type Quarantined tests tests/cli/commands/test_task_command.py --db-reset
./breeze --test-type Quarantined tests tests/cli/commands/test_task_command.py --db-reset
Run all Quarantined tests:
Run all Quarantined tests:
.. code-block:: bash
.. code-block:: bash
./breeze --test-type Quarantined tests --db-reset
./breeze --test-type Quarantined tests --db-reset
Helm Unit Tests
===============
@ -209,8 +208,8 @@ go to the ``chart/tests`` directory and add your unit test by creating a class t
To render the chart create a YAML string with the nested dictionary of options you wish to test. You can then
use our ``render_chart`` function to render the object of interest into a testable Python dictionary. Once the chart
has been rendered, you can use the ``render_k8s_object`` function to create a k8s model object that simultaneously
ensures that the object created properly conforms to the expected object spec and allows you to use object values
has been rendered, you can use the ``render_k8s_object`` function to create a k8s model object. It simultaneously
ensures that the object created properly conforms to the expected resource spec and allows you to use object values
instead of nested dictionaries.
Example test here:
@ -244,8 +243,8 @@ To run tests using breeze run the following command
Airflow Integration Tests
=========================
Some of the tests in Airflow are integration tests. These tests require not only ``airflow`` Docker
image but also extra images with integrations (such as ``redis``, ``mongodb``, etc.).
Some of the tests in Airflow are integration tests. These tests require ``airflow`` Docker
image and extra images with integrations (such as ``redis``, ``mongodb``, etc.).
Enabling Integrations
@ -254,13 +253,13 @@ Enabling Integrations
Airflow integration tests cannot be run in the local virtualenv. They can only run in the Breeze
environment with enabled integrations and in the CI. See `<CI.yml>`_ for details about Airflow CI.
When you are in the Breeze environment, by default all integrations are disabled. This enables only true unit tests
When you are in the Breeze environment, by default, all integrations are disabled. This enables only true unit tests
to be executed in Breeze. You can enable the integration by passing the ``--integration <INTEGRATION>``
switch when starting Breeze. You can specify multiple integrations by repeating the ``--integration`` switch
or by using the ``--integration all`` switch that enables all integrations.
or using the ``--integration all`` switch that enables all integrations.
NOTE: Every integration requires a separate container with the corresponding integration image.
They take precious resources on your PC, mainly the memory. The started integrations are not stopped
These containers take precious resources on your PC, mainly the memory. The started integrations are not stopped
until you stop the Breeze environment with the ``stop`` command and restart it
via ``restart`` command.
@ -280,6 +279,8 @@ The following integrations are available:
- Integration required for MongoDB hooks
* - openldap
- Integration required for OpenLDAP hooks
* - presto
- Integration required for Presto hooks
* - rabbitmq
- Integration required for Celery executor tests
* - redis
@ -397,7 +398,7 @@ Here is an example of running only postgres-specific backend tests:
pytest --backend postgres
Running Long running tests
Running Long-running tests
--------------------------
Some of the tests rung for a long time. Such tests are marked with ``@pytest.mark.long_running`` annotation.
@ -437,7 +438,7 @@ packages for them and the main "apache-airflow" package does not contain the pro
Most of the development in Breeze happens by iterating on sources and when you run
your tests during development, you usually do not want to build packages and install them separately.
Therefore by default when you enter Breeze airflow and all providers are available directly from
Therefore by default, when you enter Breeze airflow and all providers are available directly from
sources rather than installed from packages. This is for example to test the "provider discovery"
mechanism available that reads provider information from the package meta-data.
@ -445,7 +446,7 @@ When Airflow is run from sources, the metadata is read from provider.yaml
files, but when Airflow is installed from packages, it is read via the package entrypoint
``apache_airflow_provider``.
By default all packages are prepared in wheel format. In order to install Airflow from packages you
By default, all packages are prepared in wheel format. To install Airflow from packages you
need to run the following steps:
1. Prepare provider packages
@ -454,10 +455,10 @@ need to run the following steps:
./breeze prepare-provider-packages [PACKAGE ...]
If you run this command without packages, you will prepare all packages, you can however specify
If you run this command without packages, you will prepare all packages. However, You can specify
providers that you would like to build if you just want to build few provider packages.
The packages are prepared in ``dist`` folder. Note, that this command cleans up the ``dist`` folder
before running so you should run it before generating airflow package.
The packages are prepared in ``dist`` folder. Note that this command cleans up the ``dist`` folder
before running, so you should run it before generating ``apache-airflow`` package.
2. Prepare airflow packages
@ -480,18 +481,18 @@ This installs airflow and enters
Running Tests with Kubernetes
=============================
Airflow has tests that are run against real kubernetes cluster. We are using
Airflow has tests that are run against real Kubernetes cluster. We are using
`Kind <https://kind.sigs.k8s.io/>`_ to create and run the cluster. We integrated the tools to start/stop/
deploy and run the cluster tests in our repository and into Breeze development environment.
Configuration for the cluster is kept in ``./build/.kube/config`` file in your Airflow source repository and
Configuration for the cluster is kept in ``./build/.kube/config`` file in your Airflow source repository, and
our scripts set the ``KUBECONFIG`` variable to it. If you want to interact with the Kind cluster created
you can do it from outside of the scripts by exporting this variable and point it to this file.
Starting Kubernetes Cluster
---------------------------
For your testing you manage Kind cluster with ``kind-cluster`` breeze command:
For your testing, you manage Kind cluster with ``kind-cluster`` breeze command:
.. code-block:: bash
@ -500,7 +501,7 @@ For your testing you manage Kind cluster with ``kind-cluster`` breeze command:
The command allows you to start/stop/recreate/status Kind Kubernetes cluster, deploy Airflow via Helm
chart as well as interact with the cluster (via test and shell commands).
Setting up the Kind Kubernetes cluster takes some time so once you started it, the cluster continues running
Setting up the Kind Kubernetes cluster takes some time, so once you started it, the cluster continues running
until it is stopped with the ``kind-cluster stop`` command or until ``kind-cluster recreate``
command is used (it will stop and recreate the cluster image).
@ -518,7 +519,7 @@ Deploying Airflow to the Kubernetes cluster created is also done via ``kind-clus
./breeze kind-cluster deploy
The deploy commands performs those steps:
The deploy command performs those steps:
1. It rebuilds the latest ``apache/airflow:master-pythonX.Y`` production images using the
latest sources using local caching. It also adds example DAGs to the image, so that they do not
@ -535,7 +536,7 @@ Running tests with Kubernetes Cluster
You can either run all tests or you can select which tests to run. You can also enter interactive virtualenv
to run the tests manually one by one.
Running kubernetes tests via shell:
Running Kubernetes tests via shell:
.. code-block:: bash
@ -543,7 +544,7 @@ Running kubernetes tests via shell:
./scripts/ci/kubernetes/ci_run_kubernetes_tests.sh TEST [TEST ...] - runs selected kubernetes tests (from kubernetes_tests folder)
Running kubernetes tests via breeze:
Running Kubernetes tests via breeze:
.. code-block:: bash
@ -554,7 +555,7 @@ Running kubernetes tests via breeze:
Entering shell with Kubernetes Cluster
--------------------------------------
This shell is prepared to run kubernetes tests interactively. It has ``kubectl`` and ``kind`` cli tools
This shell is prepared to run Kubernetes tests interactively. It has ``kubectl`` and ``kind`` cli tools
available in the path, it has also activated virtualenv environment that allows you to run tests via pytest.
You can enter the shell via those scripts
@ -568,13 +569,16 @@ You can enter the shell via those scripts
./breeze kind-cluster shell
K9s CLI - debug kubernetes in style!
K9s CLI - debug Kubernetes in style!
------------------------------------
Breeze has built-in integration with fantastic k9s CLI tool, that allows you to debug the kubernetes
installation effortlessly and in style. K9S provides terminal (but windowed) CLI that allows you to
easily observe what's going on in the kubernetes instance, observe the resources defined (pods, secrets,
custom resource definitions), enter shell for the Pods/Containers running, see the log files and more.
Breeze has built-in integration with fantastic k9s CLI tool, that allows you to debug the Kubernetes
installation effortlessly and in style. K9S provides terminal (but windowed) CLI that helps you to:
- easily observe what's going on in the Kubernetes cluster
- observe the resources defined (pods, secrets, custom resource definitions)
- enter shell for the Pods/Containers running,
- see the log files and more.
You can read more about k9s at `https://k9scli.io/ <https://k9scli.io/>`_
@ -691,7 +695,7 @@ This prepares and enters the virtualenv in ``.build/.kubernetes_venv_<YOUR_CURRE
./breeze kind-cluster shell
Once you enter the environment you receive this information:
Once you enter the environment, you receive this information:
.. code-block:: bash
@ -719,7 +723,7 @@ Use it to observe what's going on in your cluster.
It is very easy to running/debug Kubernetes tests with IntelliJ/PyCharm. Unlike the regular tests they are
in ``kubernetes_tests`` folder and if you followed the previous steps and entered the shell using
``./breeze kind-cluster shell`` command, you can setup your IDE very easily to run (and debug) your
``./breeze kind-cluster shell`` command, you can setup your IDE very easy to run (and debug) your
tests using the standard IntelliJ Run/Debug feature. You just need a few steps:
a) Add the virtualenv as interpreter for the project:
@ -762,10 +766,10 @@ environment variable copying it from the result of "./breeze kind-cluster test":
:alt: Run/Debug tests
The configuration for kubernetes is stored in your "Airflow" source directory in ".build/.kube/config" file
The configuration for Kubernetes is stored in your "Airflow" source directory in ".build/.kube/config" file
and this is where KUBECONFIG env should point to.
You can iterate with tests while you are in the virtualenv. All the tests requiring kubernetes cluster
You can iterate with tests while you are in the virtualenv. All the tests requiring Kubernetes cluster
are in "kubernetes_tests" folder. You can add extra ``pytest`` parameters then (for example ``-s`` will
print output generated test logs and print statements to the terminal immediately.
@ -775,7 +779,7 @@ print output generated test logs and print statements to the terminal immediatel
You can modify the tests or KubernetesPodOperator and re-run them without re-deploying
airflow to KinD cluster.
Airflow to KinD cluster.
Sometimes there are side effects from running tests. You can run ``redeploy_airflow.sh`` without
@ -801,7 +805,7 @@ Kind has also useful commands to inspect your running cluster:
kind --help
However, when you change Airflow Kubernetes executor implementation you need to redeploy
However, when you change Kubernetes executor implementation, you need to redeploy
Airflow to the cluster.
.. code-block:: bash
@ -893,10 +897,10 @@ credentials stored in your ``home`` directory. Use this feature with care as it
visible to anything that you have installed inside the Docker container.
Currently forwarded credentials are:
* credentials stored in ``${HOME}/.aws`` for the aws Amazon Web Services client
* credentials stored in ``${HOME}/.azure`` for the az Microsoft Azure client
* credentials stored in ``${HOME}/.config`` for gcloud Google Cloud client (among others)
* credentials stored in ``${HOME}/.docker`` for docker client
* credentials stored in ``${HOME}/.aws`` for ``aws`` - Amazon Web Services client
* credentials stored in ``${HOME}/.azure`` for ``az`` - Microsoft Azure client
* credentials stored in ``${HOME}/.config`` for ``gcloud`` - Google Cloud client (among others)
* credentials stored in ``${HOME}/.docker`` for ``docker`` client
Adding a New System Test
--------------------------
@ -942,9 +946,9 @@ It runs two DAGs defined in ``airflow.providers.google.cloud.example_dags.exampl
Preparing provider packages for System Tests for Airflow 1.10.* series
----------------------------------------------------------------------
To run system tests with old Airflow version you need to prepare provider packages. This
To run system tests with the older Airflow version, you need to prepare provider packages. This
can be done by running ``./breeze prepare-provider-packages <PACKAGES TO BUILD>``. For
example the below command will build google postgres and mysql wheel packages:
example, the below command will build google postgres and mysql wheel packages:
.. code-block:: bash
@ -957,7 +961,7 @@ when you enter Breeze, so it is easy to automate installing those packages for t
Installing backported for Airflow 1.10.* series
-----------------------------------------------
The tests can be executed against the master version of Airflow but they also work
The tests can be executed against the master version of Airflow, but they also work
with older versions. This is especially useful to test back-ported operators
from Airflow 2.0 to 1.10.* versions.
@ -1042,7 +1046,7 @@ When you want to iterate on system tests, you might want to create slow resource
If you need to set up some external resources for your tests (for example compute instances in Google Cloud)
you should set them up and teardown in the setUp/tearDown methods of your tests.
Since those resources might be slow to create you might want to add some helpers that
Since those resources might be slow to create, you might want to add some helpers that
set them up and tear them down separately via manual operations. This way you can iterate on
the tests without waiting for setUp and tearDown with every test.
@ -1072,7 +1076,7 @@ Breeze session. They are usually expensive to run.
Note that in case you have to update your backported operators or system tests (they are part of
the provider packageS) you need to rebuild the packages outside of breeze and
``pip remove/pip install`` those packages to get them installed. This is not needed
if you run system tests with ``current`` airflow version, so it is better to iterate with the
if you run system tests with ``current`` Airflow version, so it is better to iterate with the
system tests with the ``current`` version and fix all problems there and only afterwards run
the tests with Airflow 1.10.*