From 2f65a1080fd82c350bdbb2d8ba3674d4789f0f16 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kamil=20Bregu=C5=82a?= Date: Mon, 21 Dec 2020 19:05:57 +0100 Subject: [PATCH] Fix typos and minor simplification in TESTING.rst (#13194) (cherry picked from commit 97eee350e41f6f00ad3076489a5dac2e5beaa266) --- TESTING.rst | 132 +++++++++++++++++++++++++++------------------------- 1 file changed, 68 insertions(+), 64 deletions(-) diff --git a/TESTING.rst b/TESTING.rst index 0420193bca..43fcb77493 100644 --- a/TESTING.rst +++ b/TESTING.rst @@ -26,12 +26,11 @@ Airflow Test Infrastructure * **Integration tests** are available in the Breeze development environment that is also used for Airflow CI tests. Integration tests are special tests that require - additional services running, such as Postgres, MySQL, Kerberos, etc. Currently, these tests are not - marked as integration tests but soon they will be separated by ``pytest`` annotations. + additional services running, such as Postgres, MySQL, Kerberos, etc. * **System tests** are automatic tests that use external systems like Google Cloud. These tests are intended for an end-to-end DAG execution. - The tests can be executed on both the current version of Apache Airflow and any of the older + The tests can be executed on both the current version of Apache Airflow and any older versions from 1.10.* series. This document is about running Python tests. Before the tests are run, use @@ -53,7 +52,7 @@ Follow the guidelines when writing unit tests: rather than ``unittest`` ones. See `Pytest docs `_ for details. * Use a parameterized framework for tests that have variations in parameters. -**NOTE:** We plan to convert all unit tests to standard "asserts" semi-automatically but this will be done later +**NOTE:** We plan to convert all unit tests to standard "asserts" semi-automatically, but this will be done later in Airflow 2.0 development phase. That will include setUp/tearDown/context managers and decorators. Running Unit Tests from IDE @@ -82,13 +81,13 @@ Running Unit Tests To run unit, integration, and system tests from the Breeze and your virtualenv, you can use the `pytest `_ framework. -Custom ``pytest`` plugin run ``airflow db init`` and ``airflow db reset`` the first +Custom ``pytest`` plugin runs ``airflow db init`` and ``airflow db reset`` the first time you launch them. So, you can count on the database being initialized. Currently, when you run tests not supported **in the local virtualenv, they may either fail or provide an error message**. There are many available options for selecting a specific test in ``pytest``. Details can be found -in the official documentation but here are a few basic examples: +in the official documentation, but here are a few basic examples: .. code-block:: bash @@ -130,7 +129,7 @@ for debugging purposes, enter: Running Tests for a Specified Target Using Breeze from the Host --------------------------------------------------------------- -If you wish to only run tests and not to drop into shell, apply the +If you wish to only run tests and not to drop into the shell, apply the ``tests`` command. You can add extra targets and pytest flags after the ``--`` command. Note that often you want to run the tests with a clean/reset db, so usually you want to add ``--db-reset`` flag to breeze. @@ -155,46 +154,46 @@ You can also specify individual tests or a group of tests: Running Tests of a specified type from the Host ----------------------------------------------- -You can also run tests for a specific test type. For the stability and performance point of view -we separated tests to different test types so that they can be run separately. +You can also run tests for a specific test type. For the stability and performance point of view, +we separated tests into different test types to be run separately. You can select the test type by adding ``--test-type TEST_TYPE`` before the test command. There are two kinds of test types: * Per-directories types are added to select subset of the tests based on sub-directories in ``tests`` folder. Example test types there - Core, Providers, CLI. The only action that happens when you choose the right - test folders are pre-selected. For those types of tests it is only useful to choose the test type + test folders are pre-selected. It is only useful for those types of tests to choose the test type when you do not specify test to run. -Runs all core tests: + Runs all core tests: -.. code-block:: bash + .. code-block:: bash - ./breeze --test-type Core --db-reset tests + ./breeze --test-type Core --db-reset tests -Runs all provider tests: + Runs all provider tests: -.. code-block:: bash + .. code-block:: bash - ./breeze --test-type Providers --db-reset tests + ./breeze --test-type Providers --db-reset tests -* Special kinds of tests - Integration, Heisentests, Quarantined, Postgres, MySQL which are marked with pytest +* Special kinds of tests - Integration, Heisentests, Quarantined, Postgres, MySQL, which are marked with pytest marks and for those you need to select the type using test-type switch. If you want to run such tests using breeze, you need to pass appropriate ``--test-type`` otherwise the test will be skipped. Similarly to the per-directory tests if you do not specify the test or tests to run, all tests of a given type are run -Run quarantined test_task_command.py test: + Run quarantined test_task_command.py test: -.. code-block:: bash + .. code-block:: bash - ./breeze --test-type Quarantined tests tests/cli/commands/test_task_command.py --db-reset + ./breeze --test-type Quarantined tests tests/cli/commands/test_task_command.py --db-reset -Run all Quarantined tests: + Run all Quarantined tests: -.. code-block:: bash + .. code-block:: bash - ./breeze --test-type Quarantined tests --db-reset + ./breeze --test-type Quarantined tests --db-reset Helm Unit Tests =============== @@ -209,8 +208,8 @@ go to the ``chart/tests`` directory and add your unit test by creating a class t To render the chart create a YAML string with the nested dictionary of options you wish to test. You can then use our ``render_chart`` function to render the object of interest into a testable Python dictionary. Once the chart -has been rendered, you can use the ``render_k8s_object`` function to create a k8s model object that simultaneously -ensures that the object created properly conforms to the expected object spec and allows you to use object values +has been rendered, you can use the ``render_k8s_object`` function to create a k8s model object. It simultaneously +ensures that the object created properly conforms to the expected resource spec and allows you to use object values instead of nested dictionaries. Example test here: @@ -244,8 +243,8 @@ To run tests using breeze run the following command Airflow Integration Tests ========================= -Some of the tests in Airflow are integration tests. These tests require not only ``airflow`` Docker -image but also extra images with integrations (such as ``redis``, ``mongodb``, etc.). +Some of the tests in Airflow are integration tests. These tests require ``airflow`` Docker +image and extra images with integrations (such as ``redis``, ``mongodb``, etc.). Enabling Integrations @@ -254,13 +253,13 @@ Enabling Integrations Airflow integration tests cannot be run in the local virtualenv. They can only run in the Breeze environment with enabled integrations and in the CI. See ``_ for details about Airflow CI. -When you are in the Breeze environment, by default all integrations are disabled. This enables only true unit tests +When you are in the Breeze environment, by default, all integrations are disabled. This enables only true unit tests to be executed in Breeze. You can enable the integration by passing the ``--integration `` switch when starting Breeze. You can specify multiple integrations by repeating the ``--integration`` switch -or by using the ``--integration all`` switch that enables all integrations. +or using the ``--integration all`` switch that enables all integrations. NOTE: Every integration requires a separate container with the corresponding integration image. -They take precious resources on your PC, mainly the memory. The started integrations are not stopped +These containers take precious resources on your PC, mainly the memory. The started integrations are not stopped until you stop the Breeze environment with the ``stop`` command and restart it via ``restart`` command. @@ -280,6 +279,8 @@ The following integrations are available: - Integration required for MongoDB hooks * - openldap - Integration required for OpenLDAP hooks + * - presto + - Integration required for Presto hooks * - rabbitmq - Integration required for Celery executor tests * - redis @@ -397,7 +398,7 @@ Here is an example of running only postgres-specific backend tests: pytest --backend postgres -Running Long running tests +Running Long-running tests -------------------------- Some of the tests rung for a long time. Such tests are marked with ``@pytest.mark.long_running`` annotation. @@ -437,7 +438,7 @@ packages for them and the main "apache-airflow" package does not contain the pro Most of the development in Breeze happens by iterating on sources and when you run your tests during development, you usually do not want to build packages and install them separately. -Therefore by default when you enter Breeze airflow and all providers are available directly from +Therefore by default, when you enter Breeze airflow and all providers are available directly from sources rather than installed from packages. This is for example to test the "provider discovery" mechanism available that reads provider information from the package meta-data. @@ -445,7 +446,7 @@ When Airflow is run from sources, the metadata is read from provider.yaml files, but when Airflow is installed from packages, it is read via the package entrypoint ``apache_airflow_provider``. -By default all packages are prepared in wheel format. In order to install Airflow from packages you +By default, all packages are prepared in wheel format. To install Airflow from packages you need to run the following steps: 1. Prepare provider packages @@ -454,10 +455,10 @@ need to run the following steps: ./breeze prepare-provider-packages [PACKAGE ...] -If you run this command without packages, you will prepare all packages, you can however specify +If you run this command without packages, you will prepare all packages. However, You can specify providers that you would like to build if you just want to build few provider packages. -The packages are prepared in ``dist`` folder. Note, that this command cleans up the ``dist`` folder -before running so you should run it before generating airflow package. +The packages are prepared in ``dist`` folder. Note that this command cleans up the ``dist`` folder +before running, so you should run it before generating ``apache-airflow`` package. 2. Prepare airflow packages @@ -480,18 +481,18 @@ This installs airflow and enters Running Tests with Kubernetes ============================= -Airflow has tests that are run against real kubernetes cluster. We are using +Airflow has tests that are run against real Kubernetes cluster. We are using `Kind `_ to create and run the cluster. We integrated the tools to start/stop/ deploy and run the cluster tests in our repository and into Breeze development environment. -Configuration for the cluster is kept in ``./build/.kube/config`` file in your Airflow source repository and +Configuration for the cluster is kept in ``./build/.kube/config`` file in your Airflow source repository, and our scripts set the ``KUBECONFIG`` variable to it. If you want to interact with the Kind cluster created you can do it from outside of the scripts by exporting this variable and point it to this file. Starting Kubernetes Cluster --------------------------- -For your testing you manage Kind cluster with ``kind-cluster`` breeze command: +For your testing, you manage Kind cluster with ``kind-cluster`` breeze command: .. code-block:: bash @@ -500,7 +501,7 @@ For your testing you manage Kind cluster with ``kind-cluster`` breeze command: The command allows you to start/stop/recreate/status Kind Kubernetes cluster, deploy Airflow via Helm chart as well as interact with the cluster (via test and shell commands). -Setting up the Kind Kubernetes cluster takes some time so once you started it, the cluster continues running +Setting up the Kind Kubernetes cluster takes some time, so once you started it, the cluster continues running until it is stopped with the ``kind-cluster stop`` command or until ``kind-cluster recreate`` command is used (it will stop and recreate the cluster image). @@ -518,7 +519,7 @@ Deploying Airflow to the Kubernetes cluster created is also done via ``kind-clus ./breeze kind-cluster deploy -The deploy commands performs those steps: +The deploy command performs those steps: 1. It rebuilds the latest ``apache/airflow:master-pythonX.Y`` production images using the latest sources using local caching. It also adds example DAGs to the image, so that they do not @@ -535,7 +536,7 @@ Running tests with Kubernetes Cluster You can either run all tests or you can select which tests to run. You can also enter interactive virtualenv to run the tests manually one by one. -Running kubernetes tests via shell: +Running Kubernetes tests via shell: .. code-block:: bash @@ -543,7 +544,7 @@ Running kubernetes tests via shell: ./scripts/ci/kubernetes/ci_run_kubernetes_tests.sh TEST [TEST ...] - runs selected kubernetes tests (from kubernetes_tests folder) -Running kubernetes tests via breeze: +Running Kubernetes tests via breeze: .. code-block:: bash @@ -554,7 +555,7 @@ Running kubernetes tests via breeze: Entering shell with Kubernetes Cluster -------------------------------------- -This shell is prepared to run kubernetes tests interactively. It has ``kubectl`` and ``kind`` cli tools +This shell is prepared to run Kubernetes tests interactively. It has ``kubectl`` and ``kind`` cli tools available in the path, it has also activated virtualenv environment that allows you to run tests via pytest. You can enter the shell via those scripts @@ -568,13 +569,16 @@ You can enter the shell via those scripts ./breeze kind-cluster shell -K9s CLI - debug kubernetes in style! +K9s CLI - debug Kubernetes in style! ------------------------------------ -Breeze has built-in integration with fantastic k9s CLI tool, that allows you to debug the kubernetes -installation effortlessly and in style. K9S provides terminal (but windowed) CLI that allows you to -easily observe what's going on in the kubernetes instance, observe the resources defined (pods, secrets, -custom resource definitions), enter shell for the Pods/Containers running, see the log files and more. +Breeze has built-in integration with fantastic k9s CLI tool, that allows you to debug the Kubernetes +installation effortlessly and in style. K9S provides terminal (but windowed) CLI that helps you to: + +- easily observe what's going on in the Kubernetes cluster +- observe the resources defined (pods, secrets, custom resource definitions) +- enter shell for the Pods/Containers running, +- see the log files and more. You can read more about k9s at `https://k9scli.io/ `_ @@ -691,7 +695,7 @@ This prepares and enters the virtualenv in ``.build/.kubernetes_venv_``. For -example the below command will build google postgres and mysql wheel packages: +example, the below command will build google postgres and mysql wheel packages: .. code-block:: bash @@ -957,7 +961,7 @@ when you enter Breeze, so it is easy to automate installing those packages for t Installing backported for Airflow 1.10.* series ----------------------------------------------- -The tests can be executed against the master version of Airflow but they also work +The tests can be executed against the master version of Airflow, but they also work with older versions. This is especially useful to test back-ported operators from Airflow 2.0 to 1.10.* versions. @@ -1042,7 +1046,7 @@ When you want to iterate on system tests, you might want to create slow resource If you need to set up some external resources for your tests (for example compute instances in Google Cloud) you should set them up and teardown in the setUp/tearDown methods of your tests. -Since those resources might be slow to create you might want to add some helpers that +Since those resources might be slow to create, you might want to add some helpers that set them up and tear them down separately via manual operations. This way you can iterate on the tests without waiting for setUp and tearDown with every test. @@ -1072,7 +1076,7 @@ Breeze session. They are usually expensive to run. Note that in case you have to update your backported operators or system tests (they are part of the provider packageS) you need to rebuild the packages outside of breeze and ``pip remove/pip install`` those packages to get them installed. This is not needed -if you run system tests with ``current`` airflow version, so it is better to iterate with the +if you run system tests with ``current`` Airflow version, so it is better to iterate with the system tests with the ``current`` version and fix all problems there and only afterwards run the tests with Airflow 1.10.*