From a690b16850c10921e14128a20727cc6d2a26459f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kamil=20Bregu=C5=82a?= Date: Sat, 4 Jan 2020 21:46:21 +0100 Subject: [PATCH] [AIRFLOW-XXXX] Add more cross-reference (#7045) --- docs/cli-ref.rst | 3 +++ docs/configurations-ref.rst | 3 +++ docs/executor/dask.rst | 3 ++- docs/executor/debug.rst | 1 + docs/executor/index.rst | 3 +++ docs/howto/connection/index.rst | 2 ++ docs/howto/operator/index.rst | 7 ++++--- docs/howto/set-config.rst | 6 ++++-- docs/howto/tracking-user-activity.rst | 3 +++ docs/howto/write-logs.rst | 3 +++ docs/lineage.rst | 3 +++ docs/timezone.rst | 2 ++ docs/tutorial.rst | 6 +++--- docs/usage-cli.rst | 7 +++++++ 14 files changed, 43 insertions(+), 9 deletions(-) diff --git a/docs/cli-ref.rst b/docs/cli-ref.rst index 1496763043..cef048862c 100644 --- a/docs/cli-ref.rst +++ b/docs/cli-ref.rst @@ -24,6 +24,9 @@ Airflow has a very rich command line interface that allows for many types of operation on a DAG, starting services, and supporting development and testing. +.. note:: + For more information on usage CLI, see :doc:`usage-cli` + .. contents:: Content :local: :depth: 2 diff --git a/docs/configurations-ref.rst b/docs/configurations-ref.rst index 0803739daa..01c49a5c28 100644 --- a/docs/configurations-ref.rst +++ b/docs/configurations-ref.rst @@ -22,6 +22,9 @@ Configuration Reference This page contains the list of all the available Airflow configurations that you can set in ``airflow.cfg`` file or using environment variables. +.. note:: + For more information on setting the configuration, see :doc:`howto/set-config` + .. contents:: Sections: :local: :depth: 1 diff --git a/docs/executor/dask.rst b/docs/executor/dask.rst index 0823bbec5d..998f686606 100644 --- a/docs/executor/dask.rst +++ b/docs/executor/dask.rst @@ -42,7 +42,8 @@ Next start at least one Worker on any machine that can connect to the host: dask-worker $DASK_HOST:$DASK_PORT Edit your ``airflow.cfg`` to set your executor to :class:`airflow.executors.dask_executor.DaskExecutor` and provide -the Dask Scheduler address in the ``[dask]`` section. +the Dask Scheduler address in the ``[dask]`` section. For more information on setting the configuration, +see :doc:`../howto/set-config`. Please note: diff --git a/docs/executor/debug.rst b/docs/executor/debug.rst index ca4029be2b..0067a902c2 100644 --- a/docs/executor/debug.rst +++ b/docs/executor/debug.rst @@ -30,6 +30,7 @@ blocking the execution of DAG. Additionally ``DebugExecutor`` can be used in a fail-fast mode that will make all other running or scheduled tasks fail immediately. To enable this option set ``AIRFLOW__DEBUG__FAIL_FAST=True`` or adjust ``fail_fast`` option in your ``airflow.cfg``. +For more information on setting the configuration, see :doc:`../howto/set-config`. **IDE setup steps:** diff --git a/docs/executor/index.rst b/docs/executor/index.rst index d3e1d78977..f38dee9672 100644 --- a/docs/executor/index.rst +++ b/docs/executor/index.rst @@ -23,6 +23,9 @@ Executors are the mechanism by which task instances get run. Airflow has support for various executors. Current used is determined by the ``executor`` option in the ``core`` section of the configuration file. +.. note:: + For more information on setting the configuration, see :doc:`../howto/set-config`. + .. toctree:: :maxdepth: 1 :glob: diff --git a/docs/howto/connection/index.rst b/docs/howto/connection/index.rst index 6f1e090a9f..95548980d3 100644 --- a/docs/howto/connection/index.rst +++ b/docs/howto/connection/index.rst @@ -93,6 +93,8 @@ If using with a docker ``.env`` file, you may need to remove the single quotes. AIRFLOW_CONN_MY_PROD_DATABASE=my-conn-type://login:password@host:port/schema?param1=val1¶m2=val2 +.. _connection/cli: + Creating a Connection from the CLI ---------------------------------- diff --git a/docs/howto/operator/index.rst b/docs/howto/operator/index.rst index c2355d75e5..2b8923dcd8 100644 --- a/docs/howto/operator/index.rst +++ b/docs/howto/operator/index.rst @@ -23,9 +23,10 @@ Using Operators An operator represents a single, ideally idempotent, task. Operators determine what actually executes when your DAG runs. -See the :ref:`Operators Concepts ` documentation and the -:doc:`Operators API Reference <../../_api/index>` for more -information. +.. note:: + See the :ref:`Operators Concepts ` documentation and the + :doc:`Operators API Reference <../../_api/index>` for more + information. .. toctree:: :maxdepth: 2 diff --git a/docs/howto/set-config.rst b/docs/howto/set-config.rst index 035bc2959e..d1d30ec7a4 100644 --- a/docs/howto/set-config.rst +++ b/docs/howto/set-config.rst @@ -25,8 +25,7 @@ your ``$AIRFLOW_HOME`` directory (``~/airflow`` by default). This file contains can edit it to change any of the settings. You can also set options with environment variables by using this format: ``$AIRFLOW__{SECTION}__{KEY}`` (note the double underscores). -For example, the -metadata database connection string can either be set in ``airflow.cfg`` like this: +For example, the metadata database connection string can either be set in ``airflow.cfg`` like this: .. code-block:: ini @@ -75,3 +74,6 @@ The universal order of precedence for all configuration options is as follows: #. set in ``airflow.cfg`` #. command in ``airflow.cfg`` #. Airflow's built in defaults + +.. note:: + For more information on configuration options, see :doc:`../configurations-ref` diff --git a/docs/howto/tracking-user-activity.rst b/docs/howto/tracking-user-activity.rst index 23719efea2..aa35ae2b7a 100644 --- a/docs/howto/tracking-user-activity.rst +++ b/docs/howto/tracking-user-activity.rst @@ -34,3 +34,6 @@ Edit ``airflow.cfg`` and set the ``webserver`` block to have an ``analytics_tool .. note:: You can see view injected tracker html within Airflow's source code at ``airflow/www/templates/appbuilder/baselayout.html``. The related global variables are set in ``airflow/www/templates/app.py``. + +.. note:: + For more information on setting the configuration, see :doc:`../howto/set-config` diff --git a/docs/howto/write-logs.rst b/docs/howto/write-logs.rst index 9dcbfa4035..b772bacbc1 100644 --- a/docs/howto/write-logs.rst +++ b/docs/howto/write-logs.rst @@ -27,6 +27,9 @@ Users can specify the directory to place log files in ``airflow.cfg`` using ``base_log_folder``. By default, logs are placed in the ``AIRFLOW_HOME`` directory. +.. note:: + For more information on setting the configuration, see :doc:`set-config` + The following convention is followed while naming logs: ``{dag_id}/{task_id}/{execution_date}/{try_number}.log`` In addition, users can supply a remote location to store current logs and backups. diff --git a/docs/lineage.rst b/docs/lineage.rst index 1b452fcda5..6313628767 100644 --- a/docs/lineage.rst +++ b/docs/lineage.rst @@ -116,3 +116,6 @@ properly, e.g. in your ``airflow.cfg``: Please make sure to have the ``atlasclient`` package installed. + +.. note:: + For more information on setting the configuration, see :doc:`howto/set-config` diff --git a/docs/timezone.rst b/docs/timezone.rst index 8e17607d5b..41425ec80b 100644 --- a/docs/timezone.rst +++ b/docs/timezone.rst @@ -107,6 +107,8 @@ it is therefore important to make sure this setting is equal on all Airflow node [core] default_timezone = utc +.. note:: + For more information on setting the configuration, see :doc:`howto/set-config` Time zone aware DAGs -------------------- diff --git a/docs/tutorial.rst b/docs/tutorial.rst index 1998b025de..4e310311c1 100644 --- a/docs/tutorial.rst +++ b/docs/tutorial.rst @@ -338,8 +338,8 @@ Here's a few things you might want to do next: * Take an in-depth tour of the UI - click all the things! * Keep reading the docs! Especially the sections on: - * Command line interface - * Operators - * Macros + * :doc:`usage-cli` + * :doc:`howto/custom-operator` + * :doc:`macros-ref` * Write your first pipeline! diff --git a/docs/usage-cli.rst b/docs/usage-cli.rst index f46c1f671c..2a35601000 100644 --- a/docs/usage-cli.rst +++ b/docs/usage-cli.rst @@ -21,6 +21,8 @@ Using the Command Line Interface This document is meant to give an overview of all common tasks while using the CLI. +.. note:: + For more information on CLI commands, see :doc:`cli-ref` Set Up connection to a remote Airflow instance ---------------------------------------------- @@ -68,3 +70,8 @@ If you’re using ``zsh``, add the following to your ``.zshrc``: autoload bashcompinit bashcompinit eval "$(register-python-argcomplete airflow)" + +Creating a Connection +--------------------- + +For more information on creating connection using CLI, see :ref:`connection/cli`