This commit is contained in:
Maxime Beauchemin 2015-04-29 23:06:52 -07:00
Родитель 2a7e0adf11
Коммит c926783bca
2 изменённых файлов: 16 добавлений и 16 удалений

Просмотреть файл

@ -8,8 +8,8 @@ from datetime import datetime
default_args = { default_args = {
'owner': 'airflow', 'owner': 'airflow',
'depends_on_past': True, 'depends_on_past': False,
'start_date': datetime(2015, 01, 23), 'start_date': datetime(2015, 1, 1),
'email': ['airflow@airflow.com'], 'email': ['airflow@airflow.com'],
'email_on_failure': False, 'email_on_failure': False,
'email_on_retry': False, 'email_on_retry': False,

Просмотреть файл

@ -24,8 +24,8 @@ complicated, a line by line explanation follows below.
default_args = { default_args = {
'owner': 'airflow', 'owner': 'airflow',
'depends_on_past': True, 'depends_on_past': False,
'start_date': datetime(2015, 01, 23), 'start_date': datetime(2015, 1, 1),
'email': ['airflow@airflow.com'], 'email': ['airflow@airflow.com'],
'email_on_failure': False, 'email_on_failure': False,
'email_on_retry': False, 'email_on_retry': False,
@ -40,7 +40,6 @@ complicated, a line by line explanation follows below.
t2 = BashOperator( t2 = BashOperator(
task_id='sleep', task_id='sleep',
depends_on_past=False,
bash_command='sleep 5', bash_command='sleep 5',
dag=dag) dag=dag)
@ -54,7 +53,6 @@ complicated, a line by line explanation follows below.
t3 = BashOperator( t3 = BashOperator(
task_id='templated', task_id='templated',
depends_on_past=False,
bash_command=templated_command, bash_command=templated_command,
params={'my_param': 'Paramater I passed in'}, params={'my_param': 'Paramater I passed in'},
dag=dag) dag=dag)
@ -90,8 +88,8 @@ of default parameters that we can use when creating tasks.
args = { args = {
'owner': 'airflow', 'owner': 'airflow',
'depends_on_past': True, 'depends_on_past': False,
'start_date': datetime(2015, 01, 23), 'start_date': datetime(2015, 1, 1),
'email': ['airflow@airflow.com',], 'email': ['airflow@airflow.com',],
'email_on_failure': True, 'email_on_failure': True,
'email_on_retry': True, 'email_on_retry': True,
@ -130,15 +128,15 @@ argument ``task_id`` acts as a unique identifier for the task.
t2 = BashOperator( t2 = BashOperator(
task_id='sleep', task_id='sleep',
depends_on_past=False, email_on_failure=False,
bash_command='sleep 5', bash_command='sleep 5',
dag=dag) dag=dag)
Notice how we pass a mix of operator specific arguments (``bash_command``) and Notice how we pass a mix of operator specific arguments (``bash_command``) and
an argument common to all operators (``depends_on_past``) inherited an argument common to all operators (``email_on_failure``) inherited
from BaseOperator to the operators constructor. This is simpler than from BaseOperator to the operators constructor. This is simpler than
passing every argument for every constructor call. Also, notice that in passing every argument for every constructor call. Also, notice that in
the second call we override ``depends_on_past`` parameter with ``False``. the second call we override ``email_on_failure`` parameter with ``False``.
The precendence rules for operator is: The precendence rules for operator is:
@ -174,7 +172,6 @@ curly brackets, and point to the most common template variable: ``{{ ds }}``.
t3 = BashOperator( t3 = BashOperator(
task_id='templated', task_id='templated',
depends_on_past=False,
bash_command=templated_command, bash_command=templated_command,
params={'my_param': 'Paramater I passed in'}, params={'my_param': 'Paramater I passed in'},
dag=dag) dag=dag)
@ -244,8 +241,8 @@ something like this:
default_args = { default_args = {
'owner': 'airflow', 'owner': 'airflow',
'depends_on_past': True, 'depends_on_past': False,
'start_date': datetime(2015, 01, 23), 'start_date': datetime(2015, 1, 1),
'email': ['airflow@airflow.com'], 'email': ['airflow@airflow.com'],
'email_on_failure': False, 'email_on_failure': False,
'email_on_retry': False, 'email_on_retry': False,
@ -260,7 +257,7 @@ something like this:
t2 = BashOperator( t2 = BashOperator(
task_id='sleep', task_id='sleep',
depends_on_past=False, email_on_failure=False,
bash_command='sleep 5', bash_command='sleep 5',
dag=dag) dag=dag)
@ -274,7 +271,6 @@ something like this:
t3 = BashOperator( t3 = BashOperator(
task_id='templated', task_id='templated',
depends_on_past=False,
bash_command=templated_command, bash_command=templated_command,
params={'my_param': 'Paramater I passed in'}, params={'my_param': 'Paramater I passed in'},
dag=dag) dag=dag)
@ -355,6 +351,10 @@ database to record status. If you do have a webserver up, you'll be able to
track the progress. ``airflow webserver`` will start a web server if you track the progress. ``airflow webserver`` will start a web server if you
are interested in tracking the progress visually as you backfill progresses. are interested in tracking the progress visually as you backfill progresses.
Note that if you use ``depend_on_past=True``, individual task instances
depends the success of the preceding task instance, except for the
start_date specified itself, for which this dependency is disregarded.
.. code-block:: bash .. code-block:: bash
# optional, start a web server in debug mode in the background # optional, start a web server in debug mode in the background