* Use first start date when running a rescheduled task, this also fixes
the duration. Use actual start date to record reschedule requests.
* Simplify gantt view code now that start date and duration are correct
in `task_instance` table
In ExternalTaskSensor, it may be good to provide
an option to cease waiting immediately if the external
DAG/task specified doesn't exist.
To provide an argument "check_existence". Set to True to check
if the external DAG/task exists, and immediately cease waiting
if the external DAG/task does not exist.
The default value is set to False (no check or ceasing
will happen) so it will not affect any existing DAGs or
current user expectation.
In ExternalTaskSensor, we can specify `allowed_states`.
This commit adds validation for it, so that users will not
specify any invalid state.
This change works no matter the sensor waits for a DAG or
a specific task.
* [AIRFLOW-3589] Visualize reschedule state in all views
* Add explicit `UP_FOR_RESCHEDULE` state
* Add legend and CSS to views
* [AIRFLOW-3589] Visualize reschedule state in all views
* Use set or tuple instad of list
* Use `with` statement for session handling
* [AIRFLOW-2747] Explicit re-schedule of sensors
Add `mode` property to sensors. If set to `reschedule` an
AirflowRescheduleException is raised instead of sleeping which sets
the task back to state `NONE`. Reschedules are recorded in new
`task_schedule` table and visualized in the Gantt view. New TI
dependency checks if a sensor task is ready to be re-scheduled.
* Reformat sqlalchemy imports
* Make `_handle_reschedule` private
* Remove print
* Add comment
* Add comment
* Don't record reschule request in test mode
When bucket_name is provided, and bucket_key is also provided as a full
S3:// url, the full_url obtained eventually will be wrong. It will be
like 's3://bucket_name/s3://bucket_name/object_key'.
This should be avoided by adding checking and raise exception in such
case.
Closes#3762 from XD-DENG/patch-6
Soft-fail sensor failure causes skip of all
downstream tasks. It also enables ability to set
up non-blocking and soft-fail sensors in the same
way as for regular sensors.
Closes#3509 from artem-kirillov/AIRFLOW-1786
Moving the sensors to seperate files increases
readability of the
code. Also this reduces the code in the big
core.py file.
Closes#2875 from Fokko/AIRFLOW-1889-move-sensors-
to-separate-package
Kamil Breguła
Stefan Seelmann
Xiaodong
Tao Feng
Marcin Szymański
Fokko Driesprong
dima-asana
Joshua Carp
Kaxil Naik
Artem Kirillov
Kengo Seki
Giovanni Lanzani
Bolke de Bruin
Fokko Driesprong