incubator-airflow/airflow/task
QP Hou f1d4f54b34
Fix race conditions in task callback invocations (#10917)
This race condition resulted in task success and failure callbacks being
called more than once. Here is the order of events that could lead to
this issue:

* task started running within process 2
* (process 1) local_task_job checked for task return code, returns None
* (process 2) task exited with failure state, task state updated as failed in DB
* (process 2) task failure callback invoked through taskinstance.handle_failure method
* (process 1) local_task_job heartbeat noticed task state set to
  failure, mistoken it as state bing updated externally, also invoked task
  failure callback

To avoid this race condition, we need to make sure task callbacks are
only invoked within a single process.
2021-01-18 23:39:41 +00:00
..
task_runner Fix race conditions in task callback invocations (#10917) 2021-01-18 23:39:41 +00:00
__init__.py [AIRFLOW-6714] Remove magic comments about UTF-8 (#7338) 2020-02-02 22:18:19 +01:00