- changes the order of arguments for `has_mail_attachment`, `retrieve_mail_attachments` and `download_mail_attachments`
- add `get_conn` function
- refactor code
- fix pylint issues
- add imap_mail_filter arg to ImapAttachmentToS3Operator
- add mail_filter arg to ImapAttachmentSensor
- remove superfluous tests
- changes the order of arguments in the sensors + operators __init__
Set autodetect default value from false to be true to avoid breaking downstream
services using GoogleCloudStorageToBigQueryOperator but not aware of the newly
added autodetect field.
This is to fix the current regression introduced by #3880
The recent commit 3724c2aa to master introduced a __init__.py file in
the project root folder, which basically breaks all imports in local
development (`pip install -e .`) as it turns the project root into a
package.
[ci skip]
For large DAGs, iterating over template fields to find template files can be time intensive.
Save this time for tasks that do not specify a template file extension.
* [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator)
Add support for running Docker containers via Docker Swarm
which allows the task to run on any machine (node) which
is a part of your Swarm cluster
More details: https://issues.apache.org/jira/browse/AIRFLOW-4843
Built with <3 at Agoda!
`GCPTransferServiceHook.wait_for_transfer_job` defeaults its `timeout`
parameter to 60 and assumes it is an integer or at least comparable to
one. This is a problem as some of the built-in operators that use it
like `S3ToGoogleCloudStorageTransferOperator` and
`GoogleCloudStorageToGoogleCloudStorageTransferOperator` default their
`timeout` param to `None`, and when they call this method with their
default value, it causes an error. Fix this by allowing
`wait_for_transfer_job` to accept a timeout of `None` and fill in
appropriate defaults. This also adds functionality to allow it to take
a `timedelta` instead of an integer, allows seconds to be any real, as
there is really no need for them to actually be an integer, and fixes
the counting of time for determining timeout to be a bit more accurate.
The GCS Transfer Service REST API requires that a schedule be set, even for
one-time immediate runs. This adds code to
`S3ToGoogleCloudStorageTransferOperator` and
`GoogleCloudStorageToGoogleCloudStorageTransferOperator` to set a default
one-time immediate run schedule when no `schedule` argument is passed.
Osmar Coronel
bolkedebruin
Bas Harenslak
Jarek Potiuk
Omkar Desai
Kamil Breguła
Tomek
Ry Walker
Rudison Lacerda
Chao-Han Tsai
Cooper Gillan
Qingping Hou
Felix Uellendall
bingqinzhou
Daniel Imberman
Jarek Potiuk
Kevin Yang
Kaxil Naik
Ash Berlin-Taylor
Daniel Frank
damon09273@gmail.com
Ryan Yuan
wmorris75
Akshesh Doshi
Maulik Soneji
Joel Croteau